[squid-users] Performance issue -- serving from cache

[Brad House]

On 2/23/26 11:39 AM, Alex Rousskov wrote:

> On 2026-02-23 11:13, Brad House wrote:
>> On 2/18/26 11:04 AM, Brad House wrote:
>>
>>> On 2/15/26 8:00 PM, Alex Rousskov wrote:
>>>
>>>> On 2026-02-14 14:45, Brad House wrote:
>>>>> I've got a squid deployment where serving from cache can be slower 
>>>>> than an uncached download. I'm seeing speeds of around 50MB/s when 
>>>>> serving from cache, which is much slower than anticipated. Infact, 
>>>>> when hitting fast upstream servers, serving of a non-cached asset 
>>>>> is faster (even though its still hitting squid to fetch it).
>>>>>
>>>>> I'm thinking there's got to be something wrong with my squid 
>>>>> configuration, I'm currently running on Rocky Linux 10 with Squid 
>>>>> 6.10-6.
>>>>>
>>>>> The VM I'm using currently has 4 cores, 16G RAM and 100G of usable 
>>>>> space.  I used fio to measure disk performance and I got
>>>>>
>>>>>   * Random Write: 3629MiB/s (1MB block), 33.2k (4k block) IOPS
>>>>>   * Random Read: 8391MiB/s (1MB block), 43.5k (4k block) IOPS
>>>>
>>>>
>>>> What speed to you get when Squid serves the object from the memory 
>>>> cache? Do not configure a disk cache for this test to make sure 
>>>> that Squid is not reading from disk...
>>>>
>>>> I cannot help with AUFS cache_dirs, but _if_ AUFS code uses 4KByte 
>>>> I/O blocks, then 50MByte/s you are measuring is equivalent to 12K 
>>>> IOPS which is arguably not that bad for that long-neglected AUFS 
>>>> code (compared to "raw" 44K IOPS performance you measured with fio)!
>>>>
>>>> Please note that I am not trying to imply that rock cache_dir with 
>>>> a large slot-size setting would work faster than AUFS. And even if 
>>>> properly configured rock does work faster than AUFS, I would be 
>>>> really surprised if Squid can approach Traffic Server performance! 
>>>> The two projects had vastly different focus and resources.
>>>>
>>>>
>>>> HTH,
>>>>
>>>> Alex.
>>>>
>>>
>>> I'll need to wait until the weekend to give the memory-only cache a 
>>> try.  Do you have any suggested Rock settings I should try while I'm 
>>> testing things?  I'm ok wasting disk space on smaller items if it 
>>> means it can serve from cache faster in general.
>
> Unfortunately, I cannot efficiently guide you through these 
> optimizations right now, but, FWIW, I wrote a few optimization notes 
> for rock some time ago:
> https://wiki.squid-cache.org/Features/RockStore#performance-tuning
>
> I am not implying that the above notes are applicable to your case.
>
>
>> I get 350MB/s serving from memory cache which is much more reasonable. 
>
> Great!
>
>
>> Also, I tried Rock again, and inspecting the logs further it appears 
>> it isn't caching at all and always serving direct.  I'm getting these 
>> errors in the log:
>>
>> 2026/02/23 10:58:35 kid1| ERROR: /var/spool/squid/rock exception: 
>> check failed: pending->writeRequest->len <= Ipc::Mem::PageSize()
>>      exception location: IpcIoFile.cc(381) push
>>      current master transaction: master60
>
> This ERROR looks like a sign of a Squid bug. I have not investigated 
> it, but if you are setting rock slot-size to more than 32KB, please 
> try to lower that value to 32768. Testing Squid performance while 
> Squid emits those ERRORs is probably meaningless.
>
>
>> I guess I could bump the memory up in the VM to something like 64G so 
>> the hotter items can be served faster an only needs to go to disk 
>> either after restart or when an item is no longer in memory cache.
>
> If your workload hot subset fits into that larger memory cache, it may 
> be the best first optimization step. There is also memory_cache_mode, 
> but I cannot predict whether changing that would help or hurt in your 
> use case.
>
>
> Good luck,
>
> Alex.
>
Ok, yeah, I had set the rock slot-size to 64K, so I'm guessing that's 
the issue.

I'm just going to go the memory route for now ...

Thanks.

-Brad