[webkit-help] Memory usage is up 60%

[Benjamin Poulain]

On 5/7/14, 12:36 PM, Luc R. wrote:
> Thanks Benjamin. Do I understand right that the MemoryPressureHandler 
> is an additional mechanism to reduce memory on top of other things, 
> but is not the root cause of memory increase at first place?
Yes, MemoryPressureHandler is one of our tools for managing memory, not 
the cause.

The way WebKit runs nowadays is: use as much memory as needed to improve 
the runtime performance (while being reasonable of course). When the 
system needs the memory back, MemoryPressureHandler clears the major 
caches and return the memory to the system.

It is a common pattern from OS X: if there is spare memory available, 
use if for runtime performance. When memory becomes scarce, the 
operating system reclaim memory from the processes that do not require 
high performance at this time.
> An extra  information on memory pattern:
> For the new code, amount of committed memory is actually less for 
> about 2x on average (say 150M vs 300M per process), but private memory 
> is higher say 100M vs 60M per process, as well as is the working set. 
> The scenario is a continuous processing a URL list until process 
> crashing. My interpretation that somehow processes burn more physical 
> memory, instead of letting it page out to disk. Does it make sense?
>
> This looks like a dramatic change. Are there any major changes in 
> memory management in the last 2 years? Are there any sensitive 
> settings which can be tuned up?
WebKit is an extremely active project, there are several hundred changes 
every day. Some subsystems have been entirely rewritten in the last two 
years. Without more information about which subsystem uses more memory, 
it is impossible for me to guess what is happening.

Regarding the settings, you should at least disable the page cache since 
it should be useless for your use case.

If you send the list of objects that are taking more memory, I may be 
able to give more information.

Benjamin

> On Mon, May 5, 2014 at 6:38 PM, Benjamin Poulain <benjamin at webkit.org 
> <mailto:benjamin at webkit.org>> wrote:
>
>     On 5/5/14, 6:31 PM, Luc R. wrote:
>
>         Question on memory usage.
>
>         I am using WinCairo port, and I have recently updated the
>         codebase from
>         about 2 years old one. I am running an automation with multiple
>         instances of WebKit running in parallel, and each instance is
>         processing
>         a list of URLs (until crashes, then it's restarted).
>
>         I noticed that memory usage on the new codebase is about 60%
>         higher! I
>         don't see substantial differences in the output (which is
>         pretty much
>         outerHTML) and processing speed is about the same.
>
>         Does anyone have ideas, which feature could have added such a
>         significant increase of memory usage? I would be interested to
>         look into
>         details of how to reduce that back to normal.
>
>
>     WebKit is becoming a lot better at using memory efficiently to
>     improve the runtime performance. It could simply be better use of
>     the ram.
>
>     You should try to wiring the MemoryPressureHandler with your
>     system, and see if that makes a difference.
>
>     Benjamin
>     _______________________________________________
>     webkit-help mailing list
>     webkit-help at lists.webkit.org <mailto:webkit-help at lists.webkit.org>
>     https://lists.webkit.org/mailman/listinfo/webkit-help
>
>
>
>
> -- 
> -Luc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-help/attachments/20140508/cf01b5b1/attachment.html>