[webkit-dev] Curl resourcehandle leaks in Linux/Gtk port

[zaheer ahmad]

hi mike,
The ncurl port is not yet in the official builds. meanwhile how do you
suggest to fix this in the current baseline.

one of the changes that does help is to periodically cleanup the multihandle
when the running job count drops to 0 and recreate on the next request (this
is just a temporary fix till we find the real issue in curl)

i have few comments on the ncurl patch:
https://bugs.webkit.org/show_bug.cgi?id=17972
- is it better than the current timer driven behavior, since the glib main
loop polls the fds faster if its free- but thats seems to be a small gain
since the timeout is very small
- in the current implementation curl_multi_perform may block if theres lots
of data queued up on multiple handles, but that can be easily mitigated by
returning frequently from curl_multi_perform
- what about doing select in separate thread as glibcurl does. i think this
is safe as perform happens in the main thread.

thanks,
Zaheer

On Wed, Sep 10, 2008 at 8:40 PM, Mike Emmel <mike.emmel at gmail.com> wrote:

> Look I had to change to one multi handle per handle basically just and
> asynchronous handle to get everything to clean up.
> Its a significant refactoring and better design.
>
> What it points to on the curl side is the need for and asynchronous
> simple handle.
>
> Also polling was removed as much as possible curl does not send decent
> time outs if it has real work
> to perform so this is still a issue. However open file handles are
> handled in the event loop select.
>
> Curl needs to be extended to have the concept of a work request and a
> longer term watch timeout.
>
> So in my opinion the issues are fixed at least to the extent possible
> without help from the curl team.
>
> On Wed, Sep 10, 2008 at 7:53 AM, zaheer ahmad <zaheer.mot at gmail.com>
> wrote:
> > hi,
> > The fix only helps little as we see the bigger leaks in curl. feedback
> from
> > curl experts suggests that this design is correct.. let me know if you
> are
> > aware of this issue
> >
> > == here's the mail snapshot.
> > we are seeing big leaks in curl (Curl_connect - 600-800k and Curl_open -
> > ~200k) when we browse through as little as few websites. This values
> > keep increasing as we browse more sites.
> >
> > heres the high level logic of webkit=curl interaction
> > 1- create a multi handle at the start of program
> > 2- keep creating easy handles for each request
> > 3- when request is done remove it from multi handle and clean up the
> > handle
> > 4- multi handle is never released (stays till the end of program)
> >
> > This design assumes that multi handle has a bounded memory usage as we
> > keep adding and removing easy handles, but that seems to be not true
> > with the leaks.
> > ==
> >
> >
> >>> Now I just started to use valgrind to find other memory leaks, so this
> > and other issues should be hopefully fixed soon.
> >
> > these are not traditional memory leaks, you are holding on to things
> longer
> > than you should, so they are more functional leaks. Does valgrind help in
> > that too?
> >
> > thanks,
> > Zaheer
> >
> >
> > On Wed, Sep 10, 2008 at 8:02 PM, Marco Barisione
> > <marco.barisione at collabora.co.uk> wrote:
> >>
> >> Il giorno mer, 10/09/2008 alle 07.22 -0700, Mike Emmel ha scritto:
> >> > This leak is fixed in the ncurl port.
> >>
> >> Is it possible to backport the fix?
> >>
> >> --
> >> Marco Barisione
> >>
> >> _______________________________________________
> >> webkit-dev mailing list
> >> webkit-dev at lists.webkit.org
> >> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> >
> >
> > _______________________________________________
> > webkit-dev mailing list
> > webkit-dev at lists.webkit.org
> > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.webkit.org/pipermail/webkit-dev/attachments/20080911/0e0fb81f/attachment.html