[webkit-gtk] add data chunks to received-data signal callback ?

[Jérémy Lal]

un-top-posting your reply...

Le mercredi 08 octobre 2014 à 12:45 -0700, Jim Nelson a écrit :
> On Wed, Oct 8, 2014 at 12:14 PM, Jérémy Lal <kapouer at melix.org> wrote:
> > Le mercredi 08 octobre 2014 à 16:04 -0300, Gustavo Noronha Silva a
> > écrit :
> >>  On Dom, 2014-08-31 at 23:31 +0200, Jérémy Lal wrote:
> >>  > webkit_web_resource_get_data is used to get a response data, but 
> >> in case
> >>  > of big files, video for example, it can easily eats a lot of 
> >> memory.
> >>  > The "received-data" signal would be a very nice place to receive 
> >> the
> >>  > data chunks as they are downloaded. Is this possible or does it 
> >> goes
> >>  > against webkit architecture ?
> >> 
> >>  Hmm. I think it could be. Perhaps what we need is a way of getting 
> >> an
> >>  InputStream out of WebResource, so you could use the regular read
> >>  operations to fetch the contents, how about that?
> > 
> > Looks good to me... i'll happily test it to stream the data to my
> > node-webkitgtk module :)
>
> Shotwell ran into this problem with Soup and multipart uploads some 
> time back.  Dan Winship pointed out that memory-mapped I/O would 
> solve that problem, and I suspect it would work here too.

Yes, that answers part of the problem, (if it works as expected), but
being able to access the response stream would enable a lot of other
applications...
Processing data as soon as it is received (not having to wait for the
whole file to be done loading) is a huge advantage.

Also i fear bindings to webkit in other languages might have to make a
copy at some point of the data, and it might as well do that on smaller
chunks than on one big file.

Jérémy.