[webkit-qt] [Qt-interest] Multiple page rendering using webkit
tarandeep at gmail.com
Fri Apr 15 15:09:15 PDT 2011
The pages to be rendered are not on the disk and need to be fetched over
Based on your replies, I am gonna try this-
Have multiple processes but each process should have couple of webpages, put
NAM in threads so as to improve network IO.
With multiple processes, I might run out of memory as I have seen memory
foot print of qt+webkit process keeps increasing as I keep rendering pages
over a long period of time, so I might need to do something about it - like
restart the process after some time.
On Fri, Apr 15, 2011 at 1:55 PM, Jason H <scorp1us at yahoo.com> wrote:
> Yes, I'd probably use renderWidget() to get a pixmap and find the fastest
> way to
> get that to the master app. (local socket?)
> usability becomes an issue for anything more than a thumb nail. You'd need
> a way
> to map the mouse/keyboard events back through the socket and make the
> corresponding adjustments in the corresponding webkit process.
> ----- Original Message ----
> From: Arnold Krille <arnold at arnoldarts.de>
> To: qt-interest at trolltech.com
> Sent: Fri, April 15, 2011 4:46:41 PM
> Subject: Re: [Qt-interest] Multiple page rendering using webkit
> On Friday 15 April 2011 21:33:07 Tarandeep Singh wrote:
> > I need to render a large number of pages and obviously need do this as
> > efficiently as possible.
> > Since GUI rendering needs to be done on the main thread, so I can't
> > multiple QWebPage threads.
> > So I am thinking of these 2 solutions-
> > 1) Create one process and have N QWebPage instances so that N pages can
> > rendered simultaneously. Here assumption is QT+Webkit behind the scenes
> > create threads to do parallel IO etc.
> > 2) Create N processes where each process handles 1 page request at a
> > time.
> > Can someone please advice which solution should work better or is there
> > another way to achieve this.
> > I have 16 cores and 32G of RAM.
> I assume you have the to-be-rendered webpages on disk already as html and
> "only" want the rendered output in pdf/svg/png or whatever. (If not I think
> you can easily do with three to four webpages which in turns load the files
> from network and render and save to disk, net-io will be the slowest
> As all gui-stuff has to run in the thread that instantiated QApplication,
> can really only use one thread for rendering. But who says that you can't
> multiple processes in parallel?
> One process, maybe even the master of your app, loads the html from disk
> a queue which is in shared memory and then starts as many render-processes
> you like. These render processes each take a pages html from the queue and
> render it and either put the rendered data back onto another shared queue
> save directly to files on disk.
> If you want to leave room for other processes to run (like the desktop),
> should run like min(1, idealThreadCount()/2) render processes.
> Hope that helps,
> Qt-interest mailing list
> Qt-interest at qt.nokia.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the webkit-qt