[Webkit-unassigned] [Bug 46453] [NRWT] Put the http and websocket tests to end of the test list

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=46453





--- Comment #13 from Dirk Pranke <dpranke at chromium.org>  2010-09-27 17:48:27 PST ---
(In reply to comment #12)
> (In reply to comment #10)
> > Ugh. Okay, I've taken a look at both the patch posted here and the patch on GitHub. They're troubling, but then again I find anything that requires locking, especially cross-process locking, to be troubling :)
> >
> > First, assuming NRWT is fully exploiting the parallelism of the machine -- which it should be doing, ignoring the currently open bugs that are throttling it  -- a single NRWT run should pin the machine and running multiple in parallel will at best slow things down and more likely introduce more flakiness and weird timeouts and failures.
> >
> > So, I need to step back and ask, why do we want to run more than one NRWT in parallel? Adam, Eric - how do you see what I wrote above interacting with the commit queue? Gabor, are there other reasons you want to run these things in parallel?
> >
> 
> We have some multicore servers with several build and try bots, so it would be useful if we could run more than one layout test per machine.
> 


This may seem like a dumb question, but why do you run more than one bot per machine? Are the bots not always busy at the same time, or are the machines not pinned? It seems like, if the infrastructure is working properly, trying to run multiple bots on the same machine should just make things slower.

Of course, if you can only afford one machine, things are different, but that doesn't sound like the case from what you say?

> 
> I can make an option check and put the http tests to first when we don't need locking.
>

As I described above, I'm not sure that moving when the tests are executed will actually improve things either way; I honestly don't know what will perform best, either when there's only a single NRWT running, or when we're running more than one at the same time. 

I would like to see some benchmark numbers or test results to support whatever changes we make before we land anything.

Please don't take the above as a sign that I don't want to make changes - I'm happy to do anything that makes developers more productive, reduces cycle times, or allows you to test things you can't do now. I just want to make sure we're doing the right things and do see real benefit, rather than just changing things randomly in the hopes that it'll make things better.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.