[Webkit-unassigned] [Bug 37257] REGRESSION: run-webkit-tests does not respect argument order

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=37257


Csaba Osztrogonac <ossy at webkit.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |abecsi at webkit.org




--- Comment #3 from Csaba Osztrogonac <ossy at webkit.org>  2010-04-08 15:21:47 PST ---
(In reply to comment #2)
> As far as I know, run-webkit-tests has always sorted tests in "pathcmp" order,
> and not paid attention to the order tests were passed on the command line, back
> multiple years.

I can confirm it, "pathcmp" order is a very old thing.

But I should be glad if we make a new option, for example 
--fixed-test-order and modify sorting method, because it
can be very useful for debugging:

- at testsToRun = sort pathcmp @testsToRun; 
+ at testsToRun = sort pathcmp @testsToRun unless $fixedTestOrder;

But it is a more complex and we should think it over.
For example what do you want to do, if you call
run-webkit-tests css svg ?
1.)
  - all tests in css directory (in semi-random order)
  - all tests in svg directory (in semi-random order)
2.)
  - all tests in css directory (in alphabarical order)
  - all tests in svg directory (in alphabetical order)

As for me I prefer the 2nd one, we can do a thing like this:

for my $test (@ARGV) { 
   ...
   @tests = sort pathcmp @tests;  
   push @testsToRun, @tests; 
}

What do you think? Should we start implementation?

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.