[webkit-dev] Going forward on automating W3C test suite import

[youenn fablet]

Hi all,

I continued the implementation of some scripts to help importing
regularly the whole W3C test suite as layout tests
(http://wkb.ug/134767).

The process is currently as follows:
- Import the whole test suite as a WebKitBuild sub-folder
- Run rwt on this sub-folder
- Compute conformance for each test (for example
http://youennf.github.io/w3c-reports/12092014/w3c_conformance_results.html)
- Generate TestExpectations based on conformance results (skip all
tests except the ones with at least one successful assertion)
- Update LayoutTests folder (test files, expected.txt and TestExpectations)

Using that approach, from 3000 tests, about 2000 could be run.
Running these tests takes about 10 minutes to run on my machine. The
remaining skipped tests take much more time to run.

To move on with this, I have some questions:
1. Is it fine to run rwt within a WebKitBuild sub-folder, tests
provided as absolute paths?
2. Is it ok to have a separate TestExpectationsW3C for W3C tests, one
for each port typically?
3. Is it ok to run all tests through wpt http server, thus using a
single test runner?
4. How to best ease bumping of the W3C test suite version? Should the
version be the same version for all ports, requiring some coordination
between ports when bumping? Should there be one version for each port?

Any suggestion?

Regards,
     youenn