[webkit-dev] WebKit support in web-platform-tests

[Philip Jägenstedt]

Thanks for your work on this, Žan!

That PR is now up at https://github.com/w3c/web-platform-tests/pull/9666 and
in it I linked to https://github.com/w3c/web-platform-tests/pull/8979,
which is a PR I updated last week to support Safari, including Technology
Review. I wish I'd payed closer attention to webkit-dev to see this thread
earlier :)

Given that safaridriver and webkitdriver aren't the same piece of code, it
makes sense to treat them as separate products in wpt run, and whether some
code is shared is wpt-internal concern.

The reason I tried to get Safari (TP) running is ultimately for wpt.fyi
and, and I don't imagine it'll be useful for WebKit CI.

Brian, I'll ask you on the PR about the WebDriver tests specifically.

On Fri, Feb 23, 2018 at 7:25 PM <zan at falconsigh.net> wrote:

> Hi,
>
> in case of no additional comments against this, I'll be opening a pull
> request to merge the web-platform-tests changes into that repository in the
> following days. The WebKit changes will follow after some further work.
>
> Cheers,
> Zan
>
> On Mon, Feb 12, 2018, at 4:06 PM, zan at falconsigh.net wrote:
> > Hi,
> >
> > the web-platform-tests repository includes tooling that enables running
> > those tests against a supported browser product. I'd like to propose
> > adding generic WebKit support there.
> >
> >
> > Current changes only assume usage of the WebDriver protocol, and the
> > WebDriver binary accepting the --port flag. Selenium executors are used
> > for test harness and reftests. Same WebDriver implementation can also be
> > tested against the WebDriver tests included in the web-platform-tests
> > directory, presuming the tests are enabled or explicitly specified.
> >
> > Only port-specific bit is the specification of capabilities that are
> > passed to the WebDriver binary, idea being that these capabilities are
> > the same as those supported by the WebDriver implementation.
> >
> > GTK is for now the only port that's supported, and it's leveraging the
> > WebDriver implementation under Source/WebDriver/ in WebKit. WPE will be
> > doing the same. Safari I suppose could use its own WebDriver
> > implementation, or perhaps even a separate product.
> >
> > Here's the current set of changes:
> >
> https://github.com/zdobersek/web-platform-tests/commit/c2ee920876ca6df7c4739feb8a6e03c77dffdb7f
> >
> >
> > The web-platform-tests suite can then be run like this for the GTK port,
> > assuming a tip-of-trunk build:
> >
> > $ /work/web-platform-tests/wpt run --webkit-port=gtk \
> >     --webdriver-binary=WebKitBuild/Release/bin/WebKitWebDriver \
> >     --binary=WebKitBuild/Release/bin/MiniBrowser \
> >     --binary-arg=--automation \
> >     --binary-arg=--javascript-can-open-windows-automatically=true \
> >     --binary-arg=--enable-xss-auditor=false \
> >     webkit /2dcontext
> >
> > This can be further wrapped into a python script and run as part of the
> > continuous integration system. These changes add a run-web-platform-
> > tests script that invokes the web-platform-tests runner tool, also
> > allowing each port to specify what tests to enable and what the expected
> > failures are:
> >
> https://github.com/Igalia/webkit/commit/df1aeeb9476c6dd220067f4fc3c6ad69a8f948ba
> >
> > Only a small subset of tests is enabled there, for prototype purposes.
> > The expected results system could also be improved to avoid each
> > expected failure having to be marked as such in separate .ini files.
> >
> >
> > But foremost, I'd like to have a consensus of sorts about how various
> > WebKit ports should be handled in the web-platform-tests repository, so
> > that the changes there can proceed -- whether it's fine to implement a
> > generic WebKit product, or whether Safari would like to be treated as a
> > separate browser[1], etc.
> >
> >
> > Regards,
> > Zan
> >
> >
> > [1] There's for instance this from a year ago (though not sure about its
> > functionality):
> > https://github.com/w3c/web-platform-tests/tree/wptrunner-safari
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20180226/7daad77f/attachment.html>