[webkit-dev] Best practices for landing new/changed layout test expectations?

[Ryosuke Niwa]

On Wed, Mar 13, 2013 at 3:36 PM, Eric Seidel <eric at webkit.org> wrote:

> Given the history of that page, I'm not sure it truly reflects the
> consensus of the project:
> https://trac.webkit.org/wiki/CreatingLayoutTests?action=history
>
> > Regardless of whether you are making a platform independent change or
> dependent change, it is your responsibility to ensure that the change does
> not negatively affect any other ports / platforms. This includes updating
> platform specific results or contacting a port maintainer to do this for
> you (for contacts, see: http://trac.webkit.org/wiki/WebKit%20Team).
>
> I'm not sure I see anyone following this these days.  The EWS system
> and sherriffiing cultures seem to have largely replaced
> bot-monitoring.  We also don't really have a system of "core ports",
> thus "all ports" seems a bit broad here.
>

Yeah, we probably need to loosen that rule up a little to reflect the
current state of the world.

It’s probably better to say something like “a patch author should be
available on IRC and Bugzilla to answer questions bot maintainers have with
respect to tests added or tests that have to be rebaselined”.

> Once your change is landed, there are some steps required to verify the
> change. Note: These steps apply for all changes. If you are making a
> platform dependent change, you should expect that additional results will
> be required for each platform but the actions are the same.
>
> I'm not sure it makes any sense to ask contributors to update other
> ports.  In many cases, it's not possible for contributors to even
> build other ports.


I think the point was that each contributor can grab results from
build.webkit.org but I don’t think it’s really practical given the number
of build bots we have today.

We clearly need some sort of automated system for
> this, but right now I believe the expectation is that maintainers of
> the various ports will add the port-specific results, and that those
> contributing new layout tests should just land their results as the
> common ones?  But I could be mistaken.
>

That sounds like a reasonable resolution at least for now.

I think the reality is that the project doesn’t really have consensus here,
> likely meaning this is a good topic for the contributors meeting.
>

Yup.  Meanwhile, it’s good to document what people do in practice so that
new contributors don’t have to figure that out themselves.

- R. Niwa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130316/0635e979/attachment.html>