[webkit-dev] Deployment of new EWS Non-Unified builder

[dpino]

Hi,

Last year we started a thread to discuss the possibility of deploying a
new EWS bot that builds WebKit with Non-Unified sources [1]. This thread
explains the technical reasons why a non-unified build bot is desirable.
If you're aware of the problems introduced by unified sources
compilation, skip the next paragraph.

Unified sources compilation consists of merging several source files
together into one big file to improve building times. Usually the same
sources files are grouped together, but compilation flags or downstream
changes can create different source file groups. When that happens, you
may hit a build error. Since unified sources group files together,
missing headers in one file can be satisfied by another file in the same
group. What's happening in WebKit development currently is that source
code that breaks non-unified compilation frequently lands without even
being noticed. The breaks are usually related to missing headers.
Embedders that need to support WebKit builds with different flags or
maintain downstream changes are more likely to hit these compilation
errors.

Last year's attempt to deploy an EWS Non-Unified builder ended up in a
deployment of a post-commit bot plus two workers running in the UAT. It
was actually hard to get the Non-Unified builder working [2], something
that we achieved at the beginning of this year (kudos to Adrián and
Lauro). Since then we have been maintaining the post-commit bot very
closely. There are periods of time where the bot has remained green for
a long period of time.

But lately maintaining the bot green has become harder and harder,
specially due to the big refactorizations that have happened in WebKit
source code lately. The bot very rarely stays green longer than 2 or 3
days without close maintenance. We believe that we all should share the
responsibility of keeping the non-unified build working, and therefore
we want to proceed with the deployment of a EWS Non-Unified builder.

Initially, we'll provide two workers on a modern host with 8 cores
assigned each. We have found this setup faster for most patches than
many of the existing EWS queues, as it only runs a build in non-unified
mode, without test or upload steps. If this were not fast enough, we
would consider increasing the number of workers in the queue. We set up
a page[3] with a rough comparison to the regular (unified) bot and the
build times are pretty similar.

Diego

[1]
https://www.mail-archive.com/webkit-dev@lists.webkit.org/msg30077.html
[2] https://bugs.webkit.org/show_bug.cgi?id=226088
[3] https://people.igalia.com/lmoura/webkit-bots-dashboard/unified.html