[webkit-dev] Deployment of new EWS Non-Unified builder

Alexey Proskuryakov ap at webkit.org
Wed Jun 1 15:26:50 PDT 2022


I'm not sure if we have a consensus on whether it is a project goal to keep non-unified build working at all times. As discussed last year, setting up post-commit bots is a pre-requisite for having EWS, so this part is resolved. But proactively maintaining the non-unified build is strictly more work than fixing it on demand. So the immediate response continues to be that we shouldn't be doing more work when we can do less.

You mention that embedders who build with non-default flags are more likely to hit issues. Building with non-default flags would be resulting in missing includes for non-unified builds too, do you have an estimate of how much this problem grows due to unified builds? How do we decide if everyone is responsible for the convenience of downstream embedders?

It sounds like none of actively supported ports builds non-unified by default, which I understand from the fact that a special post-commit queue had to be set up for that.

Perhaps part of the argument is that even though proactively maintaining the non-unified build is more work, this work is somehow easier than fixing builds on demand. If so, can you elaborate on why this is the case?

- Alexey

21 мая 2022 г., в 12:10 AM, Kirsling, Ross via webkit-dev <webkit-dev at lists.webkit.org <mailto:webkit-dev at lists.webkit.org> > написал(а):

This is wonderful news—thanks Diego!

From: dpino via webkit-dev <webkit-dev at lists.webkit.org <mailto:webkit-dev at lists.webkit.org> >
Sent: Friday, May 20, 2022 9:17:56 PM
To: webkit-dev at lists.webkit.org <mailto:webkit-dev at lists.webkit.org> <webkit-dev at lists.webkit.org <mailto:webkit-dev at lists.webkit.org> >
Subject: [webkit-dev] Deployment of new EWS Non-Unified builder

Last year we started a thread to discuss the possibility of deploying a
new EWS bot that builds WebKit with Non-Unified sources [1]. This thread

explains the technical reasons why a non-unified build bot is desirable.

If you're aware of the problems introduced by unified sources
compilation, skip the next paragraph.

Unified sources compilation consists of merging several source files
together into one big file to improve building times. Usually the same
sources files are grouped together, but compilation flags or downstream
changes can create different source file groups. When that happens, you
may hit a build error. Since unified sources group files together,
missing headers in one file can be satisfied by another file in the same

group. What's happening in WebKit development currently is that source
code that breaks non-unified compilation frequently lands without even
being noticed. The breaks are usually related to missing headers.
Embedders that need to support WebKit builds with different flags or
maintain downstream changes are more likely to hit these compilation

Last year's attempt to deploy an EWS Non-Unified builder ended up in a
deployment of a post-commit bot plus two workers running in the UAT. It
was actually hard to get the Non-Unified builder working [2], something
that we achieved at the beginning of this year (kudos to Adrián and

Lauro). Since then we have been maintaining the post-commit bot very
closely. There are periods of time where the bot has remained green for
a long period of time.

But lately maintaining the bot green has become harder and harder,
specially due to the big refactorizations that have happened in WebKit
source code lately. The bot very rarely stays green longer than 2 or 3
days without close maintenance. We believe that we all should share the
responsibility of keeping the non-unified build working, and therefore
we want to proceed with the deployment of a EWS Non-Unified builder.

Initially, we'll provide two workers on a modern host with 8 cores
assigned each. We have found this setup faster for most patches than
many of the existing EWS queues, as it only runs a build in non-unified
mode, without test or upload steps. If this were not fast enough, we
would consider increasing the number of workers in the queue. We set up
a page[3] with a rough comparison to the regular (unified) bot and the
build times are pretty similar.


https://urldefense.com/v3/__https://www.mail-archive.com/webkit-dev@lists.webkit.org/msg30077.html__;!!JmoZiZGBv3RvKRSx!6YGOGRuK21OCDj8_MKm9gtNOYvBg9Ry10BMna8HXDHWUcUpVeDPBeA4QBT-nC3xj3j8wgt_mjpqbrrOsbTOSWjtVsXQ$ <https://urldefense.com/v3/__https://www.mail-archive.com/webkit-dev@lists.webkit.org/msg30077.html__;!!JmoZiZGBv3RvKRSx!6YGOGRuK21OCDj8_MKm9gtNOYvBg9Ry10BMna8HXDHWUcUpVeDPBeA4QBT-nC3xj3j8wgt_mjpqbrrOsbTOSWjtVsXQ$> 
[2] https://urldefense.com/v3/__https://bugs.webkit.org/show_bug.cgi?id=226088__;!!JmoZiZGBv3RvKRSx!6YGOGRuK21OCDj8_MKm9gtNOYvBg9Ry10BMna8HXDHWUcUpVeDPBeA4QBT-nC3xj3j8wgt_mjpqbrrOsbTOSqxngQkY$ <https://urldefense.com/v3/__https://bugs.webkit.org/show_bug.cgi?id=226088__;!!JmoZiZGBv3RvKRSx!6YGOGRuK21OCDj8_MKm9gtNOYvBg9Ry10BMna8HXDHWUcUpVeDPBeA4QBT-nC3xj3j8wgt_mjpqbrrOsbTOSqxngQkY$> 
[3] https://urldefense.com/v3/__https://people.igalia.com/lmoura/webkit-bots-dashboard/unified.html__;!!JmoZiZGBv3RvKRSx!6YGOGRuK21OCDj8_MKm9gtNOYvBg9Ry10BMna8HXDHWUcUpVeDPBeA4QBT-nC3xj3j8wgt_mjpqbrrOsbTOSPFziSrI$ <https://urldefense.com/v3/__https://people.igalia.com/lmoura/webkit-bots-dashboard/unified.html__;!!JmoZiZGBv3RvKRSx!6YGOGRuK21OCDj8_MKm9gtNOYvBg9Ry10BMna8HXDHWUcUpVeDPBeA4QBT-nC3xj3j8wgt_mjpqbrrOsbTOSPFziSrI$> 
webkit-dev mailing list
webkit-dev at lists.webkit.org <mailto:webkit-dev at lists.webkit.org> 
https://urldefense.com/v3/__https://lists.webkit.org/mailman/listinfo/webkit-dev__;!!JmoZiZGBv3RvKRSx!6YGOGRuK21OCDj8_MKm9gtNOYvBg9Ry10BMna8HXDHWUcUpVeDPBeA4QBT-nC3xj3j8wgt_mjpqbrrOsbTOS8gKEMAU$ <https://urldefense.com/v3/__https://lists.webkit.org/mailman/listinfo/webkit-dev__;!!JmoZiZGBv3RvKRSx!6YGOGRuK21OCDj8_MKm9gtNOYvBg9Ry10BMna8HXDHWUcUpVeDPBeA4QBT-nC3xj3j8wgt_mjpqbrrOsbTOS8gKEMAU$> 
webkit-dev mailing list
webkit-dev at lists.webkit.org <mailto:webkit-dev at lists.webkit.org> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20220601/7589a547/attachment.htm>

More information about the webkit-dev mailing list