[webkit-dev] Python

[Michael Catanzaro]

Hi,

As you're probably already aware, in Arch Linux /usr/bin/python has 
been a symlink to /usr/bin/python3 for a long time now. In practice 
this means that Arch users are not going to be able to use basically 
any of our Python scripts, since our scripts use the shebang 
#!/usr/bin/env python but expect it to be python2. This has not really 
been a problem until now because none of Igalia's developers use Arch, 
and occasional contributions rarely need to use the Python scripts. But 
our scripts have been broken this whole time, since the Python 
maintainers have decided the only reasonable way to avoid this problem 
is for scripts to explicitly specify either python2 or python3, and we 
have not been doing that. To be clear: the problem is that 
#!/usr/bin/env python is *python3* on some systems, but *python2* on 
others. WebKit scripts incorrectly assume it is always python2. I say 
"incorrectly" because the Python folks have ratified some bizarre 
standard for how to handle this, PEP 394 [1], which states: "in 
preparation for an eventual change in the default version of Python, 
Python 2 only scripts should either be updated to be source compatible 
with Python 3 or else to use python2 in the shebang line." Lovely.

Now Fedora is planning to switch /usr/bin/python to link to 
/usr/bin/python3 instead of /usr/bin/python2 [2]. Not for another three 
years, but many of us (hi! :) use Fedora, so we're going to have to fix 
our scripts by then. This could be really easy, or it could be really 
hard.

The really easy solution is to just replace #!/usr/bin/env python with 
#!/usr/bin/env python2 everywhere. This will work on all Linux distros 
newer than Red Hat Enterprise Linux 6 or thereabouts, which we 
definitely do not care about anymore. It doesn't work in very old 
distros because they did not provide /usr/bin/python2, only 
/usr/bin/python and very specific links like /usr/bin/python2.6. That 
doesn't matter because we don't care about those old systems anymore, 
but the problem is that last I checked, macOS did not provide a python2 
binary either. I am hoping that has changed in the past few years. Has 
it? If so, I can upload a patch to do this simple find/replace and we 
can all move on. If this is the case, yay! Stop reading now! Let me 
know. We should do that!

But if not, we need to decide what to do among bad options:

 * We could replace all our Python scripts that have shebangs with .in 
files, and perform configure replacements at CMake/XCode configure time 
to generate the scripts with the right python executable path. This 
would mean the scripts would be generated into the WebKit build 
directory instead of the source directory, so it will break those of us 
who have added Tools/Scripts to our PATH. It's not clear how they would 
find the right resource paths in the source directory, especially if 
using a build directory other than WebKit/WebKitBuild. I'm not even 
sure if XCode can do configure replacements. This would be a mess.
 * We could require Mac developers to install a python2 symlink 
somewhere in PATH. Not really desirable, but possibly tolerable?
 * We could port all our Python scripts to python3 and use 
#!/usr/bin/env python3, which will surely be unambiguous. But my 
understanding is that macOS does not have python3, so it will need to 
be installed locally. I guess that would be tolerable? Anyway, the main 
disadvantage here is we'd have to make all our scripts work with 
python3. If WebKit was a Linux-only project, we should definitely do 
that anyway, since major Linux distros no longer ship python2 by 
default: it's something we install extra now. But if macOS is sticking 
with python2, it would probably be undesirable to port our scripts to 
python3.
 * We could make all our Python scripts compatible with both python2 
and python3, and maintain them this way forever. I really hope we can 
avoid this option, since we'd be breaking these scripts all the time as 
we would all surely not test one configuration or the other.

Any other ideas?

Michael

[1] https://www.python.org/dev/peps/pep-0394/
[2] https://lwn.net/SubscriberLink/729366/1d3cd2dbaea7070b/