[Webkit-unassigned] [Bug 176761] New: The WebKit build system is forcing a non UTF-8 locale and breaks meson.

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=176761

            Bug ID: 176761
           Summary: The WebKit build system is forcing a non UTF-8 locale
                    and breaks meson.
           Product: WebKit
           Version: WebKit Local Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: clopez at igalia.com
                CC: aperez at igalia.com, bugs-noreply at webkitgtk.org,
                    jbicha at ubuntu.com, lforschler at apple.com,
                    mcatanzaro at igalia.com

The WPE EWS has been failing lately a lot when trying to build the JHBuild, always with this error:

Traceback (most recent call last):
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/mesonmain.py", line 338, in run
    app.generate()
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/mesonmain.py", line 181, in generate
    intr.run()
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 2728, in run
    super().run()
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 148, in run
    self.evaluate_codeblock(self.ast, start=1)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 170, in evaluate_codeblock
    raise e
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 164, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 175, in evaluate_statement
    return self.function_call(cur)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 385, in function_call
    return self.funcs[func_name](node, self.flatten(posargs), kwargs)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 47, in wrapped
    return f(self, node, args, kwargs)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 2401, in func_subdir
    self.evaluate_codeblock(codeblock)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 170, in evaluate_codeblock
    raise e
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 164, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 175, in evaluate_statement
    return self.function_call(cur)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 385, in function_call
    return self.funcs[func_name](node, self.flatten(posargs), kwargs)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreterbase.py", line 77, in wrapped
    return f(s, node_or_state, args, kwargs)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 2546, in func_configure_file
    conf.held_object)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/mesonlib.py", line 440, in do_conf_file
    replace_if_different(dst, dst_tmp)
  File "/home/ews/igalia-wpe-ews/WebKit/WebKitBuild/DependenciesWPE/Root/lib/python3.5/site-packages/mesonbuild/mesonlib.py", line 473, in replace_if_different
    if f1.read() == f2.read():
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 52: ordinal not in range(128)


Upon further investigation, it seems the Meson build system throws this error when the system locale is not an UTF8 one.

Check https://github.com/mesonbuild/meson/issues/2268

$ env -i python3 -c "import locale; print(locale.getpreferredencoding())"
ANSI_X3.4-1968

$ env -i LC_ALL=C.UTF-8 python3 -c "import locale; print(locale.getpreferredencoding())"
UTF-8

$ env -i LANG=C.UTF-8 python3 -c "import locale; print(locale.getpreferredencoding())"
ANSI_X3.4-1968

$ env -i LC_ALL=C LANG=en_US.UTF8 python3 -c "import locale; print(locale.getpreferredencoding())"
ANSI_X3.4-1968


So, passing "C.UTF-8" is fine, but not passing just "C".

And I have been forcing this on the WPE EWS bot one way or the another until I realized its the WebKit build system itself the one overriding this and setting LC_ALL=C in order to disable GCC "smartquotes"! https://trac.webkit.org/browser/webkit/trunk/Tools/Scripts/webkitpy/common/system/environment.py?rev=221898#L37

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20170912/1e5ba2c5/attachment-0001.html>