<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[200105] trunk</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/200105">200105</a></dd>
<dt>Author</dt> <dd>fpizlo@apple.com</dd>
<dt>Date</dt> <dd>2016-04-26 13:01:13 -0700 (Tue, 26 Apr 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>WebCore on Mac ignores the user's preferred region (country) while getting the language
https://bugs.webkit.org/show_bug.cgi?id=156993
Reviewed by Geoffrey Garen.
Source/WebCore:
This is tested by the NavigatorLanguage API test.
WebCore was previously getting the list of preferred languages, and for each one, deducing
the default region. That's wrong, since for example it doesn't respect the user's choice (in
System Preferences) to display dates/calenders/etc according to a different region (like how
I have my machine set to en-pl right now).
It might be possible for the country code we get via kCFLocaleCountryCode to be something
that our ICU doesn't handle. To defend against this, we search for the resulting country
code in ICU's ISO countries list. If it doesn't appear in that list, we fall back on old
behavior.
* platform/mac/Language.mm:
(WebCore::httpStyleLanguageCode):
(WebCore::isValidICUCountryCode):
(WebCore::platformUserPreferredLanguages):
Tools:
Change the test expectations for this test. If the variant is not explicitly specified in
the AppleLanguage, then we use the locale's country code, which this test assumes is US.
* TestWebKitAPI/Tests/mac/NavigatorLanguage.mm:
(TestWebKitAPI::languageForSystemLanguage):
(TestWebKitAPI::TEST):</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCoreplatformmacLanguagemm">trunk/Source/WebCore/platform/mac/Language.mm</a></li>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsTestWebKitAPITestsmacNavigatorLanguagemm">trunk/Tools/TestWebKitAPI/Tests/mac/NavigatorLanguage.mm</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (200104 => 200105)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog 2016-04-26 19:49:19 UTC (rev 200104)
+++ trunk/Source/WebCore/ChangeLog 2016-04-26 20:01:13 UTC (rev 200105)
</span><span class="lines">@@ -1,3 +1,27 @@
</span><ins>+2016-04-26 Filip Pizlo <fpizlo@apple.com>
+
+ WebCore on Mac ignores the user's preferred region (country) while getting the language
+ https://bugs.webkit.org/show_bug.cgi?id=156993
+
+ Reviewed by Geoffrey Garen.
+
+ This is tested by the NavigatorLanguage API test.
+
+ WebCore was previously getting the list of preferred languages, and for each one, deducing
+ the default region. That's wrong, since for example it doesn't respect the user's choice (in
+ System Preferences) to display dates/calenders/etc according to a different region (like how
+ I have my machine set to en-pl right now).
+
+ It might be possible for the country code we get via kCFLocaleCountryCode to be something
+ that our ICU doesn't handle. To defend against this, we search for the resulting country
+ code in ICU's ISO countries list. If it doesn't appear in that list, we fall back on old
+ behavior.
+
+ * platform/mac/Language.mm:
+ (WebCore::httpStyleLanguageCode):
+ (WebCore::isValidICUCountryCode):
+ (WebCore::platformUserPreferredLanguages):
+
</ins><span class="cx"> 2016-04-26 Myles C. Maxfield <mmaxfield@apple.com>
</span><span class="cx">
</span><span class="cx"> Use auto-generated operators in FontPlatformData
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformmacLanguagemm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/mac/Language.mm (200104 => 200105)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/mac/Language.mm 2016-04-26 19:49:19 UTC (rev 200104)
+++ trunk/Source/WebCore/platform/mac/Language.mm 2016-04-26 20:01:13 UTC (rev 200105)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2003, 2005, 2006, 2010, 2011 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2003, 2005, 2006, 2010, 2011, 2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -30,6 +30,7 @@
</span><span class="cx"> #import "CFBundleSPI.h"
</span><span class="cx"> #import "WebCoreNSStringExtras.h"
</span><span class="cx"> #import <mutex>
</span><ins>+#import <unicode/uloc.h>
</ins><span class="cx"> #import <wtf/Assertions.h>
</span><span class="cx"> #import <wtf/Lock.h>
</span><span class="cx"> #import <wtf/NeverDestroyed.h>
</span><span class="lines">@@ -69,12 +70,14 @@
</span><span class="cx">
</span><span class="cx"> namespace WebCore {
</span><span class="cx">
</span><del>-static String httpStyleLanguageCode(NSString *language)
</del><ins>+static String httpStyleLanguageCode(NSString *language, NSString *country)
</ins><span class="cx"> {
</span><span class="cx"> SInt32 languageCode;
</span><span class="cx"> SInt32 regionCode;
</span><span class="cx"> SInt32 scriptCode;
</span><del>- CFStringEncoding stringEncoding;
</del><ins>+ CFStringEncoding stringEncoding;
+
+ bool languageDidSpecifyExplicitVariant = [language rangeOfCharacterFromSet:[NSCharacterSet characterSetWithCharactersInString:@"-_"]].location != NSNotFound;
</ins><span class="cx">
</span><span class="cx"> // FIXME: This transformation is very wrong:
</span><span class="cx"> // 1. There is no reason why CFBundle localization names would be at all related to language names as used on the Web.
</span><span class="lines">@@ -87,9 +90,19 @@
</span><span class="cx">
</span><span class="cx"> // Make the string lowercase.
</span><span class="cx"> NSString *lowercaseLanguageCode = [language lowercaseString];
</span><del>-
- // Turn a '_' into a '-' if it appears after a 2-letter language code.
</del><ins>+ NSString *lowercaseCountryCode = [country lowercaseString];
+
+ // If we see a "_" after a 2-letter language code:
+ // If the country is valid and the language did not specify a variant, replace the "_" and
+ // whatever comes after it with "-" followed by the country code.
+ // Otherwise, replace the "_" with a "-" and use whatever country
+ // CFBundleCopyLocalizationForLocalizationInfo() returned.
</ins><span class="cx"> if ([lowercaseLanguageCode length] >= 3 && [lowercaseLanguageCode characterAtIndex:2] == '_') {
</span><ins>+ if (country && !languageDidSpecifyExplicitVariant)
+ return [NSString stringWithFormat:@"%@-%@", [lowercaseLanguageCode substringWithRange:NSMakeRange(0, 2)], lowercaseCountryCode];
+
+ // Fall back to older behavior, which used the original language-based code but just changed
+ // the "_" to a "-".
</ins><span class="cx"> RetainPtr<NSMutableString> mutableLanguageCode = adoptNS([lowercaseLanguageCode mutableCopy]);
</span><span class="cx"> [mutableLanguageCode.get() replaceCharactersInRange:NSMakeRange(2, 1) withString:@"-"];
</span><span class="cx"> return mutableLanguageCode.get();
</span><span class="lines">@@ -98,6 +111,18 @@
</span><span class="cx"> return lowercaseLanguageCode;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+static bool isValidICUCountryCode(NSString* countryCode)
+{
+ const char* const* countries = uloc_getISOCountries();
+ const char* countryUTF8 = [countryCode UTF8String];
+ for (unsigned i = 0; countries[i]; ++i) {
+ const char* possibleCountry = countries[i];
+ if (!strcmp(countryUTF8, possibleCountry))
+ return true;
+ }
+ return false;
+}
+
</ins><span class="cx"> Vector<String> platformUserPreferredLanguages()
</span><span class="cx"> {
</span><span class="cx"> #if PLATFORM(MAC)
</span><span class="lines">@@ -113,13 +138,19 @@
</span><span class="cx"> Vector<String>& userPreferredLanguages = preferredLanguages();
</span><span class="cx">
</span><span class="cx"> if (userPreferredLanguages.isEmpty()) {
</span><ins>+ RetainPtr<CFLocaleRef> locale = adoptCF(CFLocaleCopyCurrent());
+ NSString *countryCode = (NSString *)CFLocaleGetValue(locale.get(), kCFLocaleCountryCode);
+
+ if (!isValidICUCountryCode(countryCode))
+ countryCode = nil;
+
</ins><span class="cx"> RetainPtr<CFArrayRef> languages = adoptCF(CFLocaleCopyPreferredLanguages());
</span><span class="cx"> CFIndex languageCount = CFArrayGetCount(languages.get());
</span><span class="cx"> if (!languageCount)
</span><span class="cx"> userPreferredLanguages.append("en");
</span><span class="cx"> else {
</span><span class="cx"> for (CFIndex i = 0; i < languageCount; i++)
</span><del>- userPreferredLanguages.append(httpStyleLanguageCode((NSString *)CFArrayGetValueAtIndex(languages.get(), i)));
</del><ins>+ userPreferredLanguages.append(httpStyleLanguageCode((NSString *)CFArrayGetValueAtIndex(languages.get(), i), countryCode));
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (200104 => 200105)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2016-04-26 19:49:19 UTC (rev 200104)
+++ trunk/Tools/ChangeLog 2016-04-26 20:01:13 UTC (rev 200105)
</span><span class="lines">@@ -1,3 +1,17 @@
</span><ins>+2016-04-26 Filip Pizlo <fpizlo@apple.com>
+
+ WebCore on Mac ignores the user's preferred region (country) while getting the language
+ https://bugs.webkit.org/show_bug.cgi?id=156993
+
+ Reviewed by Geoffrey Garen.
+
+ Change the test expectations for this test. If the variant is not explicitly specified in
+ the AppleLanguage, then we use the locale's country code, which this test assumes is US.
+
+ * TestWebKitAPI/Tests/mac/NavigatorLanguage.mm:
+ (TestWebKitAPI::languageForSystemLanguage):
+ (TestWebKitAPI::TEST):
+
</ins><span class="cx"> 2016-04-25 Ryosuke Niwa <rniwa@webkit.org>
</span><span class="cx">
</span><span class="cx"> Remove the build flag for template elements
</span></span></pre></div>
<a id="trunkToolsTestWebKitAPITestsmacNavigatorLanguagemm"></a>
<div class="modfile"><h4>Modified: trunk/Tools/TestWebKitAPI/Tests/mac/NavigatorLanguage.mm (200104 => 200105)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/TestWebKitAPI/Tests/mac/NavigatorLanguage.mm 2016-04-26 19:49:19 UTC (rev 200104)
+++ trunk/Tools/TestWebKitAPI/Tests/mac/NavigatorLanguage.mm 2016-04-26 20:01:13 UTC (rev 200105)
</span><span class="lines">@@ -68,17 +68,21 @@
</span><span class="cx"> return [webView stringByEvaluatingJavaScriptFromString:@"navigator.language"];
</span><span class="cx"> }
</span><span class="cx">
</span><del>-// These tests document current behavior. Some of the current results may not be right.
</del><ins>+// These tests document current behavior. Some of the current results may not be right. Note that
+// this oddly assumes that the user has set their language to something possibly-foreign but still
+// left their region as US. Hence the "-us" variants.
+// FIXME: These tests should also set the region to see how WebKit will handle that.
+// https://bugs.webkit.org/show_bug.cgi?id=157039
</ins><span class="cx"> NSArray *tests = @[
</span><del>- @[@"ru", @"ru-ru"], // This does not match other browsers or CFNetwork's Accept-Language, which all use "ru".
</del><ins>+ @[@"ru", @"ru-us"], // This does not match other browsers or CFNetwork's Accept-Language, which all use "ru".
</ins><span class="cx"> @[@"en", @"en-us"],
</span><span class="cx"> @[@"en-GB", @"en-gb"],
</span><span class="cx"> @[@"en-US", @"en-us"],
</span><del>- @[@"ja", @"ja-jp"],
- @[@"hi", @"hi-in"],
</del><ins>+ @[@"ja", @"ja-us"],
+ @[@"hi", @"hi-us"],
</ins><span class="cx"> @[@"zh-TW", @"zh-tw"], // This should not map to the generic zh-hant, see rdar://problem/21395180.
</span><span class="cx"> @[@"zh-HK", @"zh-tw"],
</span><del>- @[@"es", @"es-es"],
</del><ins>+ @[@"es", @"es-us"],
</ins><span class="cx"> @[@"es-MX", @"es-xl"],
</span><span class="cx"> @[@"es-ES", @"es-es"],
</span><span class="cx"> @[@"es-419", @"es-xl"],
</span><span class="lines">@@ -86,7 +90,7 @@
</span><span class="cx"> @[@"zh-Hant", @"zh-tw"],
</span><span class="cx"> @[@"pt-BR", @"pt-br"],
</span><span class="cx"> @[@"pt-PT", @"pt-pt"],
</span><del>- @[@"fr", @"fr-fr"],
</del><ins>+ @[@"fr", @"fr-us"],
</ins><span class="cx"> @[@"fr-CA", @"fr-ca"],
</span><span class="cx"> ];
</span><span class="cx">
</span></span></pre>
</div>
</div>
</body>
</html>