<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[201039] trunk</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/201039">201039</a></dd>
<dt>Author</dt> <dd>keith_miller@apple.com</dd>
<dt>Date</dt> <dd>2016-05-17 13:38:36 -0700 (Tue, 17 May 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>Add test262 harness support code
https://bugs.webkit.org/show_bug.cgi?id=157797
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
This patch adds some new tooling needed to run Test262 with the jsc
CLI. There were three options that needed to be added for Test262:
1) "--test262-async" This option overrides the print function in the test runner to look for
'Test262:AsyncTestComplete' instead of printing the passed text. If test262-async mode is on
and that string is not passed then the test is marked as failing.
2) "--strict-file=<file>" This option appends `"use strict";\n` to the beginning of the
passed file before passing the source code to the VM. This option can, in theory, be passed
multiple times.
3) "--exception=<name>" This option asserts that at the end of the last script file passed
the VM has an uncaught exception with its name property equal to the passed name.
* jsc.cpp:
(Script::Script):
(fillBufferWithContentsOfFile):
(functionPrint):
(checkUncaughtException):
(runWithScripts):
(printUsageStatement):
(CommandLine::parseArguments):
(runJSC):
Tools:
The import-test262-tests script is used to generate the yaml file used to run test262. It
takes a path to the local copy of the Test262 repository as well as an optional path to file
containing a list of failures. This script currently just creates the yaml file used to run
the script. It does not relocate the test files into our tests directory. In the future I
plan to add that feature but it didn't seem essential for the first iteration. Since many
test262 tests need to be run in both strict and non-strict mode, import-test262-tests
creates two separate runs for those tests. This enables us to distinguish between failures
in only one of the two modes.
This patch also updates the run-jsc-stress-tests Script to run tests from Test262. In order
to do so two new run commands were needed runTest262 and prepareTest262Fixture. runTest262
takes an actual test file along with the metadata associated with it. prepareTest262Fixture
takes a fixture file (used by module tests for importing) and makes sure that file is
properly relocated to the test runner directory.
The proccess I used to import the tests was to first run import-test262-tests to create a
yaml for all the tests (import-test262-tests assumes all tests pass if to failures file is
passed). Then I ran the generated yaml file with "run-jsc-stress-tests -v -c 1" piping the
output to a file and collected all the lines with "FAIL" in it. Finally, I reran
import-test262-tests with the new failure file to create the final yaml.
* Scripts/import-test262-tests: Added.
* Scripts/run-jsc-stress-tests:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceJavaScriptCoreChangeLog">trunk/Source/JavaScriptCore/ChangeLog</a></li>
<li><a href="#trunkSourceJavaScriptCorejsccpp">trunk/Source/JavaScriptCore/jsc.cpp</a></li>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsScriptsrunjscstresstests">trunk/Tools/Scripts/run-jsc-stress-tests</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunkToolsScriptsimporttest262tests">trunk/Tools/Scripts/import-test262-tests</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceJavaScriptCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ChangeLog (201038 => 201039)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ChangeLog 2016-05-17 19:38:51 UTC (rev 201038)
+++ trunk/Source/JavaScriptCore/ChangeLog 2016-05-17 20:38:36 UTC (rev 201039)
</span><span class="lines">@@ -1,3 +1,34 @@
</span><ins>+2016-05-17 Keith Miller <keith_miller@apple.com>
+
+ Add test262 harness support code
+ https://bugs.webkit.org/show_bug.cgi?id=157797
+
+ Reviewed by Filip Pizlo.
+
+ This patch adds some new tooling needed to run Test262 with the jsc
+ CLI. There were three options that needed to be added for Test262:
+
+ 1) "--test262-async" This option overrides the print function in the test runner to look for
+ 'Test262:AsyncTestComplete' instead of printing the passed text. If test262-async mode is on
+ and that string is not passed then the test is marked as failing.
+
+ 2) "--strict-file=<file>" This option appends `"use strict";\n` to the beginning of the
+ passed file before passing the source code to the VM. This option can, in theory, be passed
+ multiple times.
+
+ 3) "--exception=<name>" This option asserts that at the end of the last script file passed
+ the VM has an uncaught exception with its name property equal to the passed name.
+
+ * jsc.cpp:
+ (Script::Script):
+ (fillBufferWithContentsOfFile):
+ (functionPrint):
+ (checkUncaughtException):
+ (runWithScripts):
+ (printUsageStatement):
+ (CommandLine::parseArguments):
+ (runJSC):
+
</ins><span class="cx"> 2016-05-17 Filip Pizlo <fpizlo@apple.com>
</span><span class="cx">
</span><span class="cx"> WTF should know about Language
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejsccpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jsc.cpp (201038 => 201039)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jsc.cpp 2016-05-17 19:38:51 UTC (rev 201038)
+++ trunk/Source/JavaScriptCore/jsc.cpp 2016-05-17 20:38:36 UTC (rev 201039)
</span><span class="lines">@@ -530,6 +530,8 @@
</span><span class="cx"> const ClassInfo CustomGetter::s_info = { "CustomGetter", &Base::s_info, 0, CREATE_METHOD_TABLE(CustomGetter) };
</span><span class="cx"> const ClassInfo RuntimeArray::s_info = { "RuntimeArray", &Base::s_info, 0, CREATE_METHOD_TABLE(RuntimeArray) };
</span><span class="cx"> const ClassInfo SimpleObject::s_info = { "SimpleObject", &Base::s_info, 0, CREATE_METHOD_TABLE(SimpleObject) };
</span><ins>+static bool test262AsyncPassed { false };
+static bool test262AsyncTest { false };
</ins><span class="cx">
</span><span class="cx"> ElementHandleOwner* Element::handleOwner()
</span><span class="cx"> {
</span><span class="lines">@@ -637,13 +639,17 @@
</span><span class="cx"> static EncodedJSValue JSC_HOST_CALL functionSetGlobalConstRedeclarationShouldNotThrow(ExecState*);
</span><span class="cx">
</span><span class="cx"> struct Script {
</span><ins>+ bool parseAsStrict;
</ins><span class="cx"> bool isFile;
</span><span class="cx"> char* argument;
</span><span class="cx">
</span><del>- Script(bool isFile, char *argument)
- : isFile(isFile)
</del><ins>+ Script(bool parseAsStrict, bool isFile, char *argument)
+ : parseAsStrict(parseAsStrict)
+ , isFile(isFile)
</ins><span class="cx"> , argument(argument)
</span><span class="cx"> {
</span><ins>+ if (parseAsStrict)
+ ASSERT(isFile);
</ins><span class="cx"> }
</span><span class="cx"> };
</span><span class="cx">
</span><span class="lines">@@ -662,6 +668,7 @@
</span><span class="cx"> Vector<String> m_arguments;
</span><span class="cx"> bool m_profile { false };
</span><span class="cx"> String m_profilerOutput;
</span><ins>+ String m_uncaughtExceptionName;
</ins><span class="cx"> bool m_dumpSamplingProfilerData { false };
</span><span class="cx">
</span><span class="cx"> void parseArguments(int, char**);
</span><span class="lines">@@ -1038,12 +1045,14 @@
</span><span class="cx">
</span><span class="cx"> static bool fillBufferWithContentsOfFile(FILE* file, Vector<char>& buffer)
</span><span class="cx"> {
</span><ins>+ // We might have injected "use strict"; at the top.
+ size_t initialSize = buffer.size();
</ins><span class="cx"> fseek(file, 0, SEEK_END);
</span><span class="cx"> size_t bufferCapacity = ftell(file);
</span><span class="cx"> fseek(file, 0, SEEK_SET);
</span><del>- buffer.resize(bufferCapacity);
- size_t readSize = fread(buffer.data(), 1, buffer.size(), file);
- return readSize == buffer.size();
</del><ins>+ buffer.resize(bufferCapacity + initialSize);
+ size_t readSize = fread(buffer.data() + initialSize, 1, buffer.size(), file);
+ return readSize == buffer.size() - initialSize;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> static bool fillBufferWithContentsOfFile(const String& fileName, Vector<char>& buffer)
</span><span class="lines">@@ -1115,6 +1124,13 @@
</span><span class="cx">
</span><span class="cx"> EncodedJSValue JSC_HOST_CALL functionPrint(ExecState* exec)
</span><span class="cx"> {
</span><ins>+ if (test262AsyncTest) {
+ JSValue value = exec->argument(0);
+ if (value.isString() && WTF::equal(asString(value)->value(exec).impl(), "Test262:AsyncTestComplete"))
+ test262AsyncPassed = true;
+ return JSValue::encode(jsUndefined());
+ }
+
</ins><span class="cx"> for (unsigned i = 0; i < exec->argumentCount(); ++i) {
</span><span class="cx"> if (i)
</span><span class="cx"> putchar(' ');
</span><span class="lines">@@ -1973,8 +1989,31 @@
</span><span class="cx"> dumpException(globalObject, evaluationException->value());
</span><span class="cx"> }
</span><span class="cx">
</span><del>-static bool runWithScripts(GlobalObject* globalObject, const Vector<Script>& scripts, bool dump, bool module)
</del><ins>+static bool checkUncaughtException(VM& vm, GlobalObject* globalObject, NakedPtr<Exception> evaluationException, const String& expectedExceptionName)
</ins><span class="cx"> {
</span><ins>+ if (!evaluationException) {
+ printf("Expected uncaught exception with name '%s' but none was thrown\n", expectedExceptionName.utf8().data());
+ return false;
+ }
+
+ JSValue exception = evaluationException->value();
+ JSValue exceptionName = exception.get(globalObject->globalExec(), vm.propertyNames->name);
+
+ if (JSString* exceptionNameStr = jsDynamicCast<JSString*>(exceptionName)) {
+ const String& name = exceptionNameStr->value(globalObject->globalExec());
+ if (name == expectedExceptionName)
+ return true;
+ printf("Expected uncaught exception with name '%s' but got one with name '%s'\n", expectedExceptionName.utf8().data(), name.utf8().data());
+ dumpException(globalObject, exception);
+ return false;
+ }
+ printf("Expected uncaught exception with name '%s' but exception value did not have a name property\n", expectedExceptionName.utf8().data());
+ dumpException(globalObject, exception);
+ return false;
+}
+
+static bool runWithScripts(GlobalObject* globalObject, const Vector<Script>& scripts, const String& uncaughtExceptionName, bool dump, bool module)
+{
</ins><span class="cx"> String fileName;
</span><span class="cx"> Vector<char> scriptBuffer;
</span><span class="cx">
</span><span class="lines">@@ -1998,6 +2037,9 @@
</span><span class="cx"> JSInternalPromise* promise = nullptr;
</span><span class="cx"> if (scripts[i].isFile) {
</span><span class="cx"> fileName = scripts[i].argument;
</span><ins>+ if (scripts[i].parseAsStrict)
+ scriptBuffer.append("\"use strict\";\n", strlen("\"use strict\";\n"));
+
</ins><span class="cx"> if (module)
</span><span class="cx"> promise = loadAndEvaluateModule(globalObject->globalExec(), fileName);
</span><span class="cx"> else {
</span><span class="lines">@@ -2014,19 +2056,24 @@
</span><span class="cx"> if (module) {
</span><span class="cx"> if (!promise)
</span><span class="cx"> promise = loadAndEvaluateModule(globalObject->globalExec(), jscSource(scriptBuffer, fileName));
</span><del>- globalObject->globalExec()->clearException();
</del><ins>+ vm.clearException();
</ins><span class="cx"> promise->then(globalObject->globalExec(), nullptr, errorHandler);
</span><del>- globalObject->vm().drainMicrotasks();
</del><ins>+ vm.drainMicrotasks();
</ins><span class="cx"> } else {
</span><span class="cx"> NakedPtr<Exception> evaluationException;
</span><span class="cx"> JSValue returnValue = evaluate(globalObject->globalExec(), jscSource(scriptBuffer, fileName), JSValue(), evaluationException);
</span><del>- success = success && !evaluationException;
- if (dump && !evaluationException)
- printf("End: %s\n", returnValue.toString(globalObject->globalExec())->value(globalObject->globalExec()).utf8().data());
- dumpException(globalObject, evaluationException);
</del><ins>+ if (!uncaughtExceptionName || i != scripts.size() - 1) {
+ success = success && !evaluationException;
+ if (dump && !evaluationException)
+ printf("End: %s\n", returnValue.toString(globalObject->globalExec())->value(globalObject->globalExec()).utf8().data());
+ dumpException(globalObject, evaluationException);
+ } else
+ success = success && checkUncaughtException(vm, globalObject, evaluationException, uncaughtExceptionName);
+
</ins><span class="cx"> }
</span><span class="cx">
</span><del>- globalObject->globalExec()->clearException();
</del><ins>+ scriptBuffer.clear();
+ vm.clearException();
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> #if ENABLE(REGEXP_TRACING)
</span><span class="lines">@@ -2111,6 +2158,9 @@
</span><span class="cx"> fprintf(stderr, " -x Output exit code before terminating\n");
</span><span class="cx"> fprintf(stderr, "\n");
</span><span class="cx"> fprintf(stderr, " --sample Collects and outputs sampling profiler data\n");
</span><ins>+ fprintf(stderr, " --test262-async Check that some script calls the print function with the string 'Test262:AsyncTestComplete'\n");
+ fprintf(stderr, " --strict-file=<file> Parse the given file as if it were in strict mode (this option may be passed more than once)\n");
+ fprintf(stderr, " --exception=<name> Check the last script exits with an uncaught exception with the specified name\n");
</ins><span class="cx"> fprintf(stderr, " --options Dumps all JSC VM options and exits\n");
</span><span class="cx"> fprintf(stderr, " --dumpOptions Dumps all non-default JSC VM options before continuing\n");
</span><span class="cx"> fprintf(stderr, " --<jsc VM option>=<value> Sets the specified JSC VM option\n");
</span><span class="lines">@@ -2133,13 +2183,13 @@
</span><span class="cx"> if (!strcmp(arg, "-f")) {
</span><span class="cx"> if (++i == argc)
</span><span class="cx"> printUsageStatement();
</span><del>- m_scripts.append(Script(true, argv[i]));
</del><ins>+ m_scripts.append(Script(false, true, argv[i]));
</ins><span class="cx"> continue;
</span><span class="cx"> }
</span><span class="cx"> if (!strcmp(arg, "-e")) {
</span><span class="cx"> if (++i == argc)
</span><span class="cx"> printUsageStatement();
</span><del>- m_scripts.append(Script(false, argv[i]));
</del><ins>+ m_scripts.append(Script(false, false, argv[i]));
</ins><span class="cx"> continue;
</span><span class="cx"> }
</span><span class="cx"> if (!strcmp(arg, "-i")) {
</span><span class="lines">@@ -2197,6 +2247,23 @@
</span><span class="cx"> continue;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ if (!strcmp(arg, "--test262-async")) {
+ test262AsyncTest = true;
+ continue;
+ }
+
+ static const unsigned strictFileStrLength = strlen("--strict-file=");
+ if (!strncmp(arg, "--strict-file=", strictFileStrLength)) {
+ m_scripts.append(Script(true, true, argv[i] + strictFileStrLength));
+ continue;
+ }
+
+ static const unsigned exceptionStrLength = strlen("--exception=");
+ if (!strncmp(arg, "--exception=", exceptionStrLength)) {
+ m_uncaughtExceptionName = String(arg + exceptionStrLength);
+ continue;
+ }
+
</ins><span class="cx"> // See if the -- option is a JSC VM option.
</span><span class="cx"> if (strstr(arg, "--") == arg) {
</span><span class="cx"> if (!JSC::Options::setOption(&arg[2])) {
</span><span class="lines">@@ -2208,7 +2275,7 @@
</span><span class="cx">
</span><span class="cx"> // This arg is not recognized by the VM nor by jsc. Pass it on to the
</span><span class="cx"> // script.
</span><del>- m_scripts.append(Script(true, argv[i]));
</del><ins>+ m_scripts.append(Script(false, true, argv[i]));
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> if (hasBadJSCOptions && JSC::Options::validateOptions())
</span><span class="lines">@@ -2241,11 +2308,12 @@
</span><span class="cx"> vm->m_perBytecodeProfiler = std::make_unique<Profiler::Database>(*vm);
</span><span class="cx">
</span><span class="cx"> GlobalObject* globalObject = GlobalObject::create(*vm, GlobalObject::createStructure(*vm, jsNull()), options.m_arguments);
</span><del>- bool success = runWithScripts(globalObject, options.m_scripts, options.m_dump, options.m_module);
</del><ins>+ bool success = runWithScripts(globalObject, options.m_scripts, options.m_uncaughtExceptionName, options.m_dump, options.m_module);
</ins><span class="cx"> if (options.m_interactive && success)
</span><span class="cx"> runInteractive(globalObject);
</span><span class="cx">
</span><del>- result = success ? 0 : 3;
</del><ins>+ vm->drainMicrotasks();
+ result = success && (test262AsyncTest == test262AsyncPassed) ? 0 : 3;
</ins><span class="cx">
</span><span class="cx"> if (options.m_exitCode)
</span><span class="cx"> printf("jsc exiting %d\n", result);
</span></span></pre></div>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (201038 => 201039)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2016-05-17 19:38:51 UTC (rev 201038)
+++ trunk/Tools/ChangeLog 2016-05-17 20:38:36 UTC (rev 201039)
</span><span class="lines">@@ -1,3 +1,34 @@
</span><ins>+2016-05-17 Keith Miller <keith_miller@apple.com>
+
+ Add test262 harness support code
+ https://bugs.webkit.org/show_bug.cgi?id=157797
+
+ Reviewed by Filip Pizlo.
+
+ The import-test262-tests script is used to generate the yaml file used to run test262. It
+ takes a path to the local copy of the Test262 repository as well as an optional path to file
+ containing a list of failures. This script currently just creates the yaml file used to run
+ the script. It does not relocate the test files into our tests directory. In the future I
+ plan to add that feature but it didn't seem essential for the first iteration. Since many
+ test262 tests need to be run in both strict and non-strict mode, import-test262-tests
+ creates two separate runs for those tests. This enables us to distinguish between failures
+ in only one of the two modes.
+
+ This patch also updates the run-jsc-stress-tests Script to run tests from Test262. In order
+ to do so two new run commands were needed runTest262 and prepareTest262Fixture. runTest262
+ takes an actual test file along with the metadata associated with it. prepareTest262Fixture
+ takes a fixture file (used by module tests for importing) and makes sure that file is
+ properly relocated to the test runner directory.
+
+ The proccess I used to import the tests was to first run import-test262-tests to create a
+ yaml for all the tests (import-test262-tests assumes all tests pass if to failures file is
+ passed). Then I ran the generated yaml file with "run-jsc-stress-tests -v -c 1" piping the
+ output to a file and collected all the lines with "FAIL" in it. Finally, I reran
+ import-test262-tests with the new failure file to create the final yaml.
+
+ * Scripts/import-test262-tests: Added.
+ * Scripts/run-jsc-stress-tests:
+
</ins><span class="cx"> 2016-05-16 Enrica Casucci <enrica@apple.com>
</span><span class="cx">
</span><span class="cx"> build fix after r200968 and r200969
</span></span></pre></div>
<a id="trunkToolsScriptsimporttest262tests"></a>
<div class="addfile"><h4>Added: trunk/Tools/Scripts/import-test262-tests (0 => 201039)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/import-test262-tests (rev 0)
+++ trunk/Tools/Scripts/import-test262-tests 2016-05-17 20:38:36 UTC (rev 201039)
</span><span class="lines">@@ -0,0 +1,256 @@
</span><ins>+#!/usr/bin/env ruby
+
+# Copyright (C) 2016 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+require 'fileutils'
+require 'getoptlong'
+require 'pathname'
+require 'yaml'
+require 'find'
+
+THIS_SCRIPT_PATH = Pathname.new(__FILE__).realpath
+SCRIPTS_PATH = THIS_SCRIPT_PATH.dirname
+WEBKIT_PATH = SCRIPTS_PATH.dirname.dirname
+TEST262_YAML_PATH = WEBKIT_PATH + "Source/JavaScriptCore/tests/test262.yaml"
+
+raise unless SCRIPTS_PATH.basename.to_s == "Scripts"
+raise unless SCRIPTS_PATH.dirname.basename.to_s == "Tools"
+
+def mysys(*cmd)
+ printCommandArray(*cmd) if $verbosity >= 1
+ raise "Command failed: #{$?.inspect}" unless system(*cmd)
+end
+
+
+def usage
+ puts "import-test262-tests [options] <path-to-test262-repository>"
+ puts
+ puts "-h, --help print this help message."
+ puts "-f, --failures FAILURES Supplied file will be used to determine which tests fail."
+ puts " If a failures file is not provided all tests are assumed to pass."
+
+ exit 1
+end
+
+$failures = nil
+JS_TEST_REGEXP = /.*\.js/
+JS_FIXTURE_REGEXP = /.*(_FIXTURE\.js|_\.js)/
+GET_YAML_REGEXP = /\/\*---(?<yaml>.*?)---\*\//m
+
+GetoptLong.new(["--help", "-h", GetoptLong::NO_ARGUMENT],
+ ["--failures", "-f", GetoptLong::REQUIRED_ARGUMENT]).each {
+ | opt, arg |
+ case opt
+ when "--help"
+ usage
+ when "--failures"
+ $failures = File.open(Pathname.new(arg)).readlines
+ end
+}
+
+def didPassForMode(path, strict)
+ if $failures
+ if strict == :strict
+ return $failures.grep(/.*#{path}\.default-strict$/).length == 0
+ else
+ return $failures.grep(/.*#{path}\.default$/).length == 0
+ end
+ else
+ return true
+ end
+end
+
+class Test
+ attr_writer :failsWithException, :isModule, :isAsync
+ attr_accessor :includeFiles, :needsStrict, :needsNonStrict
+ attr_reader :path
+
+ def initialize(path)
+ @path = path
+ @failsWithException = nil
+ @includeFiles = []
+ @needsStrict = true
+ @needsNonStrict = true
+ @isModule = false
+ @isAsync = false
+ end
+
+ def check
+ # Throw an exception here since I'm not sure if the test infrastructure works in these cases.
+ raise if !@needsStrict and !@needsNonStrict
+ raise if @isModule and !@needsNonStrict
+ end
+
+ def formatFlags(strict)
+ flags = []
+ flags << strict if strict == :strict
+ flags << :module if @isModule
+ flags << :async if @isAsync
+
+ return flags.to_s
+ end
+
+ def formatCmdArguments(strict)
+ raise if strict == :strict ? !@needsStrict : !@needsNonStrict
+ passed = didPassForMode(@path, strict)
+ cmd = "runTest262"
+ cmd += passed ? " :normal, " : " :fail, "
+ cmd += @failsWithException ? @failsWithException.inspect : "\"NoException\""
+ cmd += ", #{@includeFiles.inspect}"
+ cmd += ", #{formatFlags(strict)}"
+ end
+
+ def finalizeIncludes
+ if @isAsync
+ @includeFiles << "donePrintHandle.js"
+ end
+
+ dir = Pathname.new(".")
+ @path.dirname.each_filename {
+ | part |
+ dir += ".."
+ }
+ dir += "harness"
+
+ @includeFiles.map! { | file | (dir + file).to_s }
+ end
+
+end
+
+def processTestFile(path)
+ /\/\*---(?<yaml>.*?)---\*\//m =~ File::read(path)
+
+ test = Test.new(path)
+ # These should be included by all the tests
+ test.includeFiles = ["assert.js", "sta.js"]
+
+ begin
+ yamlElements = YAML::load(yaml)
+ rescue Exception => e
+ puts "Failed to parse YAML for #{path}, threw exception:"
+ puts e.inspect
+ end
+ yamlElements.each {
+ | option |
+ case option[0]
+ when "negative"
+ test.failsWithException = option[1].to_s
+ when "includes"
+ test.includeFiles += option[1]
+ when "flags"
+ option[1].each {
+ | flag |
+ case flag
+ when "raw", "noStrict"
+ test.needsStrict = false
+ when "onlyStrict"
+ test.needsNonStrict = false
+ when "module"
+ test.isModule = true
+ test.needsStrict = false
+ when "async"
+ test.isAsync = true
+ when "generated"
+ else
+ raise "Invalid Metadata flag option, #{flag}, when parsing #{$benchmarkDirectory + $benchmark}"
+ end
+ }
+ end
+ }
+
+ test.finalizeIncludes
+ test.check
+
+ return test
+end
+
+class Fixture
+ attr_reader :path, :needsNonStrict, :needsStrict
+
+ def initialize(path)
+ @path = path
+ @needsNonStrict = true
+ @needsStrict = false
+ end
+
+ def formatCmdArguments(strict)
+ return "prepareTest262Fixture"
+ end
+
+end
+
+def processFixtureFile(path)
+ Fixture.new(path)
+end
+
+def processFilesRecursively(path)
+ # We only run the run the built-ins, language, and annexB tests.
+ # At some point we should add intl402
+ tests = []
+
+ Dir.chdir(path) {
+ paths = [Pathname.new("test/annexB"), Pathname.new("test/built-ins"), Pathname.new("test/language")]
+
+ paths.each {
+ | testPath |
+ Find.find(testPath) {
+ | file |
+ if File.file?(file) and JS_TEST_REGEXP =~ file.to_s
+ path = Pathname.new(file)
+ if JS_FIXTURE_REGEXP =~ file.to_s
+ tests << processFixtureFile(path)
+ else
+ tests << processTestFile(path)
+ end
+ end
+ }
+ }
+
+ }
+
+ return tests
+end
+
+def printYAML(tests)
+ File.open(TEST262_YAML_PATH, "w+") {
+ | outp |
+ outp.puts "---"
+ tests.each {
+ | test |
+ if test.needsNonStrict
+ outp.puts "- path: test262/" + test.path.to_s
+ outp.puts " cmd: " + test.formatCmdArguments(:nonStrict)
+ end
+ if test.needsStrict
+ outp.puts "- path: test262/" + test.path.to_s
+ outp.puts " cmd: " + test.formatCmdArguments(:strict)
+ end
+ }
+
+ }
+end
+
+tests = processFilesRecursively(Pathname.new(ARGV[0]))
+printYAML(tests)
</ins><span class="cx">Property changes on: trunk/Tools/Scripts/import-test262-tests
</span><span class="cx">___________________________________________________________________
</span></span></pre></div>
<a id="svnexecutable"></a>
<div class="addfile"><h4>Added: svn:executable</h4></div>
<a id="trunkToolsScriptsrunjscstresstests"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/run-jsc-stress-tests (201038 => 201039)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/run-jsc-stress-tests 2016-05-17 19:38:51 UTC (rev 201038)
+++ trunk/Tools/Scripts/run-jsc-stress-tests 2016-05-17 20:38:36 UTC (rev 201039)
</span><span class="lines">@@ -1011,6 +1011,61 @@
</span><span class="cx"> end
</span><span class="cx"> end
</span><span class="cx">
</span><ins>+def runTest262(mode, exception, includeFiles, flags)
+ failsWithException = exception != "NoException"
+ isStrict = false
+ isModule = false
+ isAsync = false
+
+ flags.each {
+ | flag |
+ case flag
+ when :strict
+ isStrict = true
+ when :module
+ isModule = true
+ when :async
+ isAsync = true
+ else
+ raise "Invalid flag for runTest262, #{flag}"
+ end
+ }
+
+ prepareExtraRelativeFiles(includeFiles.map { |f| "../" + f }, $collection)
+
+ args = [pathToVM.to_s] + BASE_OPTIONS
+ args << "-m" if isModule
+ args << "--exception=" + exception if failsWithException
+ args << "--test262-async" if isAsync
+ args += includeFiles
+
+ case mode
+ when :normal
+ errorHandler = simpleErrorHandler
+ outputHandler = silentOutputHandler
+ when :fail
+ errorHandler = expectedFailErrorHandler
+ outputHandler = noisyOutputHandler
+ else
+ raise "Invalid mode: #{mode}"
+ end
+
+ if isStrict
+ kind = "default-strict"
+ args << "--strict-file=#{$benchmark}"
+ else
+ kind = "default"
+ args << $benchmark.to_s
+ end
+
+ addRunCommand(kind, args, outputHandler, errorHandler)
+end
+
+def prepareTest262Fixture
+ # This function is used to add the files used by Test262 modules tests.
+ prepareExtraRelativeFiles([""], $collection)
+end
+
</ins><span class="cx"> def runES6(mode)
</span><span class="cx"> args = [pathToVM.to_s] + BASE_OPTIONS + [$benchmark.to_s]
</span><span class="cx"> case mode
</span><span class="lines">@@ -1149,7 +1204,9 @@
</span><span class="cx"> Dir.chdir($outputDir) {
</span><span class="cx"> extraFiles.each {
</span><span class="cx"> | file |
</span><del>- FileUtils.cp $extraFilesBaseDir + file, destination + file
</del><ins>+ dest = destination + file
+ FileUtils.mkdir_p(dest.dirname)
+ FileUtils.cp $extraFilesBaseDir + file, dest
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx"> end
</span></span></pre>
</div>
</div>
</body>
</html>