[Webkit-unassigned] [Bug 249720] New: import-w3c-tests appears to create a redundant *-expected.html file for each *-ref.html file

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=249720

            Bug ID: 249720
           Summary: import-w3c-tests appears to create a redundant
                    *-expected.html file for each *-ref.html file
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: ddkilzer at webkit.org
                CC: cdumez at apple.com, ntim at apple.com, rniwa at webkit.org,
                    simon.fraser at apple.com

import-w3c-tests appears to create a redundant *-expected.html file for each *-ref.html file.

In Bug 249691, I imported one new WPT (dom/xslt/sort.html), but three files were added:

* LayoutTests/imported/w3c/web-platform-tests/dom/xslt/sort-expected.html: Add.
* LayoutTests/imported/w3c/web-platform-tests/dom/xslt/sort-ref.html: Add.
* LayoutTests/imported/w3c/web-platform-tests/dom/xslt/sort.html: Add.

The `sort-expected.html` file doesn't exist upstream, and appears to have been created by `import-w3c-tests` in Tools/Scripts/webkitpy/w3c/test_importer.py (I haven't debugged this--it's just a guess that this code is responsible):

```
                if 'reference' in test_info.keys():
                    reftests += 1
                    total_tests += 1
                    test_basename = self.filesystem.basename(test_info['test'])

                    # Add the ref file, following WebKit style.
                    # FIXME: Ideally we'd support reading the metadata
                    # directly rather than relying  on a naming convention.
                    # Using a naming convention creates duplicate copies of the
                    # reference files.
                    ref_file = self.filesystem.splitext(test_basename)[0] + '-expected'
                    if 'type' in test_info and test_info['type'] == 'mismatch':
                        ref_file += '-mismatch'
                    ref_file += self.filesystem.splitext(test_info['reference'])[1]

                    copy_list.append({'src': test_info['reference'], 'dest': ref_file, 'reference_support_info': test_info['reference_support_info']})
                    copy_list.append({'src': test_info['test'], 'dest': filename})
```

There are 293 `*-ref.html` files without corresponding `*-expected.html` files:

$ for F in `find LayoutTests/imported/w3c/web-platform-tests -name \*-ref.html`; do G="`echo $F | sed -e 's/-ref.html/-expected.html/'`"; if [ ! -e $G ]; then ls -la $F; fi; done | wc -l
     293

While there are 1947 `*-ref.html` files with a `*-expected.html` file:

$ for F in `find LayoutTests/imported/w3c/web-platform-tests -name \*-ref.html`; do G="`echo $F | sed -e 's/-ref.html/-expected.html/'`"; if [ -e $G ]; then ls -la $G; fi; done | wc -l
    1947

In most cases the `*-ref.html` and `*-expected.html` files are the same, but not always:

$ cksum LayoutTests/imported/w3c/web-platform-tests/quirks/body-fills-html-quirk-{expected,ref}.html
707962029 242 LayoutTests/imported/w3c/web-platform-tests/quirks/body-fills-html-quirk-expected.html
3896252966 153 LayoutTests/imported/w3c/web-platform-tests/quirks/body-fills-html-quirk-ref.html

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20221221/9c142bee/attachment-0001.htm>