[Webkit-unassigned] [Bug 208232] New: WebDriver on non-iOS ports cannot perform ActionChain which has scrolling down to the element and click it

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=208232

            Bug ID: 208232
           Summary: WebDriver on non-iOS ports cannot perform ActionChain
                    which has scrolling down to the element and click it
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: WebDriver
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: tomoki.imai at sony.com
                CC: bburg at apple.com

Created attachment 391731

  --> https://bugs.webkit.org/attachment.cgi?id=391731&action=review

HTML and selenium script to reproduce the issue

Summary:

WebDriver on non-iOS ports cannot perform ActionChains which scrolls down to the element and click it.
Our investigation suggests that the issue comes from convertRectFromFrameClientToRootView, which was added in bug 195696.
convertRectFromFrameClientToRootView and the other calculation looks correct when frame delegates scrolling, but not when it doesn't delegate scrolling.

How to reproduce:
- Build WebKitGTK
- Unzip the attached package. It contains HTML file long-many-links.html and selenium script actionchains.py.
- Replace "/path_to_your_webkit_repo" in actionchains.py with your preference.
- Host long-many-links.html in your localhost:8080. You can edit actionchains.py if you want to use the other setting.
- Run actionchains.py with python3 selenium installed.

Expected behavior:
The script scrolls down the webpage, and then clicks "link 10".
No exceptions occurs.

Actual behavior:
The script scrolls down the webpage, and MoveTargetOutOfBoundsException occurs.
<class 'selenium.common.exceptions.MoveTargetOutOfBoundsException'>
('', None, None)

Environment:
- Ubuntu 18.04
- WebKitGTK trunk r257372
- python 3.6.9
- selenium 3.141.0 (installed by pip)

Our investigation:
We first noticed that "resultElementBounds" and "resultInViewCenterPoint" has negative y or some smaller value than expected.
It looks like we adjust the y-value by scroll position twice.
https://trac.webkit.org/browser/webkit/trunk/Source/WebKit/WebProcess/Automation/WebAutomationSessionProxy.cpp#L646

Next thing we see is that "convertRectFromFrameClientToRootView" function. As the name suggested, we expect that it returns RootView coordinate.
When the frame delegates scrolling, it seems to calculate an absolute coordinate from layout coordinate.
The returned value, for example "elementBoundsInRootCoordinates",  is passed to "absoluteTo..." functions.
https://trac.webkit.org/browser/webkit/trunk/Source/WebKit/WebProcess/Automation/WebAutomationSessionProxy.cpp#L585
On the other hand, when frame doesn't delegate scrolling, we suspect that "convertRectFromFrameClientToRootView" returns RootView coordinate.
RootView coordinate is not documented, but we guess it's coordinate which (0, 0) is the left-top corner of the root widget, which is different from absolute coordinate.
https://trac.webkit.org/browser/webkit/trunk/Source/WebKit/WebProcess/Automation/WebAutomationSessionProxy.cpp#L585

So our suspicion is that this issue is caused by the difference of which coordinates "convertRectFromFrameClientToRootView", depending on delegating scrolling.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20200226/085d7c72/attachment.htm>