mod_pywebsocket for testing Web Socket implementation
Hi, webkit-dev, I'd like to propose to use mod_pywebsocket http://code.google.com/p/pywebsocket/ to test WebKit's implementation of Web Socket. To do that, we need to install mod_python and mod_pywebsocket to the Apache HTTP server used for testing WebKit. (I assume mod_ssl is already there.) I think I can modify .conf files under LayoutTests/http/conf but need instruction as to how to ensure that each test server has mod_python and mod_pywebsocket installed. Can anyone help? Any pointers or examples? Yuzo
On 2009-09-07, at 18:28, Yuzo Fujishima wrote:
Hi, webkit-dev,
I'd like to propose to use mod_pywebsocket http://code.google.com/p/pywebsocket/ to test WebKit's implementation of Web Socket.
To do that, we need to install mod_python and mod_pywebsocket to the Apache HTTP server used for testing WebKit. (I assume mod_ssl is already there.)
I think I can modify .conf files under LayoutTests/http/conf but need instruction as to how to ensure that each test server has mod_python and mod_pywebsocket installed. Can anyone help? Any pointers or examples?
The WebKit regression tests use the system version of Apache on each platform that they run on. mod_python and mod_pywebsocket are not Apache modules that are included out of the box on most platforms, which makes them difficult to use in our regression tests. I recall work was done on a server implementation for WebSocket regression testing that was written purely in Python. What happened to that? Why was that simple approach dropped in favor of an approach that requires multiple third-party Apache modules to be built and installed? - Mark
Hi, Mark, Thank you for the response. I've switched to mod_python-based approach because: - It should be closer to what real web sites would do. (See a comment by ap@: https://bugs.webkit.org/show_bug.cgi?id=27490#c3) - I thought extending httpd is preferred to adding a new thing. (See a comment by eric@: https://bugs.webkit.org/show_bug.cgi?id=27491#c7) Maybe I misunderstood the intention. - I don't have to worry about SSL (mod_ssl would do it) And, I thought there is a way to install extension modules to the development httpd. Would installing mod_python and mod_pywebsocket be very difficult? Then perhaps I need to write an adapter to call mod_pywebsocket from a standalone Python web server such as SimpleHTTPServer-derived one. (SSL could be an issue here. Python 2.3 doesn't have a module for SSL and we need to use a third party module such as pyOpenSSL. How easy it is to add a third party Python module to WebKit development environment?) Yuzo On Tue, Sep 8, 2009 at 11:21 AM, Mark Rowe<mrowe@apple.com> wrote:
On 2009-09-07, at 18:28, Yuzo Fujishima wrote:
Hi, webkit-dev,
I'd like to propose to use mod_pywebsocket http://code.google.com/p/pywebsocket/ to test WebKit's implementation of Web Socket.
To do that, we need to install mod_python and mod_pywebsocket to the Apache HTTP server used for testing WebKit. (I assume mod_ssl is already there.)
I think I can modify .conf files under LayoutTests/http/conf but need instruction as to how to ensure that each test server has mod_python and mod_pywebsocket installed. Can anyone help? Any pointers or examples?
The WebKit regression tests use the system version of Apache on each platform that they run on. mod_python and mod_pywebsocket are not Apache modules that are included out of the box on most platforms, which makes them difficult to use in our regression tests. I recall work was done on a server implementation for WebSocket regression testing that was written purely in Python. What happened to that? Why was that simple approach dropped in favor of an approach that requires multiple third-party Apache modules to be built and installed?
- Mark
On 2009-09-07, at 21:30, Yuzo Fujishima wrote:
Hi, Mark,
Thank you for the response.
I've switched to mod_python-based approach because: - It should be closer to what real web sites would do. (See a comment by ap@: https://bugs.webkit.org/show_bug.cgi?id=27490#c3 )
- I thought extending httpd is preferred to adding a new thing. (See a comment by eric@: https://bugs.webkit.org/show_bug.cgi?id=27491#c7 ) Maybe I misunderstood the intention.
Eric's comment seems to be about code duplication in run-webkit-tests more than anything.
- I don't have to worry about SSL (mod_ssl would do it)
And, I thought there is a way to install extension modules to the development httpd. Would installing mod_python and mod_pywebsocket be very difficult?
Our goal is to have regression tests that work out of the box on as many platform as is possible. On Windows this means that the required software be available from Cygwin. On Mac OS X this means that required software is included with the operating system. I'm not aware of any existing regression tests that require extra software to be installed on all platforms before they can be run.
Then perhaps I need to write an adapter to call mod_pywebsocket from a standalone Python web server such as SimpleHTTPServer-derived one. (SSL could be an issue here. Python 2.3 doesn't have a module for SSL and we need to use a third party module such as pyOpenSSL. How easy it is to add a third party Python module to WebKit development environment?)
Python starting with v2.6 has reasonable built-in support for SSL servers. This is the version that ships with Mac OS X 10.6. Mac OS X 10.5 ships with Python 2.5 (predating this built-in support) but includes pyOpenSSL. Mac OS X 10.4 ships with Python 2.3 and does not include pyOpenSSL. From what I can see, Cygwin currently has Python 2.5 and does not appear to support pyOpenSSL. It seems that only a small portion of the WebSockets tests would need to deal with SSL, while the majority of the tests would work fine on any platform supporting Python 2.3. For the SSL tests it would be reasonable to have tests that worked out of the box on only some platforms, especially when those platforms are used by the majority of WebKit developers (Mac OS X >= 10.5 and Linux). Doing this seems like a simpler approach than requiring two third-party Apache modules be built + configured on all platforms in order to run even the basic tests. - Mark
On Tue, Sep 8, 2009 at 11:21 AM, Mark Rowe<mrowe@apple.com> wrote:
On 2009-09-07, at 18:28, Yuzo Fujishima wrote:
Hi, webkit-dev,
I'd like to propose to use mod_pywebsocket http://code.google.com/p/pywebsocket/ to test WebKit's implementation of Web Socket.
To do that, we need to install mod_python and mod_pywebsocket to the Apache HTTP server used for testing WebKit. (I assume mod_ssl is already there.)
I think I can modify .conf files under LayoutTests/http/conf but need instruction as to how to ensure that each test server has mod_python and mod_pywebsocket installed. Can anyone help? Any pointers or examples?
The WebKit regression tests use the system version of Apache on each platform that they run on. mod_python and mod_pywebsocket are not Apache modules that are included out of the box on most platforms, which makes them difficult to use in our regression tests. I recall work was done on a server implementation for WebSocket regression testing that was written purely in Python. What happened to that? Why was that simple approach dropped in favor of an approach that requires multiple third-party Apache modules to be built and installed?
- Mark
Hi, On Tue, Sep 8, 2009 at 2:00 PM, Mark Rowe<mrowe@apple.com> wrote:
On 2009-09-07, at 21:30, Yuzo Fujishima wrote:
Hi, Mark,
Thank you for the response.
I've switched to mod_python-based approach because: - It should be closer to what real web sites would do. (See a comment by ap@: https://bugs.webkit.org/show_bug.cgi?id=27490#c3)
- I thought extending httpd is preferred to adding a new thing. (See a comment by eric@: https://bugs.webkit.org/show_bug.cgi?id=27491#c7) Maybe I misunderstood the intention.
Eric's comment seems to be about code duplication in run-webkit-tests more than anything.
Yes, I actually agree now. :)
- I don't have to worry about SSL (mod_ssl would do it)
And, I thought there is a way to install extension modules to the development httpd. Would installing mod_python and mod_pywebsocket be very difficult?
Our goal is to have regression tests that work out of the box on as many platform as is possible. On Windows this means that the required software be available from Cygwin. On Mac OS X this means that required software is included with the operating system. I'm not aware of any existing regression tests that require extra software to be installed on all platforms before they can be run.
It should probably because Web Socket is the only feature so far that requires new behavior on the server side.
Then perhaps I need to write an adapter to call mod_pywebsocket from a standalone Python web server such as SimpleHTTPServer-derived one. (SSL could be an issue here. Python 2.3 doesn't have a module for SSL and we need to use a third party module such as pyOpenSSL. How easy it is to add a third party Python module to WebKit development environment?)
Python starting with v2.6 has reasonable built-in support for SSL servers. This is the version that ships with Mac OS X 10.6. Mac OS X 10.5 ships with Python 2.5 (predating this built-in support) but includes pyOpenSSL. Mac OS X 10.4 ships with Python 2.3 and does not include pyOpenSSL. From what I can see, Cygwin currently has Python 2.5 and does not appear to support pyOpenSSL.
It seems that only a small portion of the WebSockets tests would need to deal with SSL, while the majority of the tests would work fine on any platform supporting Python 2.3. For the SSL tests it would be reasonable to have tests that worked out of the box on only some platforms, especially when those platforms are used by the majority of WebKit developers (Mac OS X
= 10.5 and Linux). Doing this seems like a simpler approach than requiring two third-party Apache modules be built + configured on all platforms in order to run even the basic tests.
It sounds like a good news to me that we can give up SSL tests on some platforms! Yuzo
- Mark
On Tue, Sep 8, 2009 at 11:21 AM, Mark Rowe<mrowe@apple.com> wrote:
On 2009-09-07, at 18:28, Yuzo Fujishima wrote:
Hi, webkit-dev,
I'd like to propose to use mod_pywebsocket http://code.google.com/p/pywebsocket/ to test WebKit's implementation of Web Socket.
To do that, we need to install mod_python and mod_pywebsocket to the Apache HTTP server used for testing WebKit. (I assume mod_ssl is already there.)
I think I can modify .conf files under LayoutTests/http/conf but need instruction as to how to ensure that each test server has mod_python and mod_pywebsocket installed. Can anyone help? Any pointers or examples?
The WebKit regression tests use the system version of Apache on each platform that they run on. mod_python and mod_pywebsocket are not Apache modules that are included out of the box on most platforms, which makes them difficult to use in our regression tests. I recall work was done on a server implementation for WebSocket regression testing that was written purely in Python. What happened to that? Why was that simple approach dropped in favor of an approach that requires multiple third-party Apache modules to be built and installed?
- Mark
On Sep 7, 2009, at 10:00 PM, Mark Rowe wrote:
Python starting with v2.6 has reasonable built-in support for SSL servers. This is the version that ships with Mac OS X 10.6. Mac OS X 10.5 ships with Python 2.5 (predating this built-in support) but includes pyOpenSSL. Mac OS X 10.4 ships with Python 2.3 and does not include pyOpenSSL. From what I can see, Cygwin currently has Python 2.5 and does not appear to support pyOpenSSL.
It seems that only a small portion of the WebSockets tests would need to deal with SSL, while the majority of the tests would work fine on any platform supporting Python 2.3. For the SSL tests it would be reasonable to have tests that worked out of the box on only some platforms, especially when those platforms are used by the majority of WebKit developers (Mac OS X >= 10.5 and Linux). Doing this seems like a simpler approach than requiring two third-party Apache modules be built + configured on all platforms in order to run even the basic tests.
Building and installing some Apache modules doesn't seem like that big a deal to me, if it would really make testing more practical. However, it seems like using a premade websocket server implementation would make some forms of testing harder. In particular, it would be more difficult to test what happens in the face of malformed results from the server, which is a very important part of testing. So based on that, it probably makes more sense to do at least some of the testing and a lower level, so we can easily test invalid server responses. Regards, Maciej
Hi, Maciej, Thank you for your comment. On Tue, Sep 8, 2009 at 2:18 PM, Maciej Stachowiak<mjs@apple.com> wrote:
On Sep 7, 2009, at 10:00 PM, Mark Rowe wrote:
Python starting with v2.6 has reasonable built-in support for SSL servers. This is the version that ships with Mac OS X 10.6. Mac OS X 10.5 ships with Python 2.5 (predating this built-in support) but includes pyOpenSSL. Mac OS X 10.4 ships with Python 2.3 and does not include pyOpenSSL. From what I can see, Cygwin currently has Python 2.5 and does not appear to support pyOpenSSL.
It seems that only a small portion of the WebSockets tests would need to deal with SSL, while the majority of the tests would work fine on any platform supporting Python 2.3. For the SSL tests it would be reasonable to have tests that worked out of the box on only some platforms, especially when those platforms are used by the majority of WebKit developers (Mac OS X
= 10.5 and Linux). Doing this seems like a simpler approach than requiring two third-party Apache modules be built + configured on all platforms in order to run even the basic tests.
Building and installing some Apache modules doesn't seem like that big a deal to me, if it would really make testing more practical.
However, it seems like using a premade websocket server implementation would make some forms of testing harder. In particular, it would be more difficult to test what happens in the face of malformed results from the server, which is a very important part of testing.
So based on that, it probably makes more sense to do at least some of the testing and a lower level, so we can easily test invalid server responses.
Using mod_pywebsocket, we can write a server-side Web Socket handler for a URL and put it in a corresponding directory under LayoutTests/... A handler is a Python script and can send (or not send) arbitrary bytes, malformed or not, to the browser. A handler is called after Web Socket handshake. There is no way to send malformed messages during handshake, as far as mod_pywebsocket is functioning properly. We may need a different mechanism here. How do you test a case where the Web server returns broken HTTP headers, for example? Perhaps we can do similar for Web Sockets? Yuzo
Regards, Maciej
On Sep 7, 2009, at 10:41 PM, Yuzo Fujishima wrote:
Building and installing some Apache modules doesn't seem like that big a deal to me, if it would really make testing more practical.
However, it seems like using a premade websocket server implementation would make some forms of testing harder. In particular, it would be more difficult to test what happens in the face of malformed results from the server, which is a very important part of testing.
So based on that, it probably makes more sense to do at least some of the testing and a lower level, so we can easily test invalid server responses.
Using mod_pywebsocket, we can write a server-side Web Socket handler for a URL and put it in a corresponding directory under LayoutTests/...
A handler is a Python script and can send (or not send) arbitrary bytes, malformed or not, to the browser.
Sounds like that part will be easy to test, then, though I'm surprised that mod_pywebsocket doesn't implement the WebSocket protocol framing.
A handler is called after Web Socket handshake. There is no way to send malformed messages during handshake, as far as mod_pywebsocket is functioning properly. We may need a different mechanism here.
How do you test a case where the Web server returns broken HTTP headers, for example? Perhaps we can do similar for Web Sockets?
One way is to control the status line and headers is using .asis files. Regards, Maciej
Hi, Maciej, On Tue, Sep 8, 2009 at 3:24 PM, Maciej Stachowiak<mjs@apple.com> wrote:
On Sep 7, 2009, at 10:41 PM, Yuzo Fujishima wrote:
Building and installing some Apache modules doesn't seem like that big a deal to me, if it would really make testing more practical.
However, it seems like using a premade websocket server implementation would make some forms of testing harder. In particular, it would be more difficult to test what happens in the face of malformed results from the server, which is a very important part of testing.
So based on that, it probably makes more sense to do at least some of the testing and a lower level, so we can easily test invalid server responses.
Using mod_pywebsocket, we can write a server-side Web Socket handler for a URL and put it in a corresponding directory under LayoutTests/...
A handler is a Python script and can send (or not send) arbitrary bytes, malformed or not, to the browser.
Sounds like that part will be easy to test, then, though I'm surprised that mod_pywebsocket doesn't implement the WebSocket protocol framing.
mod_pywebsocket provides utility classes that handles framing. Normal handlers are expected to use them. http://code.google.com/p/pywebsocket/source/browse/trunk/src/example/echo_ws... mod_pywebsocket allows direct access to the connection in addition.
A handler is called after Web Socket handshake. There is no way to send malformed messages during handshake, as far as mod_pywebsocket is functioning properly. We may need a different mechanism here.
How do you test a case where the Web server returns broken HTTP headers, for example? Perhaps we can do similar for Web Sockets?
One way is to control the status line and headers is using .asis files.
Thank you! Yuzo
Regards, Maciej
07.09.2009, в 21:30, Yuzo Fujishima написал(а):
Would installing mod_python and mod_pywebsocket be very difficult?
As others already stated, making the user install additional modules is undesirable. My original comment was about having a simple module implementation that could be built from source by run-webkit-tests, and used without installation. Clearly, keeping source of mod_python in WebKit repository and building it from source is not going to work well.
- It should be closer to what real web sites would do. (See a comment by ap@: https://bugs.webkit.org/show_bug.cgi? id=27490#c3)
I still think that being close to real web sites is important. In particular, it is likely that Web Sockets servers will need to share host with HTTP in order to share cookies and authentication credentials. It seems that an Apache module handling the Upgrade header is a natural approach for this. On the other hand, we cannot know how exactly Web Sockets will be deployed in practice, so maybe we can just do what's easiest for us for now. - WBR, Alexey Proskuryakov
participants (4)
-
Alexey Proskuryakov
-
Maciej Stachowiak
-
Mark Rowe
-
Yuzo Fujishima