Script programming language: Perl, Python, or Ruby?
Do we have any recommendation of programming language for scripts such as WebKitTools/Scripts? It seems new scripts are written by Python and Ruby code is very rare. Is it reasonable to port a Ruby script to Python? I tried to port PrettyPatch.rb to Python in https://bugs.webkit.org/show_bug.cgi?id=43617 in order to remove Ruby dependency for many developers and buildbots. However bdash objected it. -- TAMURA Kent Software Engineer, Google
The fewer languages we use, the easier it is for people to hack on stuff. Python is especially nice because it's optimized for readability and predictability, so it's more approachable to people who aren't experts in that language. If we currently have only a small dependency on Ruby, it makes sense to me to bring that dependence to 0 and to try and stick with Python from now on. J On Tue, Aug 31, 2010 at 3:56 PM, TAMURA, Kent <tkent@chromium.org> wrote:
Do we have any recommendation of programming language for scripts such as WebKitTools/Scripts? It seems new scripts are written by Python and Ruby code is very rare.
Is it reasonable to port a Ruby script to Python? I tried to port PrettyPatch.rb to Python in https://bugs.webkit.org/show_bug.cgi?id=43617 in order to remove Ruby dependency for many developers and buildbots. However bdash objected it.
-- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
I think generally most scripts are written in Python nowadays, and we have a large legacy of perl scripts that are getting ported over. bdash mentions in the bug that there are other scripts written in Ruby besides PrettyPatch, but that's news to me. What are they, and what are they used for? Is there a particular reason that they're in Ruby? I agree with Jeremy that, all other things being equal, fewer languages is better, and I'm not sure that Ruby offers many compelling advantages over Python (disclaimer: I personally like Ruby the language better than Python, although I have a lot more experience w/ Python than I do w/ Ruby). On Tue, Aug 31, 2010 at 7:56 AM, TAMURA, Kent <tkent@chromium.org> wrote:
Do we have any recommendation of programming language for scripts such as WebKitTools/Scripts? It seems new scripts are written by Python and Ruby code is very rare. Is it reasonable to port a Ruby script to Python? I tried to port PrettyPatch.rb to Python in https://bugs.webkit.org/show_bug.cgi?id=43617 in order to remove Ruby dependency for many developers and buildbots. However bdash objected it. -- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Oh, I should have mentioned that there is actually a compelling reason to port PrettyPatch over, namely that if we can port it to a Python library than we can avoid having to shell out to it from new-run-webkit-tests. Shelling out, in addition to being slower, seems to contribute to the Python multithreading deadlocking issues we've seen. -- Dirk On Tue, Aug 31, 2010 at 12:26 PM, Dirk Pranke <dpranke@chromium.org> wrote:
I think generally most scripts are written in Python nowadays, and we have a large legacy of perl scripts that are getting ported over.
bdash mentions in the bug that there are other scripts written in Ruby besides PrettyPatch, but that's news to me. What are they, and what are they used for? Is there a particular reason that they're in Ruby?
I agree with Jeremy that, all other things being equal, fewer languages is better, and I'm not sure that Ruby offers many compelling advantages over Python (disclaimer: I personally like Ruby the language better than Python, although I have a lot more experience w/ Python than I do w/ Ruby).
On Tue, Aug 31, 2010 at 7:56 AM, TAMURA, Kent <tkent@chromium.org> wrote:
Do we have any recommendation of programming language for scripts such as WebKitTools/Scripts? It seems new scripts are written by Python and Ruby code is very rare. Is it reasonable to port a Ruby script to Python? I tried to port PrettyPatch.rb to Python in https://bugs.webkit.org/show_bug.cgi?id=43617 in order to remove Ruby dependency for many developers and buildbots. However bdash objected it. -- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
WebKitTools/iExploder has some ruby scripts. LayoutTests/platform/mac/fast/loader/resources/mimeTypeExamples/example-rb.rb is an empty file. I don't know others. PrettyPatch on Python would remove Ruby dependency from new-run-webkit-tests and webkit-patch. But I'm not sure if we can remove BugsSite/PrettyPatch. Can we run python scripts on bugs.webkit.org server? On Wed, Sep 1, 2010 at 04:26, Dirk Pranke <dpranke@chromium.org> wrote:
I think generally most scripts are written in Python nowadays, and we have a large legacy of perl scripts that are getting ported over.
bdash mentions in the bug that there are other scripts written in Ruby besides PrettyPatch, but that's news to me. What are they, and what are they used for? Is there a particular reason that they're in Ruby?
I agree with Jeremy that, all other things being equal, fewer languages is better, and I'm not sure that Ruby offers many compelling advantages over Python (disclaimer: I personally like Ruby the language better than Python, although I have a lot more experience w/ Python than I do w/ Ruby).
On Tue, Aug 31, 2010 at 7:56 AM, TAMURA, Kent <tkent@chromium.org> wrote:
Do we have any recommendation of programming language for scripts such as WebKitTools/Scripts? It seems new scripts are written by Python and Ruby code is very rare. Is it reasonable to port a Ruby script to Python? I tried to port PrettyPatch.rb to Python in https://bugs.webkit.org/show_bug.cgi?id=43617 in order to remove Ruby dependency for many developers and buildbots. However bdash objected it. -- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
-- TAMURA Kent Software Engineer, Google
Thanks. So porting PrettyPatch to Python will make good benefit and should have no drawbacks. I should go ahead. On Wed, Sep 1, 2010 at 09:21, William Siegrist <wsiegrist@apple.com> wrote:
On Aug 31, 2010, at 4:43 PM, TAMURA, Kent wrote:
Can we run python scripts on bugs.webkit.org server?
Yes.
-Bill
-- TAMURA Kent Software Engineer, Google
Sorry, I seemed to have missed this thread. Please do not rewrite scripts just to rewrite them in a language you like. WebKit's dependency on Ruby is here to stay. Just because python is common at Google does not make it universal. -Sam On Wed, Sep 1, 2010 at 4:48 PM, TAMURA, Kent <tkent@chromium.org> wrote:
Thanks. So porting PrettyPatch to Python will make good benefit and should have no drawbacks. I should go ahead.
On Wed, Sep 1, 2010 at 09:21, William Siegrist <wsiegrist@apple.com>wrote:
On Aug 31, 2010, at 4:43 PM, TAMURA, Kent wrote:
Can we run python scripts on bugs.webkit.org server?
Yes.
-Bill
-- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Here are the list of files in the project which use ruby: Scripts/check-for-inappropriate-files-in-framework:#!/usr/bin/env ruby Scripts/check-for-webkit-framework-include-consistency:#!/usr/bin/env ruby Scripts/clean-header-guards:#!/usr/bin/ruby Scripts/roll-over-ChangeLogs:#!/usr/bin/env ruby iExploder/htdocs/iexploder.cgi:#!/usr/bin/ruby iExploder/htdocs/iexploder.rb: # These if statements are so that mod_ruby doesn't have to reload the files iExploder/htdocs/webserver.rb:#!/usr/bin/ruby iExploder/tools/lasthit.rb:#!/usr/bin/ruby iExploder/tools/osx_last_crash.rb:#!/usr/bin/ruby iExploder/tools/showtest.rb:#!/usr/bin/ruby -eric On Wed, Sep 1, 2010 at 7:40 PM, Sam Weinig <sam.weinig@gmail.com> wrote:
Sorry, I seemed to have missed this thread. Please do not rewrite scripts just to rewrite them in a language you like. WebKit's dependency on Ruby is here to stay. Just because python is common at Google does not make it universal. -Sam
On Wed, Sep 1, 2010 at 4:48 PM, TAMURA, Kent <tkent@chromium.org> wrote:
Thanks. So porting PrettyPatch to Python will make good benefit and should have no drawbacks. I should go ahead.
On Wed, Sep 1, 2010 at 09:21, William Siegrist <wsiegrist@apple.com> wrote:
On Aug 31, 2010, at 4:43 PM, TAMURA, Kent wrote:
Can we run python scripts on bugs.webkit.org server?
Yes.
-Bill
-- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Hi Sam, Did you see the reply I sent on this thread? There are actually decent reasons to rewrite the code into Python, to simplify and speed up the new-run-webkit-tests implementation. Given that, do you still object to the patch landing (since the work has already been done)? -- Dirk On Wed, Sep 1, 2010 at 7:40 PM, Sam Weinig <sam.weinig@gmail.com> wrote:
Sorry, I seemed to have missed this thread. Please do not rewrite scripts just to rewrite them in a language you like. WebKit's dependency on Ruby is here to stay. Just because python is common at Google does not make it universal. -Sam
On Wed, Sep 1, 2010 at 4:48 PM, TAMURA, Kent <tkent@chromium.org> wrote:
Thanks. So porting PrettyPatch to Python will make good benefit and should have no drawbacks. I should go ahead.
On Wed, Sep 1, 2010 at 09:21, William Siegrist <wsiegrist@apple.com> wrote:
On Aug 31, 2010, at 4:43 PM, TAMURA, Kent wrote:
Can we run python scripts on bugs.webkit.org server?
Yes.
-Bill
-- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
From reading this thread and the bug, it sounds like there is one key issue installing something new on build machines.
As discussed before in regards to python, there was a desire not to go above a certain version so that the tiger build machine could run a script. That seemed reasonable to me. Similarly there is a desire in this case to avoid Ruby because it means installing something new on other build machines. This also seems reasonable to me. There are other issues that seem tangential which are being brought up and only confuse the subject for me: - Speed -- without numbers this is meaningless to me. - A hang that might be reduced -- since it will remain, it sounds like there is a deeper issue to fix. - People like python -- me too, but perhaps not a deciding factor. It probably didn't help that the subject of the email focuses on languages in a highlander smack down style as if there can be only one. Focusing again, given the build machine issue: - Is it best to have a separate script that is similar to PrettyPatch solve the issue? - Or replace the script? - Or can the dependency on PrettyPatch be controlled via a commandline option and just turned off on build machines? - Or have folks explain why they don't want to install more on their build machine? dave
On Sep 1, 2010, at 10:46 PM, David Levin wrote:
From reading this thread and the bug, it sounds like there is one key issue installing something new on build machines.
As discussed before in regards to python, there was a desire not to go above a certain version so that the tiger build machine could run a script. That seemed reasonable to me.
Similarly there is a desire in this case to avoid Ruby because it means installing something new on other build machines. This also seems reasonable to me.
Hmm, I thought Ruby was a very longstanding dependency for the layout tests but it looks like it was a little over a year ago: 2009-04-24 Eric Seidel <eric@webkit.org> Reviewed by Adam Roben. Add PrettyPatch support to run-webkit-tests * Scripts/run-webkit-tests:
On Sep 1, 2010, at 9:28 PM, Dirk Pranke wrote:
Hi Sam,
Did you see the reply I sent on this thread? There are actually decent reasons to rewrite the code into Python, to simplify and speed up the new-run-webkit-tests implementation. Given that, do you still object to the patch landing (since the work has already been done)?
If there is a practical improvement that is best achieved by rewriting the tool in another language, then by all means let's do it. And I would also prefer to see fewer different languages used for scripts, since it is hard to be good at all of them. (I know Perl a lot better than Python or Ruby, but I would not recommend it for new tools.) I do agree with Sam that language rewrites just to use a new language are not so great. I am mildly skeptical that using PrettyPatch as a library is a major performance win, since in the common case of running tests there are few failures so it should not need to be invoked many times. (If it is invoked even for tests that didn't fail, then a much better performance improvement would be to not do that.) My skepticism would be 100% cured by a measurement of the actual performance improvement. I also note that the "make new-run-webkit-tests faster" motivation was not cited at all in the bug, just a "reduce dependencies" motivation, which doesn't seem well-aligned with where we are currently. If this was our only Ruby script I might feel differently. I think for new tools, we should definitely encourage a particular language, not just make it a free-for-all. And Python seems like the most plausible candidate going forward (Perl is too crufty and Ruby is less popular without being clearly better). OTOH it seems like some third-party tools such as bugzilla or iexploder will continue to use Perl or Ruby respectively since it would be unwise to work them just to change the language. Regards, Maciej
-- Dirk
On Wed, Sep 1, 2010 at 7:40 PM, Sam Weinig <sam.weinig@gmail.com> wrote:
Sorry, I seemed to have missed this thread. Please do not rewrite scripts just to rewrite them in a language you like. WebKit's dependency on Ruby is here to stay. Just because python is common at Google does not make it universal. -Sam
On Wed, Sep 1, 2010 at 4:48 PM, TAMURA, Kent <tkent@chromium.org> wrote:
Thanks. So porting PrettyPatch to Python will make good benefit and should have no drawbacks. I should go ahead.
On Wed, Sep 1, 2010 at 09:21, William Siegrist <wsiegrist@apple.com> wrote:
On Aug 31, 2010, at 4:43 PM, TAMURA, Kent wrote:
Can we run python scripts on bugs.webkit.org server?
Yes.
-Bill
-- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Oh, I didn't see your second post either. Sounds like porting everything into Python would speed up new-run-webkit-tests? Then I'm all for it. I think other folks would be equally convinced as well if you could kindly measure the time difference between Ruby + Python + shell script implementation vs Python implementation. - Ryosuke On Wed, Sep 1, 2010 at 9:28 PM, Dirk Pranke <dpranke@chromium.org> wrote:
Hi Sam,
Did you see the reply I sent on this thread? There are actually decent reasons to rewrite the code into Python, to simplify and speed up the new-run-webkit-tests implementation. Given that, do you still object to the patch landing (since the work has already been done)?
-- Dirk
On Wed, Sep 1, 2010 at 7:40 PM, Sam Weinig <sam.weinig@gmail.com> wrote:
Sorry, I seemed to have missed this thread. Please do not rewrite scripts just to rewrite them in a language you like. WebKit's dependency on Ruby is here to stay. Just because python is common at Google does not make it universal. -Sam
On Wed, Sep 1, 2010 at 4:48 PM, TAMURA, Kent <tkent@chromium.org> wrote:
Thanks. So porting PrettyPatch to Python will make good benefit and should have
no
drawbacks. I should go ahead.
On Wed, Sep 1, 2010 at 09:21, William Siegrist <wsiegrist@apple.com> wrote:
On Aug 31, 2010, at 4:43 PM, TAMURA, Kent wrote:
Can we run python scripts on bugs.webkit.org server?
Yes.
-Bill
-- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
I suspect that Maciej is right and whatever speedup we would see would be inconsequential. It's still a slightly cleaner design to not have to shell out to Ruby from Python, jumping from one scripting language to another. The deadlocking issue is a serious issue, blocking us from increasing our usage of new-run-webkit-tests, but simply porting PrettyPatch isn't going to fix it (for the record, I am actively working on fixing the deadlocking issue). But until we can put a stake into old-run-webkit-tests, we can't delete the ruby version anyway (well, I guess we could shell out from Perl to Python, but that would be only slightly less goofy). -- Dirk On Wed, Sep 1, 2010 at 11:30 PM, Ryosuke Niwa <rniwa@webkit.org> wrote:
Oh, I didn't see your second post either. Sounds like porting everything into Python would speed up new-run-webkit-tests? Then I'm all for it. I think other folks would be equally convinced as well if you could kindly measure the time difference between Ruby + Python + shell script implementation vs Python implementation. - Ryosuke
On Wed, Sep 1, 2010 at 9:28 PM, Dirk Pranke <dpranke@chromium.org> wrote:
Hi Sam,
Did you see the reply I sent on this thread? There are actually decent reasons to rewrite the code into Python, to simplify and speed up the new-run-webkit-tests implementation. Given that, do you still object to the patch landing (since the work has already been done)?
-- Dirk
On Wed, Sep 1, 2010 at 7:40 PM, Sam Weinig <sam.weinig@gmail.com> wrote:
Sorry, I seemed to have missed this thread. Please do not rewrite scripts just to rewrite them in a language you like. WebKit's dependency on Ruby is here to stay. Just because python is common at Google does not make it universal. -Sam
On Wed, Sep 1, 2010 at 4:48 PM, TAMURA, Kent <tkent@chromium.org> wrote:
Thanks. So porting PrettyPatch to Python will make good benefit and should have no drawbacks. I should go ahead.
On Wed, Sep 1, 2010 at 09:21, William Siegrist <wsiegrist@apple.com> wrote:
On Aug 31, 2010, at 4:43 PM, TAMURA, Kent wrote:
Can we run python scripts on bugs.webkit.org server?
Yes.
-Bill
-- TAMURA Kent Software Engineer, Google
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
On Sep 1, 2010, at 11:38 PM, Dirk Pranke wrote:
I suspect that Maciej is right and whatever speedup we would see would be inconsequential. It's still a slightly cleaner design to not have to shell out to Ruby from Python, jumping from one scripting language to another. The deadlocking issue is a serious issue, blocking us from increasing our usage of new-run-webkit-tests, but simply porting PrettyPatch isn't going to fix it (for the record, I am actively working on fixing the deadlocking issue).
But until we can put a stake into old-run-webkit-tests, we can't delete the ruby version anyway (well, I guess we could shell out from Perl to Python, but that would be only slightly less goofy).
PrettyPatch is also used by our bugzilla setup, which has no prospect of either going away or being rewritten from Perl to Python in the near feature. And it's useful standalone for other purposes. So there definitely needs to be a command-line version, even if there is also a library. Regards, Maciej
participants (9)
-
David Levin
-
Dirk Pranke
-
Eric Seidel
-
Jeremy Orlow
-
Maciej Stachowiak
-
Ryosuke Niwa
-
Sam Weinig
-
TAMURA, Kent
-
William Siegrist