I spent the entire day today figuring out how to get a backtrace from the installed copy of webkit-gtk. But I don't _just_ want to complain. I'd like to figure out a way to do this easily, and to update the documentation. The existing docs, https://trac.webkit.org/wiki/WebKitGTK/Debugging are pretty misleading: * Core dumps don't happen any more because of bubblewrap * WEBKIT2_PAUSE_WEB_PROCESS_ON_LAUNCH quietly does nothing unless you happen to have built with DEVELOPER_MODE (won't be the case on a linux distro) * The minibrowser isn't going to be available on a distro, and if you aren't using the minibrowser, gdb won't follow the right children... * And you can't reliably guess the right PID of a running process quick enough to attach to it in e.g. epiphany. Rebuilding from a git checkout and hacking on the webkit source to enable DEVELOPER_MODE or the minibrowser is not always feasible. (Or desirable -- I want to debug the code that is actually crashing, and not spend three days guessing!) I've been dealing with crashes on my laptop for years because it has very little RAM -- it simply doesn't have the resources to build a development copy of webkit-gtk. Even now, on a system with 64 low-power cores, it still takes more than a day, and that's after spending half the day figuring out how to configure the build exactly like the distro does, so that you can reproduce the crash. So for starters: is there something I missed? Some easy way to get a traceback from a modern, system copy of webkit-gtk? One idea: how hard would it be to skip bubblewrap in a release build if some environment variable is set?
Hey, sorry you had a bad time. Unfortunately everything on https://trac.webkit.org is obsolete. All documentation is on https://docs.webkit.org/ nowadays. But we don't yet have any warning banner to tell you of this. I have created an issue report: https://github.com/WebKit/Documentation/issues/99 Anyway, that documentation has not actually been migrated to the new docs website, so there is no newer documentation to point you to. :(
* Core dumps don't happen any more because of bubblewrap
Hm, I guess the "without systemd" instructions from that wiki page will not work anymore, since the core dump is probably created inside the sandbox now instead of on your host system? I had never considered this. I strongly recommend using systemd-coredump. Manual handling of core dumps is primitive and, as you've discovered, a waste of your time. I wrote a blog post on how to enable this if you haven't already: https://blogs.gnome.org/mcatanzaro/2021/09/18/creating-quality-backtraces-fo... There is just one trick with WebKit: you have to raise the core size limit to prevent systemd from discarding the core dump. Both the trac wiki page and my blog post contain instructions for how to do this. We really ought to document that somewhere on https://docs.webkit.org.
* WEBKIT2_PAUSE_WEB_PROCESS_ON_LAUNCH quietly does nothing unless you happen to have built with DEVELOPER_MODE (won't be the case on a linux distro)
I agree that limiting this to developer mode only makes debugging unnecessarily difficult. We should probably change that; I can't think of any good reason for limiting it. (But don't use this just to take a backtrace, since it's too much effort and a waste of your time.)
One idea: how hard would it be to skip bubblewrap in a release build if some environment variable is set?
Use the environment variable WEBKIT_DISABLE_SANDBOX_THIS_IS_DANGEROUS=1. (But certainly don't use this just to take a backtrace!) Michael
On Fri, 2024-09-06 at 17:28 -0500, Michael Catanzaro wrote:
Hey, sorry you had a bad time. Unfortunately everything on https://trac.webkit.org is obsolete. All documentation is on https://docs.webkit.org/ nowadays. But we don't yet have any warning banner to tell you of this. I have created an issue report: https://github.com/WebKit/Documentation/issues/99
Anyway, that documentation has not actually been migrated to the new docs website, so there is no newer documentation to point you to. :(
I figured. It's OK, that's why I brought it up.
* Core dumps don't happen any more because of bubblewrap
Hm, I guess the "without systemd" instructions from that wiki page will not work anymore, since the core dump is probably created inside the sandbox now instead of on your host system? I had never considered this.
Right. I was pretty close to editing bubblewrap.c and doing a sudo mv bwrap /usr/bin.
I strongly recommend using systemd-coredump. Manual handling of core dumps is primitive and, as you've discovered, a waste of your time. I wrote a blog post on how to enable this if you haven't already:
I too am primitive. None of these systems have systemd :)
Use the environment variable WEBKIT_DISABLE_SANDBOX_THIS_IS_DANGEROUS=1. (But certainly don't use this just to take a backtrace!)
I left the problem machine at the office overnight, but this sounds like exactly what I needed. I'm familiar enough with getting core dumps the old-fashioned way outside of bwrap. Is it dangerous in some new and exciting way, or is it simply that the sandbox is not in effect? If it's the latter, capturing the dump as a new unprivileged user would make it "safe" once again (so long as your filesystem permissions aren't messed up).
On Fri, Sep 6 2024 at 08:20:07 PM -04:00:00, Michael Orlitzky via webkit-dev <webkit-dev@lists.webkit.org> wrote:
Is it dangerous in some new and exciting way, or is it simply that the sandbox is not in effect? If it's the latter, capturing the dump as a new unprivileged user would make it "safe" once again (so long as your filesystem permissions aren't messed up).
You've simply got no sandbox. You'll be fine unless the website you're debugging is malicious.
On Sat, 2024-09-07 at 07:39 -0500, Michael Catanzaro wrote:
You've simply got no sandbox. You'll be fine unless the website you're debugging is malicious.
I tested it today and it does work for obtaining core dumps in the absence of systemd. Problem solved, thank you. A final note: I actually have three processes crashing in a row, and in that case it's necessary to e.g. sysctl "kernel.core_pattern=core.%P" to get them all.
A final final note: On the https://webkitgtk.org/ homepage, the links to these mailing list still use plain http, but the server responds only to https.
On Sat, Sep 7 2024 at 03:59:30 PM -04:00:00, Michael Orlitzky via webkit-dev <webkit-dev@lists.webkit.org> wrote:
On the https://webkitgtk.org/ homepage, the links to these mailing list still use plain http, but the server responds only to https.
As of today, the server is redirecting.
В Sat, 07 Sep 2024 15:49:09 -0400 Michael Orlitzky via webkit-dev <webkit-dev@lists.webkit.org> пишет:
On Sat, 2024-09-07 at 07:39 -0500, Michael Catanzaro wrote:
You've simply got no sandbox. You'll be fine unless the website you're debugging is malicious.
I tested it today and it does work for obtaining core dumps in the absence of systemd. Problem solved, thank you.
A final note: I actually have three processes crashing in a row, and in that case it's necessary to e.g. sysctl "kernel.core_pattern=core.%P" to get them all.
I suggest you also include %e (or even %E if you have several installations of same program in different places) into core_pattern to see easily which executable crashed.
participants (3)
-
Konstantin Tokarev
-
Michael Catanzaro
-
Michael Orlitzky