[Webkit-unassigned] [Bug 163155] Web Inspector: The following program hangs the heap snapshot tool

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=163155

--- Comment #2 from Joseph Pecoraro <joepeck at webkit.org> ---
This test creates 10 million strings with an array referencing each string. The snapshot contains:

  - 10M Nodes and eventually the data for each string.
  - 10M Edges (from the Array to Strings) containing a large numbers for indexes and node identifiers (mostly bloats the JSON)

On my machine the HeapSnapshot eventually completes but it takes a while and it was very large. The single JSON message is likely multiple GB. This is especially bad because all of the data is in unique strings which get serialized across. If they were objects referencing other objects there would be a lot of edges and the string data deduplicated.

I can inspect and drill into the HeapSnapshot but things are slow at certain stages:

  (1) Taking the snapshot and receiving it
      - Marking nodes is done in parallel
      - Sorting the snapshot by node ID could take a long time (10 million nodes)
      - Converting to a single large JSON message takes time and requires a lot of memory

To address issues here we could:

  - Chunk messages instead of sending 1 massive JSON message
    - Treat the data as a "Stream" that can parsed in pieces instead of JSON.parse requiring a complete object
      - NOTE: Blink does this, and they use this as a way to provide progress notifications while sending

  - Avoid sorting on the backend
    - This would require changes to the Snapshot data structure, how its processed and built-up on the frontend
    - This would likely grow the total snapshot memory size, but that could be mitigated if we chunk

  - Show Progress UI
    - Taking a snapshot involves multiple stages which we can portray explicitly or just a loading indicator
    - Backend Stages: gather nodes + edges, sort, deduplicate strings, serialize nodes, serialize edges, serialize data
    - Frontend Stages (on a Worker): JSON.parse, build structures for nodes, edges, post order traversal, dominators, category metadata

The other point that was slow:

  (2) Drilling into sections of the Snapshot
      - expanding "strings" sorts 10 million values (frontend sorts by its own criteria, such as size)
      - once loaded interacting with this is fine

To address issues here we could:

  - This work is already non-blocking work done on a Worker
  - Show progress indicator and UI string

---

Compared to other browsers:

  - Chrome DevTools is able to take the snapshot with nice progress indicators but crashes when trying to look at it
  - Firefox fails to take the snapshot with unknown errors
  - Safari eventually completes, is initially slow, but then is usable.

---

This seems like a great stress test for very large snapshots.

No matter what, this will be large and slow. So the apparent "hang" is the problem that should be mitigated.

I think addressing (1) with chunking and progress messages would have the best results.

Ultimately the inspected page will always be frozen while taking the snapshot, and that produces the SPOD Hanging indicator. I'm not sure how we workaround that. The same SPOD issue shows up in Debugging, when we pause inside a user gesture.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-unassigned/attachments/20161129/ec28a30f/attachment.html>