[Webkit-unassigned] [Bug 11924] New: WebCore crash on 64-bit code when loading a web page with a table tag

Fri Dec 22 00:03:54 PST 2006

http://bugs.webkit.org/show_bug.cgi?id=11924

           Summary: WebCore crash on 64-bit code when loading a web page
                    with a table tag
           Product: WebKit
           Version: 420+ (nightly)
          Platform: PC
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: Blocker
          Priority: P2
         Component: WebCore Misc.
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: reddy at pixar.com


If you try to build WebKit as 64-bit code (-m64) then it will core dump
whenever you try to load an HTML page that contains a <table> tag. For example,
the following simple web page will crash WebKit when compiled 64-bit:

  <html><body>
  
  <table><tr><td></td></tr></table>
  </body></html>

The problem is in WebCore/rendering/RenderTableSection.cpp, in the ensureRows()
method. The issue is this line:

if (numRows > static_cast<int>(numeric_limits<size_t>::max() /
sizeof(RowStruct)))

In a 64-bit environment, size_t is a 64-bit value but int is 32-bit, so casting
this very large number to a signed int will give you a negative number. This
causes the vector to not be resized in the normal case. Instead, the result of
the division should be kept as a size_t and numRows should be promoted to a
size_t for the comparison (either implicitly or explicitly). Here's the result
of 'svn diff' on a fix that I made for this bug in my tree:

Index: RenderTableSection.cpp
===================================================================

--- RenderTableSection.cpp      (revision 18367)
+++ RenderTableSection.cpp      (working copy)
@@ -149,7 +149,8 @@
     int nRows = gridRows;
     if (numRows > nRows) {
         if (numRows > static_cast<int>(grid.size())) {
-            if (numRows > static_cast<int>(numeric_limits<size_t>::max() /
sizeof(RowStruct)))
+            size_t maxSize = numeric_limits<size_t>::max() /
sizeof(RowStruct);
+            if (static_cast<size_t>(numRows) > maxSize)
                 return false;
             grid.resize(numRows);
         }

This explicitly casts numRows to a size_t, which you don't have to do of
course, but it does make it a bit more clear. Feel free to employ a better
solution if you prefer though.

I don't have write privileges to the WebKit tree, so I'm submitting this as a
bug. I'd be happy to help out more though.

Cheers,

Martin.


-- 
Configure bugmail: http://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

