[webkit-dev] HTML5 tokenizer landing soon

[Darin Adler]

On Jun 14, 2010, at 10:55 AM, Adam Barth wrote:

> The one complication might be that TextResourceDecoder::checkForHeadCharset seems to operate on the raw
> bytestream whereas the lexer operators on a stream of code units.

Just in case that’s not already clear, that’s because it’s TextResourceDecoder’s job is to produce that stream of code units.

    -- Darin