<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[185357] trunk/Source/WebKit2</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/185357">185357</a></dd>
<dt>Author</dt> <dd>jhoneycutt@apple.com</dd>
<dt>Date</dt> <dd>2015-06-09 01:18:31 -0700 (Tue, 09 Jun 2015)</dd>
</dl>
<h3>Log Message</h3>
<pre>[iOS] UI process memory spike, hang when uploading a very large JPEG
<https://bugs.webkit.org/show_bug.cgi?id=145567>
<rdar://problem/21206699>
When uploading an image, we generate a thumbnail to use for an icon
that sits next to the control. To do this, we load the image and draw
it into a thumbnail-sized canvas. This can be very expensive for large
images.
To fix this, we'll use CGImageSource API, which will allow us to get
any existing thumbnail that exists in the source file.
Reviewed by Darin Adler.
* UIProcess/ios/forms/WKFileUploadPanel.mm:
(squareCropRectForSize):
Use std::round, based on Darin's review feedback that a) rintf is
incorrect in 64-bit systems where CGFloat is a double, b) rintf uses a
variable rounding mode, which is undesirable.
(squareImage):
Changed to take a CGImageRef.
Use RetainPtr and +[UIImage imageWithCGImage:] to match other code in
this file.
(thumbnailSizedImageForImage):
Changed to take a CGImageRef and to use iconSideLength.
(iconForImageFile):
Create an image source for the file, then use CGImageSource API to
generate a thumbnail.
(-[_WKImageFileUploadItem initWithFileURL:originalImage:]): Deleted.
(-[_WKImageFileUploadItem displayImage]):
Use iconForImageFile() rather than trying to generate an icon from the
UIImage.
(-[WKFileUploadPanel _uploadItemForImageData:imageName:successBlock:failureBlock:]):
Removed originalImage parameter; _WKImageFileUploadItem no longer uses
it.
(-[WKFileUploadPanel _uploadItemForJPEGRepresentationOfImage:successBlock:failureBlock:]):
(-[WKFileUploadPanel _uploadItemForImage:withAssetURL:successBlock:failureBlock:]):</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWebKit2ChangeLog">trunk/Source/WebKit2/ChangeLog</a></li>
<li><a href="#trunkSourceWebKit2UIProcessiosformsWKFileUploadPanelmm">trunk/Source/WebKit2/UIProcess/ios/forms/WKFileUploadPanel.mm</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWebKit2ChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/ChangeLog (185356 => 185357)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/ChangeLog 2015-06-09 07:45:12 UTC (rev 185356)
+++ trunk/Source/WebKit2/ChangeLog 2015-06-09 08:18:31 UTC (rev 185357)
</span><span class="lines">@@ -1,3 +1,44 @@
</span><ins>+2015-06-02 Jon Honeycutt <jhoneycutt@apple.com>
+
+ [iOS] UI process memory spike, hang when uploading a very large JPEG
+
+ <https://bugs.webkit.org/show_bug.cgi?id=145567>
+ <rdar://problem/21206699>
+
+ When uploading an image, we generate a thumbnail to use for an icon
+ that sits next to the control. To do this, we load the image and draw
+ it into a thumbnail-sized canvas. This can be very expensive for large
+ images.
+
+ To fix this, we'll use CGImageSource API, which will allow us to get
+ any existing thumbnail that exists in the source file.
+
+ Reviewed by Darin Adler.
+
+ * UIProcess/ios/forms/WKFileUploadPanel.mm:
+ (squareCropRectForSize):
+ Use std::round, based on Darin's review feedback that a) rintf is
+ incorrect in 64-bit systems where CGFloat is a double, b) rintf uses a
+ variable rounding mode, which is undesirable.
+ (squareImage):
+ Changed to take a CGImageRef.
+ Use RetainPtr and +[UIImage imageWithCGImage:] to match other code in
+ this file.
+ (thumbnailSizedImageForImage):
+ Changed to take a CGImageRef and to use iconSideLength.
+ (iconForImageFile):
+ Create an image source for the file, then use CGImageSource API to
+ generate a thumbnail.
+ (-[_WKImageFileUploadItem initWithFileURL:originalImage:]): Deleted.
+ (-[_WKImageFileUploadItem displayImage]):
+ Use iconForImageFile() rather than trying to generate an icon from the
+ UIImage.
+ (-[WKFileUploadPanel _uploadItemForImageData:imageName:successBlock:failureBlock:]):
+ Removed originalImage parameter; _WKImageFileUploadItem no longer uses
+ it.
+ (-[WKFileUploadPanel _uploadItemForJPEGRepresentationOfImage:successBlock:failureBlock:]):
+ (-[WKFileUploadPanel _uploadItemForImage:withAssetURL:successBlock:failureBlock:]):
+
</ins><span class="cx"> 2015-06-08 Dan Bernstein <mitz@apple.com>
</span><span class="cx">
</span><span class="cx"> [Cocoa] Use generics in framework headers
</span></span></pre></div>
<a id="trunkSourceWebKit2UIProcessiosformsWKFileUploadPanelmm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/UIProcess/ios/forms/WKFileUploadPanel.mm (185356 => 185357)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/UIProcess/ios/forms/WKFileUploadPanel.mm 2015-06-09 07:45:12 UTC (rev 185356)
+++ trunk/Source/WebKit2/UIProcess/ios/forms/WKFileUploadPanel.mm 2015-06-09 08:18:31 UTC (rev 185357)
</span><span class="lines">@@ -90,43 +90,42 @@
</span><span class="cx">
</span><span class="cx"> #pragma mark - Icon generation
</span><span class="cx">
</span><ins>+static const CGFloat iconSideLength = 100;
+
</ins><span class="cx"> static CGRect squareCropRectForSize(CGSize size)
</span><span class="cx"> {
</span><span class="cx"> CGFloat smallerSide = MIN(size.width, size.height);
</span><span class="cx"> CGRect cropRect = CGRectMake(0, 0, smallerSide, smallerSide);
</span><span class="cx">
</span><span class="cx"> if (size.width < size.height)
</span><del>- cropRect.origin.y = rintf((size.height - smallerSide) / 2);
</del><ins>+ cropRect.origin.y = std::round((size.height - smallerSide) / 2);
</ins><span class="cx"> else
</span><del>- cropRect.origin.x = rintf((size.width - smallerSide) / 2);
</del><ins>+ cropRect.origin.x = std::round((size.width - smallerSide) / 2);
</ins><span class="cx">
</span><span class="cx"> return cropRect;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-static UIImage *squareImage(UIImage *image)
</del><ins>+static UIImage *squareImage(CGImageRef image)
</ins><span class="cx"> {
</span><span class="cx"> if (!image)
</span><span class="cx"> return nil;
</span><span class="cx">
</span><del>- CGImageRef imageRef = [image CGImage];
- CGSize imageSize = CGSizeMake(CGImageGetWidth(imageRef), CGImageGetHeight(imageRef));
</del><ins>+ CGSize imageSize = CGSizeMake(CGImageGetWidth(image), CGImageGetHeight(image));
</ins><span class="cx"> if (imageSize.width == imageSize.height)
</span><del>- return image;
</del><ins>+ return [UIImage imageWithCGImage:image];
</ins><span class="cx">
</span><span class="cx"> CGRect squareCropRect = squareCropRectForSize(imageSize);
</span><del>- CGImageRef squareImageRef = CGImageCreateWithImageInRect(imageRef, squareCropRect);
- UIImage *squareImage = [[UIImage alloc] initWithCGImage:squareImageRef imageOrientation:[image imageOrientation]];
- CGImageRelease(squareImageRef);
- return [squareImage autorelease];
</del><ins>+ RetainPtr<CGImageRef> squareImage = adoptCF(CGImageCreateWithImageInRect(image, squareCropRect));
+ return [UIImage imageWithCGImage:squareImage.get()];
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-static UIImage *thumbnailSizedImageForImage(UIImage *image)
</del><ins>+static UIImage *thumbnailSizedImageForImage(CGImageRef image)
</ins><span class="cx"> {
</span><span class="cx"> UIImage *squaredImage = squareImage(image);
</span><span class="cx"> if (!squaredImage)
</span><span class="cx"> return nil;
</span><span class="cx">
</span><del>- CGRect destRect = CGRectMake(0, 0, 100, 100);
</del><ins>+ CGRect destRect = CGRectMake(0, 0, iconSideLength, iconSideLength);
</ins><span class="cx"> UIGraphicsBeginImageContext(destRect.size);
</span><span class="cx"> CGContextSetInterpolationQuality(UIGraphicsGetCurrentContext(), kCGInterpolationHigh);
</span><span class="cx"> [squaredImage drawInRect:destRect];
</span><span class="lines">@@ -140,18 +139,26 @@
</span><span class="cx"> ASSERT_ARG(file, [file isFileURL]);
</span><span class="cx">
</span><span class="cx"> UIDocumentInteractionController *interactionController = [UIDocumentInteractionController interactionControllerWithURL:file];
</span><del>- return thumbnailSizedImageForImage(interactionController.icons[0]);
</del><ins>+ return thumbnailSizedImageForImage(interactionController.icons[0].CGImage);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> static UIImage* iconForImageFile(NSURL *file)
</span><span class="cx"> {
</span><span class="cx"> ASSERT_ARG(file, [file isFileURL]);
</span><span class="cx">
</span><del>- if (UIImage *image = [UIImage imageWithContentsOfFile:file.path])
- return thumbnailSizedImageForImage(image);
</del><ins>+ NSDictionary *options = @{
+ (id)kCGImageSourceCreateThumbnailFromImageIfAbsent: @YES,
+ (id)kCGImageSourceThumbnailMaxPixelSize: @(iconSideLength),
+ (id)kCGImageSourceCreateThumbnailWithTransform: @YES,
+ };
+ RetainPtr<CGImageSource> imageSource = adoptCF(CGImageSourceCreateWithURL((CFURLRef)file, 0));
+ RetainPtr<CGImageRef> thumbnail = adoptCF(CGImageSourceCreateThumbnailAtIndex(imageSource.get(), 0, (CFDictionaryRef)options));
+ if (!thumbnail) {
+ LOG_ERROR("WKFileUploadPanel: Error creating thumbnail image for image: %@", file);
+ return fallbackIconForFile(file);
+ }
</ins><span class="cx">
</span><del>- LOG_ERROR("WKFileUploadPanel: Error creating thumbnail image for image: %@", file);
- return fallbackIconForFile(file);
</del><ins>+ return thumbnailSizedImageForImage(thumbnail.get());
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> static UIImage* iconForVideoFile(NSURL *file)
</span><span class="lines">@@ -169,8 +176,7 @@
</span><span class="cx"> return fallbackIconForFile(file);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- RetainPtr<UIImage> image = adoptNS([[UIImage alloc] initWithCGImage:imageRef.get()]);
- return thumbnailSizedImageForImage(image.get());
</del><ins>+ return thumbnailSizedImageForImage(imageRef.get());
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> static UIImage* iconForFile(NSURL *file)
</span><span class="lines">@@ -238,22 +244,10 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> @interface _WKImageFileUploadItem : _WKFileUploadItem
</span><del>-- (instancetype)initWithFileURL:(NSURL *)fileURL originalImage:(UIImage *)originalImage;
</del><span class="cx"> @end
</span><span class="cx">
</span><del>-@implementation _WKImageFileUploadItem {
- RetainPtr<UIImage> _originalImage;
-}
</del><ins>+@implementation _WKImageFileUploadItem
</ins><span class="cx">
</span><del>-- (instancetype)initWithFileURL:(NSURL *)fileURL originalImage:(UIImage *)originalImage
-{
- if (!(self = [super initWithFileURL:fileURL]))
- return nil;
-
- _originalImage = originalImage;
- return self;
-}
-
</del><span class="cx"> - (BOOL)isVideo
</span><span class="cx"> {
</span><span class="cx"> return NO;
</span><span class="lines">@@ -261,7 +255,7 @@
</span><span class="cx">
</span><span class="cx"> - (UIImage *)displayImage
</span><span class="cx"> {
</span><del>- return thumbnailSizedImageForImage(_originalImage.get());
</del><ins>+ return iconForImageFile(self.fileURL);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> @end
</span><span class="lines">@@ -735,10 +729,9 @@
</span><span class="cx"> [self _uploadItemFromMediaInfo:info successBlock:uploadItemSuccessBlock failureBlock:failureBlock];
</span><span class="cx"> }
</span><span class="cx">
</span><del>-- (void)_uploadItemForImageData:(NSData *)imageData originalImage:(UIImage *)originalImage imageName:(NSString *)imageName successBlock:(void (^)(_WKFileUploadItem *))successBlock failureBlock:(void (^)(void))failureBlock
</del><ins>+- (void)_uploadItemForImageData:(NSData *)imageData imageName:(NSString *)imageName successBlock:(void (^)(_WKFileUploadItem *))successBlock failureBlock:(void (^)(void))failureBlock
</ins><span class="cx"> {
</span><span class="cx"> ASSERT_ARG(imageData, imageData);
</span><del>- ASSERT_ARG(originalImage, originalImage);
</del><span class="cx"> ASSERT(!isMainThread());
</span><span class="cx">
</span><span class="cx"> NSString * const kTemporaryDirectoryName = @"WKWebFileUpload";
</span><span class="lines">@@ -762,7 +755,7 @@
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- successBlock(adoptNS([[_WKImageFileUploadItem alloc] initWithFileURL:[NSURL fileURLWithPath:filePath] originalImage:originalImage]).get());
</del><ins>+ successBlock(adoptNS([[_WKImageFileUploadItem alloc] initWithFileURL:[NSURL fileURLWithPath:filePath]]).get());
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> - (void)_uploadItemForJPEGRepresentationOfImage:(UIImage *)image successBlock:(void (^)(_WKFileUploadItem *))successBlock failureBlock:(void (^)(void))failureBlock
</span><span class="lines">@@ -785,7 +778,7 @@
</span><span class="cx"> // naming each of the individual uploads image.jpg? This won't work for photos taken with
</span><span class="cx"> // the camera, but would work for photos picked from the library.
</span><span class="cx"> NSString * const kUploadImageName = @"image.jpg";
</span><del>- [self _uploadItemForImageData:jpeg originalImage:image imageName:kUploadImageName successBlock:successBlock failureBlock:failureBlock];
</del><ins>+ [self _uploadItemForImageData:jpeg imageName:kUploadImageName successBlock:successBlock failureBlock:failureBlock];
</ins><span class="cx"> });
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -817,7 +810,7 @@
</span><span class="cx">
</span><span class="cx"> RetainPtr<CFStringRef> extension = adoptCF(UTTypeCopyPreferredTagWithClass((CFStringRef)dataUTI, kUTTagClassFilenameExtension));
</span><span class="cx"> NSString *imageName = [@"image." stringByAppendingString:(extension ? (id)extension.get() : @"jpg")];
</span><del>- [self _uploadItemForImageData:imageData originalImage:image imageName:imageName successBlock:successBlock failureBlock:failureBlock];
</del><ins>+ [self _uploadItemForImageData:imageData imageName:imageName successBlock:successBlock failureBlock:failureBlock];
</ins><span class="cx"> }];
</span><span class="cx"> });
</span><span class="cx"> }
</span></span></pre>
</div>
</div>
</body>
</html>