<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On May 22, 2016, at 3:39 PM, Daniel Olegovich Lazarenko <<a href="mailto:danielo@opera.com" class="">danielo@opera.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">What if I make a bug report in bugzilla about making a design spec of this feature. I could then write down implementation details options and summarize all points we've discussed here. Maybe in a form of a google document. This spec could then be reviewed and approved for action by you and any other interested people you want to be involved. Do you think it could work better?</div></div></div></blockquote><div><br class=""></div><div>Before getting to the design of things, we usually spend quite a bit of time figuring out what the use cases are, and from the those use cases, what the different possible solutions are that address those use cases. I don’t feel we have gotten past the use case part yet.</div><div><br class=""></div><div>- Sam</div><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Sun, May 22, 2016 at 11:59 PM, Brady Eidson <span dir="ltr" class=""><<a href="mailto:beidson@apple.com" target="_blank" class="">beidson@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 22, 2016, at 3:14 AM, Daniel Olegovich Lazarenko <<a href="mailto:danielo@opera.com" target="_blank" class="">danielo@opera.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><div class=""><div style="font-size:12.8px" class="">> It’s not yet clear what the ideal architecture is, which is one of the reasons why the mentioned issued remains unresolved.</div></div><div class=""><br class=""></div><div class="">What are the other reasons?</div></div></div></blockquote><div class=""><br class=""></div></span>Perhaps I misrepresented - AFAIK that is the only important reason.</div><span class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="">Are there any reasons that block us from discussing the right architecture?</div><div class="">I'd like to start working on a patch, but I need directions from you.</div></div></div></blockquote><div class=""><br class=""></div></div></span><div class="">I replied to this thread to describe significant issues with the two approaches you suggested.</div><div class=""><br class=""></div><div class="">But I am not able to conclude the thread by unilaterally giving directions to a lone contributor on how to properly implement this feature.</div><div class=""><br class=""></div><div class="">That’s a much broader conversation than just you and I.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">~Brady</div><div class=""><div class="h5"><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class="">I'd like to come up with some sort of a plan for this as well. Since the desired approach sounds complicated, it would be nice to split it as a series of patches where each patch is committed separately and improves the feature towards the goal.</div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Sun, May 22, 2016 at 6:16 AM, Brady Eidson <span dir="ltr" class=""><<a href="mailto:beidson@apple.com" target="_blank" class="">beidson@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 21, 2016, at 2:05 PM, Daniel Olegovich Lazarenko <<a href="mailto:danielo@opera.com" target="_blank" class="">danielo@opera.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><span style="font-size:12.8px" class="">> We are exploring ways to restore that full functionality - </span><a href="https://bugs.webkit.org/show_bug.cgi?id=138169" style="font-size:12.8px" target="_blank" class="">https://bugs.webkit.org/show_bug.cgi?id=138169</a><br class=""><div class=""><br class=""></div><div class="">Having custom scheme protocols is important to me too. I didn't know that it is broken with WKWebView. Do you know what exactly is broken?</div></div></div></blockquote><div class=""><br class=""></div></span>From most developer’s perspective, what is broken is that their NSURLProtocol they can register in their “UI Process” that used to work in WK1 views no longer has any effect in WK2.</div><div class=""><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class="">I thought that if you call [<span style="white-space:pre-wrap" class="">WKBrowsingContextController registerSchemeForCustomProtocol:] with your scheme, then it works. When I checked last (a year ago) it was implemented in a way that the WebProcess/</span><span style="white-space:pre-wrap" class="">NetworkingProcess</span><span style="white-space:pre-wrap" class=""> would pass a message to UIProcess, and handle the network request in the UIProcess. Did it change?</span></div></div></div></blockquote><div class=""><br class=""></div></span>That did not change.</div><div class=""><br class=""></div><div class="">But that mechanism was never API, and even as SPI it is formally deprecated.</div><div class=""><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><font class=""><span style="white-space:pre-wrap" class="">Assuming that </span></font><span style="white-space:pre-wrap" class="">registerSchemeForCustomProtocol still works the same way, you basically state that you dislike the current solution (that does the work in the UIProcess), and want to </span><span style="white-space:pre-wrap" class="">have a different architecture.</span></div><div class=""><span style="white-space:pre-wrap" class=""><br class=""></span></div><div class=""><div class=""><span style="white-space:pre-wrap" class="">For custom networking or proxying you have to run the app-provided code.</span><span style="white-space:pre-wrap" class=""> The basic strategy I proposed was to run it in the app process (i.e. UIProcess). Since you don't want any networking in UIProcess, it means that the app needs to spawn a dedicated process to do custom networking. This process would run app-specific code (including NSURLProtocol-s), and communicate by IPC with the NetworkingProcess. Is this a kind of architecture you would like to have?</span><br class=""></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">It’s not yet clear what the ideal architecture is, which is one of the reasons why the mentioned issued remains unresolved.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">~Brady</div><div class=""><div class=""><br class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><div class=""><br class=""></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Fri, May 20, 2016 at 5:58 PM, Brady Eidson <span dir="ltr" class=""><<a href="mailto:beidson@apple.com" target="_blank" class="">beidson@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 20, 2016, at 2:30 AM, Daniel Olegovich Lazarenko <<a href="mailto:danielo@opera.com" target="_blank" class="">danielo@opera.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class="">Thank you for such a fast reply. That is amazing! :)<div class="">Back to questions...<br class=""><div class=""><br class=""></div><div class=""><div style="font-size:12.8px" class="">> Are you primarily focused on a custom networking layer (e.g. your own HTTP implementation?),</div><div style="font-size:12.8px" class="">> or with custom protocol handling for non-http protocols?</div><div class=""><br class=""></div><div class="">I'm primarily concerned about HTTP and friends (HTTPS, SPDY, HTTP2), but if there are any other widely used protocols that's also interesting. FTP support is not interesting for me. Do you have any other specific things in mind?</div><div class=""><br class=""></div><div class="">If there's a custom proprietary protocol that people handle in the app with their own code, for example, acme://<a href="http://acme.com:1234/" target="_blank" class="">acme.com:1234</a>, then proxying this thing is not very interesting to me, because it's very easy to proxy my own protocol handled by my own code. There's a case when "acme" is provided by some 3rd party, and the app author doesn't have the processing code. In such case it might be interesting to proxy it as well, but then again, I'm asking for a concrete example of such protocol (in WebKit context).</div></div></div></div></div></blockquote><div class=""><br class=""></div></span>In a WebKit1 app (WebView on Mac, UIWebView on iOS), app authors were able to use NSURLProtocol to override any scheme they wished.</div><div class=""><br class=""></div><div class="">While some such as yourself might’ve used it to override http from the default handling, *many more* used it to implement custom protocols. e.g. “<a class="">acme://my-app-specific-url</a>”</div><div class=""><br class=""></div><div class="">We are exploring ways to restore that full functionality - <a href="https://bugs.webkit.org/show_bug.cgi?id=138169" target="_blank" class="">https://bugs.webkit.org/show_bug.cgi?id=138169</a></div><div class=""><br class=""></div><div class=""><span class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><span style="font-size:12.8px" class="">> You seem to dismiss the Networking process’ crash recovery aspect.</span></div><div class=""><div style="font-size:12.8px" class=""><span style="font-size:12.8px" class="">></span><span style="font-size:12.8px" class=""> </span>"because in practice we know that most of the crashes happen in the WebProcess parts”.</div><div style="font-size:12.8px" class=""><span style="font-size:12.8px" class="">></span><span style="font-size:12.8px" class=""> </span>I’m curious what data you’re using to make that claim?</div><div style="font-size:12.8px" class=""><br class=""></div><div style="font-size:12.8px" class="">Well, I'm not dismissing it, it's definitely a trade off that an app author will make by choosing to enable this option.</div><div style="font-size:12.8px" class="">The data comes from our web browser apps. We certainly see networking faults, but in total it was usually a minor part of all the WebKit crashes. To not sound subjective, I've looked through our current app version, which already has enough data to judge, and in the top WebKit crashes there are none in the network code. Most are crashes in JavaScriptCore, DOM and graphics subsystems. This is the experience we have over many versions and years of service. I might be able to show you the data in private if you want, although I'm sure that you have your own crash analysis system with much more data.</div></div></div></div></blockquote><div class=""><br class=""></div></span>Without getting in to specifics, the NetworkingProcess does crash.</div><div class=""><br class=""></div><div class="">And while the WebContent process does crash way more, it usually only effects that one web page.</div><div class=""><br class=""></div><div class="">If networking code was back inside the UI Process, and it crashed, that takes down the whole browser.</div><div class=""><br class=""></div><div class="">Doing so would be reverting towards the single process architecture of yesteryear, not progressing away from it.</div><div class=""><span class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><div style="font-size:12.8px" class=""></div><div style="font-size:12.8px" class=""><span style="font-size:12.8px" class="">Let's discuss the sandboxing a little bit. First of all, and correct me if I'm wrong: I thought that there's no 3rd party code execution in the networking process,</span></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Currently, there is no 3rd party code execution in the networking process.</div><span class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><div style="font-size:12.8px" class=""><span style="font-size:12.8px" class=""> and all the JS execution happens inside the WebProcess. </span></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">That’s correct, but I’m not necessarily talking about JS.</div><span class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><div style="font-size:12.8px" class=""><span style="font-size:12.8px" class="">The code that we want to run is our own code that we control that we implement in a safe and secure manner. </span></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">The 1st party is “the Modern WebKit framework.”</div><div class="">The 3rd party is “the application’s code, as well as stuff downloaded from the Internet.</div><span class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><div style="font-size:12.8px" class=""><span style="font-size:12.8px" class="">In this sense it's no less secure. </span></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">A single process app that does whatever it wants with the networking stack is less secure to the user than a multi-process app that has well defined behavior inside certain sandboxed parts of the app.</div><span class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><div style="font-size:12.8px" class=""><span style="font-size:12.8px" class="">In the end we are always protected by the iOS/Mac sandbox. </span></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">For a WKWebView app on iOS, while the app itself is sandboxed, the Networking process is *much more* sandboxed.</div><span class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><div style="font-size:12.8px" class=""><span style="font-size:12.8px" class="">M</span><span style="font-size:12.8px" class="">aybe I'm wrong about JS here, or d</span><span style="font-size:12.8px" class="">o you have some other use case in mind?</span></div></div></div></div></blockquote><div class=""><br class=""></div></span>I think you misunderstood that I was talking about the native app code that is a 3rd party client to the WebKit framework.</div><div class=""><br class=""></div><div class="">But, there is an important JS aspect to this; Since there is messaging between the WebContent process and the Networking process, a vulnerability in the WebContent process can turn into a “remote exploit” of the Networking process.</div><div class="">Therefore it is beneficial to have the Networking process tightened down as much as possible.</div><div class=""><br class=""></div><div class="">---</div><div class=""><br class=""></div><div class="">There’s another very important reason why offering a “networking-in-UI-process mode” is not appealing. From a project maintenance standpoint, the more configurations we have the harder it is to test and develop within the project.</div><div class=""><br class=""></div><div class="">Adding this “optional” networking mode would double certain testing matrices and make development in this area (which there has been a lot of lately) more difficult.</div><div class=""><br class=""></div><div class=""><span class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><div class=""><span style="font-size:12.8px" class="">> Debugging the multi process architecture of WebKit2 </span></div><div class=""><span style="font-size:12.8px" class="">> has not gotten any harder in years, </span><span style="font-size:12.8px" class="">active developers have all adapted, </span></div><div class=""><span style="font-size:12.8px" class="">> and new developers tend to pick it up pretty quickly. </span><span style="font-size:12.8px" class="">This is not a useful point.</span></div><div class=""><span style="font-size:12.8px" class=""><br class=""></span></div><div class=""><span style="font-size:12.8px" class="">I'm sorry that you are rejecting this. Of course you can adapt to that, but it inevitably has a steeper learning curve, and takes longer time. Many app developers come from a single-process background and</span><span style="font-size:12.8px" class=""> find multi-process debugging much harder.</span><span style="font-size:12.8px" class=""> Often i</span><span style="font-size:12.8px" class="">t's a real challenge to understand what's going on. I'm sure that you in your team have multiple stories that show how non-trivial it is, and tricks about dealing with it. Nevertheless, I agree, it's not a decisive point.</span></div></div></div></div></blockquote><div class=""><br class=""></div></span>AFAIK, we haven’t had a potential contributor to the WebKit Open Source project decide to not contribute because of the multi-process architecture.</div><div class=""><br class=""></div><div class="">But, regardless, the MP architecture is primarily how well WebKit works as a browser engine for the user, and not about how easy it is for single-process-only developers to contribute.</div><div class=""><br class=""></div><div class="">Such developers can still actively contribute to the cross platform code of the project (WebCore) in single process mode using MiniBrowser or DumpRenderTree.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">~Brady<div class=""><div class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><div class=""><span style="font-size:12.8px" class=""><br class=""></span></div><div class=""><span style="font-size:12.8px" class=""><br class=""></span></div><span style="font-size:12.8px" class=""></span></div></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Thu, May 19, 2016 at 6:43 PM, Brady Eidson <span dir="ltr" class=""><<a href="mailto:beidson@apple.com" target="_blank" class="">beidson@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 19, 2016, at 8:41 AM, Daniel Olegovich Lazarenko <<a href="mailto:danielo@opera.com" target="_blank" class="">danielo@opera.com</a>> wrote:</div><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class="">I'd like to ask your for advice about implementation of a custom networking layer</div></div></div></blockquote><div class=""><br class=""></div></span>Are you primarily focused on a custom networking layer (e.g. your own HTTP implementation?), or with custom protocol handling for non-http protocols?</div><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="">...with WKWebView on iOS.</div></div></div></blockquote><div class=""><br class=""></div>WKWebView is an API that ships on both OS X and iOS. When a design aspect of it affects both platforms (such as the networking behavior), we must consider both platforms.</div><div class=""><span class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class="">Our current solution is based on NSURLProtocol, and the issues we had with it in 2014 are unresolved:</div><div class=""><a href="https://bugs.webkit.org/show_bug.cgi?id=137302" target="_blank" class="">https://bugs.webkit.org/show_bug.cgi?id=137302</a><br class=""></div><div class=""><a href="https://bugs.webkit.org/show_bug.cgi?id=138131" target="_blank" class="">https://bugs.webkit.org/show_bug.cgi?id=138131</a><br class=""></div><div class=""><br class=""></div><div class="">It was kind of a shoehorn hack, and so it was rejected by Benjamin Poulain and Alexey Proskuryakov among other reviewers.</div></div></blockquote><div class=""><br class=""></div></span><div class=""><div class="">I’m not sure it’s useful for WebKit to spend energy testing and maintaining a mechanism that *only* allows for HTTP-handling replacement and doesn’t also allow for the oft-requested feature of custom protocol handling.</div></div></div><div class=""><span class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class="">Now I'm again looking for a better solution.</div><div class=""><div class="">I'd really like to discuss it with somebody responsible, </div></div></div></blockquote><div class=""><br class=""></div></span>There is no single person responsible; the project works largely on consensus. When dealing with platform specific concerns such as this, it works on consensus of the platform owners.</div><div class=""><br class=""></div><div class="">That said, I have been the primary caretaker of the Networking process since it’s inception, as well one of the primary caretakers of Mac/iOS networking in general for many years, so I’ll share my thoughts below.</div><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><span class=""><div class="">There's currently 2 solutions I'm weighting:</div></span><div class=""><ol class=""><li class="">Pass and use NetworkProcessCreationParameters.httpProxy to NSURLSessionConfiguration (in NetworkSession and maybe other places). The httpProxy solution is easy to implement and would look clean design-wise. It would let us spawn an HTTP proxy on localhost and filter the traffic there. There might be some complications, because it's not fully transparent to the client side. For example HTTPS will have issues. All in all this could be a fine short-term solution.</li></ol></div></div></div></blockquote><div class="">While ToT WebKit contains an NSURLSession-based networking implementation for Mac/iOS, it also still contains an NSURLConnection implementation, which is unaffected by NSURLSession considerations.</div><div class=""><br class=""></div><div class="">That a solution doesn’t work on all supported platforms is not a deal breaker, but it certainly makes it less interesting than one that does.</div><div class=""><br class=""></div><div class="">HTTPS losing reliability is likely an unacceptable red flag. </div><div class=""><br class=""></div><div class=""><div class=""><div class=""><div class="">I’m not sure it’s useful for WebKit to spend energy testing and maintaining a mechanism that *only* allows for HTTP-handling replacement and doesn’t also allow for the oft-requested feature of custom protocol handling.</div></div></div></div><span class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><ol start="2" class=""><li class="">Add a new mode to the NetworkProcess, which would do all networking in UIProcess (instead of spawning a new process). A mode would be optional and controlled with some configuration setting (or NSUserDefaults).</li></ol></div></div></div></blockquote></span><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="">The UIProcess solution is harder to implement, and it will affect more code. It is somewhat controversial. One of the reasons of splitting out a NetworkProcess was to have it respawn after crashes. Nevertheless we can take this risk, because in practice we know that most of the crashes happen in the WebProcess parts.</div></div></div></blockquote><div class=""><br class=""></div><div class="">You seem to dismiss the Networking process’ crash recovery aspect. "because in practice we know that most of the crashes happen in the WebProcess parts”. I’m curious what data you’re using to make that claim?</div><span class=""><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""> I don't see any other significant downsides of having the UIProcess handling networking.</div></div></blockquote><br class=""></div></span><div class="">The Networking process provides significant benefit unrelated to crash recovery that should not be abandoned for convenience sake. e.g. Sandboxing.</div><div class=""><br class=""></div><div class="">Especially when moving the networking to the UI process would also end up moving 3rd party code execution into the UI process, this seems like an unacceptable regression.</div><span class=""><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class="">In addition it can simplify the NetworkProcess debugging.</div></div></blockquote><br class=""></div></span><div class="">Debugging the multi process architecture of WebKit2 has not gotten any harder in years, active developers have all adapted, and new developers tend to pick it up pretty quickly. This is not a useful point.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">~Brady</div><div class=""><br class=""></div></div></div></blockquote></div><br class=""></div>
</blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div><br class=""></div></div></div></blockquote></div><br class=""></div>
_______________________________________________<br class="">webkit-dev mailing list<br class=""><a href="mailto:webkit-dev@lists.webkit.org" class="">webkit-dev@lists.webkit.org</a><br class="">https://lists.webkit.org/mailman/listinfo/webkit-dev<br class=""></div></blockquote></div><br class=""></body></html>