SharePoint, VirtualPathProviders and Application Restarts

Given that the only way to unload dynamically compiled assemblies (to reclaim memory) is to unload the app domain, how does SharePoint rely on VirtualPathProviders, for master pages and page layouts in particular, without bumping into this limitation?

The restart can be delayed through various settings but not avoided completely when master pages and page layouts are updated and published frequently, correct?

(Is the lack of info on this attributed to it being a more theoretical limit that's not common in publishing patterns? Have you personally noticed the rate of changes to master pages or layouts causing app instability? Should SharePoint come with a warning?)

Any CMS-esque capability that leverages dynamic WebForms (including, by default, MVC views) is susceptible to rate-of-change instability, correct?

Update on No-Compile Pages:

No-compile pages In ASP.NET 2.0, the compilation model has been significantly refactored and extended. Site precompilation is perhaps the most popular and loudly requested of the new features. Another quite interesting feature is no-compile pages. They are special pages that just never get compiled. So what’s the ultimate purpose of no-compile pages, and what’s the difference between them and static HTML pages? To start off, you create a no-compile page by setting the CompilationMode attribute on the @Page directive to Never. When a no-compile page is requested, no page assembly is created and persisted to disk. Instead, an instance of the page builder component is cached in memory and used to create the page output for each and every request. The page builder is a special component that supports the page parser in building the page control tree. When compilation is turned on, the control tree is used to obtain a class to compile. When compilation is off, the control tree is used to obtain markup. Needless to say, classes are necessary if you want to give programmers the power of attaching their own code to the page. No-compile pages are made of server controls and literals but contain no code at all.

No-compile pages are not for every application. They are exclusively designed for improving the scalability on very large web sites with thousands of pages. No-compile pages can’t be bound to a code file and can’t contain a server-side block. The only executable piece of code allowed in a no-compile page are $-expressions. There are two main benefits out of no-compile pages. In a secure environment like SharePoint, no-compile pages prevent developers from writing potentially buggy code that can cause problems to the hosting environment and even tear it down. In a large content-based web site, no-compile pages avoid the need to compile thousands of pages.

References:

1 – http://haacked.com/开发者_C百科archive/2009/04/22/scripted-db-views.aspx

First thing to note is that customized pages (Could be masterpages or page layouts) are always stored in the database.

The page request cycle differs from the vanilla ASP.net version in the way SPVirtualPathProvider routes requests. Once it's found the request is made for a customized page (as opposed to an uncustomized page sitting on the file system, and subject to the usual ASP.net page compilation mode) the page's code is pulled from the database, handed off to the ASP.net runtime, and parsed in "no-compile mode."

Here's a visual rendition of the process when a request for a customized page is made:

Here is also good description of ASP.net's no compile mode

No Compile Pages:

No Compile Pages enables improved scaling for large sites with 1000s of pages, as windows has limit on number of DLLs loaded into an app and perf degrades as you hit this limit. Set the <%@ Page CompilationMode="Auto" %> directive to compile conditionally to obtain the scaling benefits without limitations on code. You can also set CompilationMode to "Never" to prevent the page from ever being compiled. You can set this property on the <pages/> section in Web.config to apply to all pages in an application. A no-compile page will throw an error when it contains user code.

So this basically addresses the issue you mentioned with regards to excessive application resets when unloading/loading an app domain due to multiple customizations happening in real-time (an issue any CMS built on the ASP.net runtime would have to address).

On top of this; you get the multiuser capabilities and scalability of SQL Server "for free" when storing customized content in this way; as opposed to a file system approach which would necessitate extra effort to manage locking.

Found a nice explanation of this 1:

As a developer, your initial reaction to this might be to question why customized pages are processed in no-compile mode. Your instincts likely tell you that compiled pages run faster than no-compile pages. However, no-compile pages can be more efficient and more scalable in certain scenarios. This is especially true in a large WSS environment where the number of customized pages can reach into the thousands or tens of thousands.

No-compile pages can be loaded into memory and then unloaded in a manner that is not possible for compiled pages because the .NET Framework doesn’t really support the concept of unloading an assembly DLL from memory. The closest equivalent would be to recycle the current Windows process or the current .NET AppDomain. However, this type of recycling involves unloading all assembly DLLs from memory, not just those assembly DLLs that haven’t been used recently. Furthermore, the .NET Framework places an upper limit on the number of assembly DLLs that can be loaded into a .NET AppDomain.

No-compile pages provide higher levels of scalability because they do not require loading new assembly DLLs or managed classes into memory. Instead, the processing of no-compile pages involves loading control trees into memory. WSS can manage the memory usage for the control trees associated with customized pages more efficiently because they are not compiled into assembly DLLs. For example, once WSS has finished processing a customized page, it can unload the page’s control tree to free up memory for other purposes. Furthermore, no-compile pages eliminate the need to go through the compilation process, which actually provides faster response times for pages upon first access.

They key takeway is simply that no-compile pages can be unloaded (the associated page builder can be unloaded), which isn't possible with compiled pages. At its core though, this is a scalability measure (more pages can be handled under this model) and not a performance enhancement (after initial setup, compiled pages should perform better than their non-compiled counterparts).

1 - http://chiragrdarji.wordpress.com/2007/10/12/page-ghosting-unghosting-and-effect-of-pageghosting-on-performance-in-sharepoint-2007/