In some embodiments, at hyper-kernel initialization time, the hyper-kernel marks all guest pages as dormant in its data structures managing and tracking placement of pages. The page tables in the hyper-kernel corresponding to dormant pages are empty in that they do not point at real, physical memory.
In some embodiments, conversion of dormant pages to “real” pages is done when the page is first read or written. In some embodiments, this conversion first looks to the pre-zero list to find a page that is pre-zeroed on the node where the conversion needs to take place. During guest OS initialization, this will usually succeed, but if there are no remaining pre-zeroed pages, in some embodiments, the hyper-kernel, acting on behalf of a vcpu, must wait until the node has one. Further details regarding obtaining of zeroed-pages will be described below. Other mechanisms in the hyper-kernel, such as the TidalTree (as described in further detail above), which may invoke memory capacity maintenance services, ensure that memory usage never reaches 100% of node capacity.
It may not be possible to know where a vcpu will be when it stalls for a missing page and discovers that the page is dormant. This means that it may not be possible to know a priori which nodes will provide pages for the guest from their pre-zeroed list, since it may not be possible to know on which node a vcpu will be when it needs that page.