The Vulkan Window System Integration (WSI) is a means of abstracting windowing from the core Vulkan API. This is useful because windowing mechanisms are platform-dependent, and it would be inconvenient to have multiple ways of creating and managing a window in the core API. Notably, since displaying an image to the viewer is not required for Vulkan implementations, the WSI is an extension. We're going to be looking mainly at one specific part of the WSI: The swapchain. According to the Vulkan spec, the WSI Swapchain is "an abstraction for an array of presentable images that are associated with a surface". In other words, it's a list of images can be displayed on a VK_KHR_surface. The presentation engine is the system that takes those images and puts them onto a display.
The number of images in the swapchain is something the user can control. We normally provide the minImageCount and maxImageCount paramters for this purpose. Common values are 2 images for double buffering and 3 images for triple buffering. However, once the swapchain is created, you cannot add or remove images.
The swapchain functions in a fairly straightforward manner. Whenever we want to display an image on the associated surface, we must:
After this, the presentation engine will asynchronously display the image on the surface. Once the presentation engine is finished with the image, it will become available for acquisition again, and a call to vkAcquireNextImageKHR() may return it.
There are several different present modes available for surfaces, and it's important to query which ones are available with vkGetPhysicalDeviceSurfacePresentModesKHR() before using one. The present mode you choose is highly dependent on your goals, but the general best practice is to go for VK_PRESENT_MODE_FIFO_KHR with triple buffering to maximize performance. You should also avoid using VK_PRESENT_MODE_MAILBOX_KHR unless you specifically need that behavior. Consult the Vulkan refpage for the different present modes.
When we want to render something onto an image on the swapchain, we need to first ask the swapchain for an available image with a vkAcquireNextImageKHR() call. The presentation engine can determine which image is going to become available next (returned as an index), but it doesn't necessarily guarantee that the image is ready to be written to as soon as we acquire that index. We must wait for the appropriate synchronization primitive before accessing it. Something else to note is that the order that image indices are acquired is not predictable.
At first glance, this seems somewhat useless. Why would we want the image index early if we're not guaranteed to be able to write to it? Why not just return the image index when it's ready?
Remember that rendering commands are only executed on the GPU after we submit them to a queue. Since recording commands into a command buffer is a CPU-side operation, this means that we can still record the rendering commands without a useable image! The command buffer still needs to know which image it's going to operate on, which is why we bother getting the index first. We can then submit those commands to the queue immediately, specifying a synchronization primitive that the queue must wait on before executing them. This allows the CPU to continue working while the GPU waits for the image to become available.
There are 2 important Vulkan synchronization primitives that we should concern ourselves with. Semaphores offer GPU-to-GPU synchronization between queue operations. We use semaphores when we want to order events that occur GPU-side. Fences offer CPU-to-GPU synchronization. We mainly use fences when we need the CPU to wait on a GPU operation to finish before continuing.
As we're looking for some sort of synchronization between 2 GPU-side events (the presentation system making an image available and the GPU rendering to that image), we can resolve this with a semaphore. Note that it's also possible to resolve this with a fence: we can tell the CPU to wait until the next image is available for use. However, this is obviously less efficient than utilizing the CPU while we wait for the image to be ready.
A binary semaphore is either in the signalled or unsignalled state at any given time. We can pass an unsignalled semaphore as an argument to the vkAcquireNextImageKHR() call, and the semaphore will be signalled when the image is ready for use. We can similarly pass the same semaphore to vkQueueSubmit() as a "wait semaphore", which means that the submitted work doesn't begin executing until that semaphore is signalled. With that, we have succeeded in creating a dependency between the two commands such that we don't render to an image until it's ready to be rendered to.
We have another synchronization issue to watch out for, and it's basically the opposite of what we just looked at. Previously, we didn't want to write to an image until it was "ready" (no longer being presented or queued for presentation). Now, we don't want to present an image until we've fully written to it. This is handled very similarly. We first pass a new, unsignalled "signal semaphore" to vkQueueSubmit() which gets signalled whenever the rendering work is finished. We then pass that same semaphore as a wait semaphore to vkQueuePresentKHR() so that it doesn't begin executing until the semaphore is signalled.
However, we run into another source of inefficiency here. The swapchain has more than one image (usually 2 when double buffering, or 3 when triple buffering) but we are currently only dealing with one image at a time. As an anlogy, consider an auto repair shop with 3 workstations (swapchain images) and a stream of broken cars (frames). What we're currently doing equates to having 1 workstation fix the first car to completion while the other workstations lay empty (even with broken cars lined up for repair).
You can imagine that it's much more efficient to have the second and third workstations start repairing the second and third cars while the first the first car is being repaired. Likewise, it's much more efficient to have multiple frames being processed at different stages of rendering and presentation rather than waiting for one frame to finish completely before starting the next. In Vulkan we can do this by having multiple "in-flight" frames at a time.
We should clarify the difference between a "frame" and an "image". An image in the swapchain is nothing more than a buffer of GPU memory abstracted as a VkImage object. An image has a specific format, dimensions, and layout. A frame is abstract, and isn't represented by a Vulkan object. A frame represents a unit of CPU work, usually one full iteration of the render loop, as well as the resources that that iteration needs.
A frame "in-flight" represents an iteration of the render loop that has been submitted to the GPU, but has not finished executing on the GPU yet. In other words, the more in-flight frames we have, the further ahead the CPU is of the GPU.
When the number of in-flight frames gets too large, we will experience a lot of latency in the application. Frames that were processed recently might take a while to actually show up on the screen because the GPU has not finished executing the commands to render and present them to the display. The other issue to consider is that frames require a certain amount of resources. For example, if we wanted N frames to be in-flight, we would then need N unique command buffers to hold all the instructions. This of course means increased memory usage. If we didn't have N unique command buffers, we would have to reuse command buffers, which would likely lead to undefined behavior as we may be overwriting a command buffer that is currently executing on the GPU.
We can achieve N in-flight frames by creating an array of size N that keeps track of all the command buffers. For each iteration of the render loop, we can use the command buffer at index (i % N) where i is the iteration number.
You'll come to realize that having N unique command buffers isn't enough: we also need unique semaphores and any other GPU resources that a frame utilizes. These can be in their own separate arrays, similar to what we did with the command buffers. However, we still need a way to guarantee that we don't start using resources that are already in-use by the GPU.
Consider the following situation: we only have 2 frames in-flight (A and B) and we constantly ping-pong between them for each iteration of the render loop. However, the CPU is much faster than the GPU execution:
Iteration 0: Submit Command Buffer A to the GPU
Iteration 1: Submit Command Buffer B to the GPU
Iteration 2: Write commands into Command Buffer A (Oops, it's still being executed on the GPU -> undefined behavior)
As you can see, we need some way to control when the CPU can use the resources for a given frame. This is done with a fence, as we only want the CPU to clear a given command buffer if we know the GPU finished executing its commands. We can accomplish this by passing in a fence when calling VkQueueSubmit2(), and then waiting on that fence at the start of the render loop. Note that since binary (wait/signal) semaphores are GPU<->GPU synchronizations, the Vulkan spec states that the GPU automatically consumes and resets them. However, fences need to be manually reset by the CPU. The execution now looks like this:
Iteration 0: Submit Command Buffer A to the GPU and signal Fence A when done executing on the GPU
Iteration 1: Submit Command Buffer B to the GPU and signal Fence B when done executing on the GPU
Iteration 2: Fence A is not signalled... waiting... Fence A signalled... reset Fence A... Write commands into Command Buffer A and submit without any problems, signal Fence A when done executing on the GPU
I've found that the following image from KDAB does the best job at depicting what we've got so far:
The vertical bars that are colored red depict the semaphores that allow ordering of the presentation and rendering capabilities. We must wait for rendering to finish before presenting an image, and we must wait for an image to finish presenting before rendering to it. Likewise, we must wait for an image to finish rendering before we start using its frame resources, such as its command buffer.
There's one last detail we should consider: how do we know that images are getting presented in order? Vulkan solves this with implicit synchronization, namely the first synchronization scope of signal semaphores. Directly from the spec: "Because the first synchronization scope for a semaphore signal operation contains all semaphore signal operations which occur earlier in submission order, all semaphore signal operations contained in any given batch are guaranteed to happen-after all semaphore signal operations contained in any previous batches. However, no ordering guarantee is provided between the semaphore signal operations defined within a single batch."
Synchronization scopes will be discussed further in another post, but what you should take from this is that since the semaphore signal operations on the second image are in a batch that comes after the first, they will necessarily occur after.
Additional resources: