Implementing Render Hardware Interface for DirectX 12 & Vulkan 1.3
Result
Sponza with textures running on Vulkan 1.3
Sponza with textures running on DirectX 12
Why Render Hardware Interface?
During our fourth project at The Game Assembly, I started working on my own renderer in DirectX 11. This was my very first renderer and it was really slow and had no real structure to it. After this engine, I wanted to try more modern APIs, so I started working on a Vulkan renderer called Titan. I noticed that many AAA games have a Graphics API Dropdown, and I thought it was really cool. I wanted to create a game engine that could use two of the most popular graphics APIs today.
When our specialization course started, I heard about Render Hardware Interface (RHI). I started researching what it was and found out that it was an incredibly awesome way to handle multiple Graphics APIs. So I decided to create an RHI that supports both Vulkan and DirectX 12.
Design
When designing my RHI, I wrote down features that would be needed for it to be useful if I were to choose the RHI as my choice of Graphics API. From my experience in the Vulkan API, I knew that it needed some features.
Support for different device queue.
API garbage collectors.
Shader compiler with reflection.
Command Buffers/Lists with unified interface.
Handle device queues.
I wanted to give users the ability to specify what type of device queues they wanted to use during rendering. Device queues are basically a way for the CPU and GPU to communicate and synchronize work, and different types of queues can have different performance characteristics depending on the workload.
After some research, I came up with a solution: adding the specified device queues to the device class. This way, users can specify the type of queues they want to use, and the RHI can create those queues during initialization.
Here's how I did it:
First, I defined the types of device queues that my RHI supports. For example, I have a graphics queue for rendering graphics, a compute queue for running compute shaders, and a transfer queue for copying data between CPU and GPU memory.
Then, I added the device queues to the device class. I did this using a struct that contains the type of queue the device supports. This way, the RHI can iterate over the device's supported queues and create the requested queues during initialization.
During RHI initialization, users can specify which device queues they want to use by passing in a bitmask of the desired queue types. The RHI then iterates over the device's supported queues, checks if the queue type is set in the bitmask, and creates the requested queues.
API Garbage collection.
Working with modern APIs can be frustrating, especially when it comes to memory management. Unlike older APIs, modern APIs require much more manual memory management, which can be time-consuming and error-prone. To address this issue, I wanted to create an RHI that allows the user to focus on rendering, without having to worry about memory management.
To achieve this, I created Allocators for both APIs that include a simple garbage collector. This garbage collector ensures that resources are automatically destroyed when they are no longer needed, without the user having to manually destroy them.
the vulkan allocator with garbage collector.
the directX 12 allocator with garbage collector.
Let's talk about Allocators. Allocators are objects that allocate memory for resources, such as textures, buffers, and shaders. They are responsible for requesting memory from the operating system, and then managing that memory for the user.
In my RHI, I created Allocators for both modern and older APIs. These Allocators allow the user to allocate memory for resources without worrying about the details of memory management. The Allocators also track the resources that are allocated, which is important for the garbage collector.
The garbage collector is a simple system that automatically destroys resources when they are no longer needed. When a resource is allocated through an Allocator, the Allocator tracks the resource. If the user does not manually destroy the resource, the garbage collector will ensure that it is destroyed safely.
This way, the user can allocate resources as needed, without worrying about destroying them later. This not only simplifies memory management, but it also helps prevent memory leaks and other memory-related issues.
HLSL shader compiler.
When it comes to rendering, shaders are essential. However, creating shaders that work across different APIs can be a challenge. I wanted to make it easier for users of my RHI to use the same shader for both DirectX 12 and Vulkan. But there's a problem: DirectX 12 can't take SPIR-V, and Vulkan can't take raw HLSL binary. So, I needed a solution.
After researching different approaches, I found a way to bridge the cross-API gap with the DirectX Shader Compiler provided by Microsoft. The DXC can compile HLSL to DXIL for DirectX 12, and can also compile HLSL to SPIR-V for Vulkan. This allowed me to create a tool that compiles HLSL shaders for both APIs.
the vulkan shader compiler sub-class.
the directX 12 shader compiler sub-class.
Shaders are typically written in HLSL for DirectX 12, and GLSL for Vulkan. However, HLSL and GLSL are not compatible with each other, and neither is SPIR-V. This means that we need a way to convert between different shader formats.
To solve this problem, I used the DirectX Shader Compiler (DXC). The DXC is a command-line tool that can be used to compile HLSL shaders to different binary formats. It includes support for compiling HLSL shaders to DXIL for DirectX 12, and HLSL shaders to SPIR-V for Vulkan.
By using the DXC, I was able to create a shader compiler that compiles HLSL shaders for both DirectX 12 and Vulkan. This Compiler takes HLSL source code as input, and produces either DXIL or SPIR-V as output, depending on the API being targeted. The compiled shader can then be loaded into the RHI pipeline and used for rendering.
But there was still one more problem to solve. I also wanted pipelines to be created without needing to specify what the shader has bound. This means that the user would not have to worry about manually describing the bound resources in the shader, which can be tedious and error-prone.
To solve this problem, I used shader reflection. Shader reflection is a technique used to extract information from a compiled shader. By using shader reflection, I could get the necessary information for creating the input assembler, root signature for DirectX 12, and the right descriptor sets for Vulkan. This allowed me to automatically generate pipeline layouts without needing any input from the user.
Streamlining Command Submitting.
Command submitting is something that has changed from previous APIs. In modern APIs like Vulkan and DirectX 12, command submitting is done by having a buffer of commands that can be processed by the GPU. This buffer acts as an instruction manual for your scene, telling the GPU what to draw and how to draw it.
For my RHI, I wanted the command buffer to be easy to understand and require no overhead. However, there was an issue. You see, Vulkan binds its resources through a descriptor set, which holds all the information about a specific bound set. But in DirectX 12, you do specific API commands to bind resources to the root signature.
To solve this problem, I added an extension to Vulkan 1.3 called “VK_KHR_push_descriptor”. This extension allows my command buffer to have a similar way of submitting bound resources in both Vulkan and DirectX 12. By using push descriptors, I was able to simplify command submiting resouces in my RHI.
the vulkan command buffer binding with the push descrptor a constant buffer.
the directX 12 command buffer binding with the rootconstant a constant buffer.