basysKom AnwendungsEntwicklung

Hello, RHI – How to get started with Qt RHI
Essential Summary
For some time now, Qt has been internally utilizing RHI (Rendering Hardware Interface), a new cross-platform technology for graphic rendering. Since Qt 6.6, this API has been semi-public, meaning that the API is mature for practical use but may still be subject to potential changes between major Qt versions. In this blog post, we demonstrate how to to get started with RHI.
Professionelle Entwicklung Ihrer HMI

Sie benötigen ein Geräte-HMI, Ihnen fehlt es aber an Zeit oder speziellem HMI-Know-How?​

Warum Dienstleistungen von basysKom?

Von der Konzeption über die Implementierung bis zum Testen unterstützen wir Sie in der Entwicklung Ihrer individuellen HMI. Unsere Services umfassen zudem die Technische Beratung, individuelle Trainings, Coaching, Verstärkung Ihrer Entwicklungsteams bis hin zur vollständigen Auftragsabwicklung im gesamten Lebenszyklus Ihres Produktes.

A bit of history

More than a decade ago, Qt solely focused on OpenGL to deliver hardware accelerated graphics. That was the time of QGLWidget to use OpenGL in QWidget applications. The second generation of QtQuick has been built on an OpenGL scenegraph to render scenes written in QML.

But the times have changed: OpenGL could never truly succeed on Windows; instead Microsoft has further developed and pushed Direct3D. Apple introduced the API Metal as the canoncial way to render graphic scenes on its operating systems while deprecating OpenGL at the same time. The Khronos consortium – the same consortium that is responsible for OpenGL – unveiled Vulkan in the year 2016. All of the new graphics APIs have in common that theyoptimize GPU utilization more effectively by significantly reducing overhead.

Meet RHI

Qt’s answer to the fragmented landscape is RHI (Rendering Hardware Interface): RHI delivers a low-level API that makes it possible to write code once and target all the different graphic backends:
  • OpenGL ES
  • Metal
  • Direct3D 11
  • Direct3D 12
  • Vulkan

The API has been available since the late Qt5 days, and it has become publicly accessible starting from Qt 6.6. As RHI is very low level, even drawing something as simple as a single triangle requires some amount of code. If you just want to add a shader to your QtQuick programm or light and display a 3D object, you may consider other solutions (like a ShaderEffect in QtQuick or Qt 3D for a 3D scene). But if you need full control to implement your ideas, RHI may be a helpful API in your toolbox.

For example, we used RHI to implement the RiveQtQuickPlugin.

This blog post will guide you step by step through a basic demo application: We will create a QWindow and initialize a rendering pipeline to render some basic geometry to it.

The building blocks

In its most basic form, an application needs the following building blocks to render something into a window:

At initialization time:

  • An instance of a QRhi object: This is the basic class that helps to setup everything:
  • A QRhiSwapChain: A swap chain represents the connection between the API and the window surface.
  • Some buffers to store the geometries and inputs to the shaders. When using RHI, you use the class QRhiBuffer to store this kind of data. A QRhiResourceUpdateBatch is used to move the data to memory accessible by the GPU
  • A rendering pipeline that defines what to do with the aforementioned buffers. A pipeline consists of shader stages and information about how the vertex data is structured. You also need to define what to do with the rendering result. In detail, the following must be considered
    • To define how the geometry is laid out, use QRhiVertexInputLayout: Does the geometry buffer just store vertex positions? Does it also include texture coordinates and per vertex colors? Is the data distributed over multiple buffers or stored in one buffer? This defines how the geometry buffer is used as the input for the vertex shader (see below).
    • You need at least two shader stages: A vertex shader and a fragment shader. The vertex shader takes the vertices and transforms them to so-called clip space positions. The fragment shader determines how the primitive is filled.
    • You need to inform the pipeline about uniforms and textures. Uniforms are inputs that remain valid throughout all invocations of the shader (for example screen resolutions, time steps, etc.). This is done with QRhiShaderResourceBindings
    • Last but not least the pipeline needs to know something about where to render into. In our example, the triangle is rendered via the swap chain to the window. But you can also render to an offscreen texture. Besides one or multiple color buffers (the image that is rendered), also a depth and/or stencil buffer may be attached to the target. This is where QRhiRenderPassDescriptor comes into place. In RHI the descriptor is created from the swap chain or the render target.

During rendering:

  • Rendering of a frame is done between beginFrame() and endFrame()
  • QRhiCommandBuffer: A command buffer needs to be created during rendering: Modern architectures collect rendering commands to move them to the GPU in one batch. For architectures that don’t support command buffers, RHI emulates the behavior
  • Rendering of a single pass is done between beginPass() and endPass(). Hereby, rendering to a frame may consist of multiple passes. For example you may decide to render different parts of your scene to different offscreen textures and blend them together in a last pass.
  • Set a graphics pipeline you configured during initialization
  • Set the viewport size
  • Set your input geometry
  • Call draw() to render the geometry

So much for the theory. The source code in the next section demonstrates how all of this fits together by drawing a triangle to the screen:

Walkthrough

We begin by implementing the main.cpp:

#include <QGuiApplication>
#include <QLoggingCategory>
 
#include "hellotriangle.h"
 
 
int main(int argc, char **argv)
{
    QGuiApplication app(argc, argv);
    QLoggingCategory::setFilterRules(QLatin1String("qt.rhi.*=true"));
     
    HelloTriangle window;
    window.resize(600, 600);
    window.setTitle(QCoreApplication::applicationName());
    window.show();
 
    int retVal = app.exec();
 
    if (window.handle()) {
        window.releaseSwapChain();
    }
 
    return retVal;
} 

Here, we just create a QGuiApplication and the HelloTriangle window. When the logging category ist set to the qt.rhi domain, Qt prints some debug messages about the graphics driver on the console. The HelloTriangle class is implemented below:


First let’s have a look into the header file:

#pragma once
 
#include <QWindow>
#include <QVector>
 
#include <private/qrhi_p.h>
 
class QExposeEvent;
class QEvent;
 
class HelloTriangle : public QWindow
{
 
  Q_OBJECT
 
public:
  HelloTriangle(QWindow *parent = nullptr);
  ~HelloTriangle();
 
  void releaseSwapChain();
 
public slots:
  void render();
 
protected:
  void exposeEvent(QExposeEvent *exposeEvent) override;
  bool event(QEvent *event) override;
 
 
private:
  void maybeInitialize();
  QShader loadShader(const QString &filename);
 
 
  bool m_isInitialized { false };
  bool m_maybeResizeSwapChain { false };
 
  QRhi *m_rhi { nullptr };
  QRhiSwapChain *m_swapChain { nullptr };
 
  QRhiRenderPassDescriptor *m_renderPassDescriptor { nullptr };
  QRhiResourceUpdateBatch *m_initialUpdates { nullptr };
 
  QRhiBuffer *m_vertexBuffer { nullptr };
  QRhiShaderResourceBindings *m_shaderResourceBindings { nullptr };
  QRhiGraphicsPipeline *m_renderingPipeline { nullptr };
 
 
  QVector<QRhiResource *> m_releasePool;
 
}; 
And below is the implementation, where the interesting parts are happening. We added number markers as comments to guide you step by step through the code:
#include "hellotriangle.h"
 
#include <QCoreApplication>
#include <QExposeEvent>
#include <QPlatformSurfaceEvent>
#include <QEvent>
#include <QFile>
#include <QOffscreenSurface>
 
#if QT_VERSION >= QT_VERSION_CHECK(6, 6, 0)
  #include <QRhi>
#else
  #include <private/qrhigles2_p.h>
#endif
 
static float vertexData[] = {
    -0.5f, -0.5f, 0.0f,
     0.5f, -0.5f, 0.0f,
     0.0f,  0.5f, 0.0f
};
 
HelloTriangle::HelloTriangle(QWindow *parent)
    : QWindow(parent)
{
    setSurfaceType(OpenGLSurface);                                                 // 1
}
 
HelloTriangle::~HelloTriangle()
{
    qDeleteAll(m_releasePool);                                                     // 2
    m_releasePool.clear();
 
    delete m_rhi;
    m_rhi = nullptr; 
}
 
QShader HelloTriangle::loadShader(const QString &filename)                         // 3
{
    QFile shaderFile(filename);
    bool success = shaderFile.open(QIODevice::ReadOnly);
    if (success) {
        return QShader::fromSerialized(shaderFile.readAll());
    }
 
    qWarning() << "Cannot open shader file:" << filename;
    return QShader();
}
 
 
void HelloTriangle::exposeEvent(QExposeEvent *exposeEvent)                        // 4
{
    if (isExposed()) {
        m_maybeResizeSwapChain = true;
        maybeInitialize();
        render();
    }
 
    return QWindow::exposeEvent(exposeEvent);
}
 
bool HelloTriangle::event(QEvent *event)                                          // 5
{
     
    switch (event->type()) {
    case QEvent::UpdateRequest:
        render();
        break;
    case QEvent::PlatformSurface:
        if (static_cast<QPlatformSurfaceEvent *>(event)->surfaceEventType() == QPlatformSurfaceEvent::SurfaceAboutToBeDestroyed) {
            releaseSwapChain();
        }
        break;
    }
    return QWindow::event(event);
}
 
void  HelloTriangle::maybeInitialize()                                           // 6
{
    if (m_isInitialized) {
        return;
    }
 
    const int sampleCount = 1;
     
    // init RHI and swap chain to render into a window
    QRhiGles2InitParams params;                                                 // 7
    params.window = this;
    QOffscreenSurface *fallbackSurface = QRhiGles2InitParams::newFallbackSurface();
    params.fallbackSurface = fallbackSurface;
 
    m_rhi = QRhi::create(QRhi::OpenGLES2, &params);
     
    m_swapChain = m_rhi->newSwapChain();                                       // 8
    m_releasePool << m_swapChain;                                              // 9
 
    m_swapChain->setWindow(this);                                              // 10
    m_swapChain->setSampleCount(sampleCount);
     
    m_renderPassDescriptor = m_swapChain->newCompatibleRenderPassDescriptor(); // 11
    m_releasePool << m_renderPassDescriptor;
     
    m_swapChain->setRenderPassDescriptor(m_renderPassDescriptor);
    m_swapChain->createOrResize();
 
    // init vertex data
    m_initialUpdates = m_rhi->nextResourceUpdateBatch();                       // 12
 
    m_vertexBuffer = m_rhi->newBuffer(QRhiBuffer::Immutable, QRhiBuffer::VertexBuffer, sizeof(vertexData));     // 13
    m_vertexBuffer->create();
    m_releasePool << m_vertexBuffer;
    m_initialUpdates->uploadStaticBuffer(m_vertexBuffer, 0, sizeof(vertexData), vertexData);
 
    // define vertex layout                                                   // 14
    QRhiVertexInputLayout vertexInputLayout;
    vertexInputLayout.setBindings({ { 3 * sizeof(float) } });
    vertexInputLayout.setAttributes({
        { 0, 0, QRhiVertexInputAttribute::Float3, 0 },
    });
 
    // there must be a QRhiShaderResourceBindings
    // the hello triangle example does not have any uniforms and textures as shader inputs, so we leave it empty
    m_shaderResourceBindings = m_rhi->newShaderResourceBindings();           // 15
    m_releasePool << m_shaderResourceBindings;
    m_shaderResourceBindings->create();
 
    // init rendering pipeline
    m_renderingPipeline = m_rhi->newGraphicsPipeline();                      // 16
    m_releasePool << m_renderingPipeline;
    m_renderingPipeline->setShaderStages({
        { QRhiShaderStage::Vertex, loadShader(QLatin1String(":/shaders/hellotriangle.vert.qsb")) },
        { QRhiShaderStage::Fragment, loadShader(QLatin1String(":/shaders/hellotriangle.frag.qsb")) }
    });
    m_renderingPipeline->setTopology(QRhiGraphicsPipeline::Triangles);
    m_renderingPipeline->setVertexInputLayout(vertexInputLayout);
    m_renderingPipeline->setShaderResourceBindings(m_shaderResourceBindings);
    m_renderingPipeline->setRenderPassDescriptor(m_renderPassDescriptor);
    m_renderingPipeline->create();
 
    // request next frame
    m_isInitialized = true;
 
}
 
void  HelloTriangle::render()
{
    if (!isExposed() || !m_isInitialized || !m_swapChain) {
        return;
    }
 
    if (m_swapChain->currentPixelSize() != m_swapChain->surfacePixelSize() || m_maybeResizeSwapChain) {         // 17
        bool resizeDidWork = m_swapChain->createOrResize();
        if (!resizeDidWork) {
            return;
        }
        m_maybeResizeSwapChain = false;
    }
 
    const QSize outputPixelSize = m_swapChain->currentPixelSize();
 
    m_rhi->beginFrame(m_swapChain);                                                                             // 18
    QRhiCommandBuffer *commandBuffer = m_swapChain->currentFrameCommandBuffer();                                // 19
    QRhiResourceUpdateBatch *updateBatch = m_rhi->nextResourceUpdateBatch();                                    // 20
 
    if (m_initialUpdates) {
        updateBatch->merge(m_initialUpdates);
        m_initialUpdates->release();
        m_initialUpdates = nullptr;
    }
 
    commandBuffer->beginPass(m_swapChain->currentFrameRenderTarget(), Qt::black, { 1.0, 0}, updateBatch);      // 21
    commandBuffer->setGraphicsPipeline(m_renderingPipeline);                                                   // 22
    commandBuffer->setViewport({ 0, 0, float(outputPixelSize.width()), float(outputPixelSize.height()) });
    commandBuffer->setShaderResources();
    const QRhiCommandBuffer::VertexInput vertexInputBinding(m_vertexBuffer, 0);                                // 23
    commandBuffer->setVertexInput(0, 1, &vertexInputBinding);
    commandBuffer->draw(3);                                                                                    // 24
    commandBuffer->endPass();
     
    m_rhi->endFrame(m_swapChain);                                                                              // 25
 
    requestUpdate();                                                                                           // 26
 
}
 
void HelloTriangle::releaseSwapChain()                                                                         // 27
{
    if (m_swapChain) {
        m_swapChain->destroy();
    }
} 

To break down the source code piece by piece, here is what happens:

1) For didactic purposes, we simplify the code by hardcoding the window’s surface type to OpenGLSurface. In a real world application, you most probably want to be more flexible and determine the surface type from the platform on which the application has been compiled

#if defined(Q_OS_WIN)
    setSurfaceType(Direct3DSurface); ;
#elif defined(Q_OS_MACOS) || defined(Q_OS_IOS)
    setSurfaceType(MetalSurface);
#elif defined(Q_VULKAN)
    setSurfaceType(VulkanSurface);
#else
    setSurfaceType(OpenGLSurface);
#endif 

2) The destructor: It is a common practice to store all RHI resources in a list for convenient batch deletion. QRhi does not inherit from QRhiResources, so we handle its deletion separately.

3) A helper function to load shaders. A shader must be distributed in the QSB format – a container format to store (compiled) shader code for different graphics APIs. See also the next chapter.

4) Overrides the QWindow::exposeEvent method. When the window is first exposed, we need to initialize the rendering pipeline. This event is also triggered, when the user resizes the window. In this case, we may have to resize the swap chain later in the rendering code, which is indicated by the m_maybeResizeSwapChain flag.

5) Overrides QWindow::event: We listen to update and platform surface events. Whenever there is an update request, a render step is triggered; when the window is closed, we get notified, that the surface is about to get destroyed, and we are going to destroy the swap chain.

6) The initialization code; everything that needs to get done before the first call to render

7) Create the central QRhi object; depending on the platform, the initialization process is different. The RHI backend must match the surface type of the window. For the different initialization parameters refer to the documentation.

8) The swap chain connect RHI and the windows

9) To avoid deleting all ressources separately, we just append them to a release pool list

10) Initialize the swap chain: A sample count value above 1 enables Multi-Sample Antialiasing (MSAA). Don’t forget to set the window, otherwise the application will crash at startup.

11) The renderpass descriptor is essential for the render pipeline that we are going to create in a few more steps. When the pipeline renders the triangle, it must know what to do with the results, i.e. which buffers to use for storing color, depth and stencil values, and their respective types. In RHI, you create compatible descriptors from the swap chain or texture targets. After creating a descriptor, it must be set for the swap chain or the texture target, as RHI doesn’t handle this automatically. In this example, the pipeline will only render into a color buffer; there is no attached depth or stencil target.

12) RHI batches upload operations; here we create an initial upload batch that we are going to commit when rendering the first frame

13) here, we create a buffer to store the vertices for the triangle. The buffer is static, i.e. we don’t update the vertex data between draw calls.

14) From the outside, the vertex buffer contains just numbers. The pipeline needs to know about the structure of the buffer. In other words, we need a mapping between the data format and the attributes we declare in the vertex shader. As a reminder, this ist the triangle’s geometry:

static float vertexData[] = {
-0.5f, -0.5f, 0.0f,
0.5f, -0.5f, 0.0f,
0.0f, 0.5f, 0.0f
};  
There a three points with an x-, y- and z-coordinate. For each point the vertex shader is invoked once. The input to the vertex shader is declared in the following fashion:
layout(location = 0) in vec3 position; 
Now the declaration of the QRhiVertexInputLayout should make more sense:
// define vertex layout
QRhiVertexInputLayout vertexInputLayout;
vertexInputLayout.setBindings({ { 3 * sizeof(float) } });
vertexInputLayout.setAttributes({
    { 0, 0, QRhiVertexInputAttribute::Float3, 0 },
}); 
We have one attribute. The first 0 in the attribute definition is the binding point for the vertex buffer. The second 0 is the location (the same location we declared in the vertex shader) of the attribute. The attribute expects a vector consisting of three floats (Float3) with an offset of 0 in the vertex buffer. So, what happens if we need also a texture coordinate next to the vertex position?
// geometry with position and texture coordinate (x, y, z, u, v)
static float vertexData[] = {
-0.5f, -0.5f, 0.0f, 0.0f, 0.0f,
0.5f, -0.5f, 0.0f,  1.0f, 0.0f,
0.0f, 0.5f, 0.0f,   0.5f, 1.0f,
};
 
// vertex shader input
layout(location = 0) in vec3 position;
layout(location = 1) in vec3 texCoord;
 
// define vertex layout in C++
    QRhiVertexInputLayout vertexInputLayout;
    vertexInputLayout.setBindings({ { 5 * sizeof(float) } });
    vertexInputLayout.setAttributes({
        { 0, 0, QRhiVertexInputAttribute::Float3, 0 },
        { 0, 1, QRhiVertexInputAttribute::Float2, quint32(3 * sizeof(float))},
   }); 

Now, each vertex consists of five numbers: x, y, z positions and the u, v texture coordinates.  The vertex shader gets a second attribute with the location 1. Each invocation fetches five numbers from the buffer where the first three numbers form the input of the first attribute, and the next two numbers form the input of the second attribute.The binding point for both attributes is 0, as they are fed from the same buffer at binding point 0.

15) In the basic triangle example, the shaders don’t get any uniforms or textures as input. Nevertheless, the pipeline expects a shader resource binding object. Here, we just create one without setting any bindings on it.

16) Finally, we are prepared to create the pipeline. A pipeline must have at least one vertex shader and one fragment shader. The pipeline needs to know how the input is laid out (vertex attributes and shader resources like uniforms and textures), and where to render the results (render pass descriptor).

17) After initialization and setting up at least one rendering pipeline, we are ready to render the triangle. First, we check whether the swap chain needs to be resized (for example, when the user resizes the window and triggers an expose event)

18) Rendering a frame happens always between a call to beginFrame() and endFrame()

19) Modern rendering APIs collect all rendering commands in a rendering buffer, that is commited to the GPU in one batch

20) Here, we request a new update batch from RHI that can be used to update any dynamic buffers like uniform buffers. Before rendering the first frame, we merge the initial updates with the new update batch. We pass the update batch as a parameter to beginPass(), so that RHI can commit the batch.

21) A render pass is executed. A render pass renders to a specific target like a texture or the swap chain. In this case, we render to the window via the swap chain. We set the clear color to black and initialization values for the depth and stencil buffers to 1.0 and 0 respectively (note, that depth and stencil buffers do not exist in the example, but it is not possible to omit the values).

22) During one render pass, we may use multiple rendering pipelines to render to the same render target. For this example, we set the rendering pipeline responsible for the triangle.

23) Set the vertex input, and bind the vertex buffer to binding point 0

24) Draw three points

25) Request a new frame. Since the triangle is not animated, the line can be commented out. Qt ensures that update events triggered with requestUpdate() are in sync with the display vsync signal of your monitor, if possible.

Et voilà, after executing these 25 steps, the application renders a triangle:

Hello, RHI - How to get started with Qt RHI 1 basysKom, HMI Dienstleistung, Qt, Cloud, Azure

The shader code

The graphics pipeline must have at least two programmable shader stages: a vertex shader stage to process vertex geometry and a fragment shader stage to colorize the geometry. For the triangle, we are using a simple passthrough vertex shader that doesn’t do any transformations. In most applications, you most probably want to multiply the vertex with a model view matrix and a projection matrix.

#version 440
 
layout(location = 0) in vec3 position;
 
out gl_PerVertex { vec4 gl_Position; };
 
void main()
{
    gl_Position = vec4(position, 1.0);
} 
The fragment shader outputs a red color for each pixel inside the triangle:
#version 440
 
layout(location = 0) out vec4 fragColor;
 
void main()
{
    fragColor = vec4(1.0, 0.0, 0.0, 1.0);
} 
How does it work to distribute the shader code to different backend APIs? Each API comes with its own shader language: Direct3D uses HLSL, Metal uses MSL, and when using OpenGL, you would write your shaders in GLSL. The answer to the question is to write the shader code in modern GLSL, compile it to an intermediate format called SPIR-V and transpile it to an end result suitable for the different backends. Fortunately, Qt comes with a tool called qsb (Qt Shader Baker) to do the work for us and to embed the results for all the different API backends in a single file. The tool can easily be included in your build system when you are using cmake. To manually transpile the shader given in the above example, use the command:
qsb --glsl "100 es,120,150" --hlsl 50 --msl 12 -o hellotriangle.vert.qsb hellotriangle.vert
qsb --glsl "100 es,120,150" --hlsl 50 --msl 12 -o hellotriangle.frag.qsb hellotriangle.frag 
This makes it comfortable to write your shader code once, and use it on different target platforms.

Conclusion

We have illustrated the process of setting up a graphics pipeline with RHI to render hardware-accelerated content on a variety of platform-specific APIs, including OpenGL, DirectX 11 & 12, Metal, and Vulkan. After reading the blog post, you should be able to upload and render geometry onto a window.

Of course, in real world applications you might want to render onto a QWidget or within a QtQuick application. Many shaders receive uniform buffers and textures as input.

But this is not all: RHI enables more complex tasks, such as utilizing tesselation, geometry or even compute shader stages, configuring stencil buffers and scissor rectangles, or rendering to offscreen buffers. In upcoming blog posts, we’ll delve deeper into these technologies.

Please let us know in the comments, if you have already worked with RHI and in which use cases you are interested.

Picture of Berthold Krevert

Berthold Krevert

Since over 10 years, Berthold Krevert works as a senior software developer here at basysKom GmbH. He has a strong background in developing user interfaces using technologies like Qt, QtQuick and HTML5 for embedded systems and industrial applications. He holds a diploma of computer science from the University of Paderborn. During his studies he focused - amongst other topics - on medical imaging systems and machine learning. His interests include data visualisation, image processing and modern graphic APIs and how to utilize them to render complex UIs in 2d and 3d.

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.

Weitere Blogartikel

basysKom Newsletter

We collect only the data you enter in this form (no IP address or information that can be derived from it). The collected data is only used in order to send you our regular newsletters, from which you can unsubscribe at any point using the link at the bottom of each newsletter. We will retain this information until you ask us to delete it permanently. For more information about our privacy policy, read Privacy Policy