(Vulkan) CmdSetVertexBuffers required even if pipeline is created without a vertex input (RENAMED)

[Marlax0]

I apologize for the terrible title. I cannot think of a decent summary.
It's probably best if I just describe a simplified scenario.
For context, I'm porting a D3D12-based renderer, and this all works as I expect it to when using D3D12 for NRI's API.

I'm creating a nri::Buffer for a mesh that contains 2 blocks of data: (In reality I use a few more.)

1. Indices
2. Positions

So the vertex shader looks something like this:
NRI_RESOURCE(StructuredBuffer<float3>, Positions[], t, 0, 0);

nri::DescriptorRangeDesc rangeDesc = {};
rangeDesc.descriptorNum = 4096;
rangeDesc.descriptorType = nri::DescriptorType::STRUCTURED_BUFFER;
rangeDesc.shaderStages = nri::StageBits::VERTEX_SHADER;
rangeDesc.flags = nri::DescriptorRangeBits::VARIABLE_SIZED_ARRAY | nri::DescriptorRangeBits ::PARTIALLY_BOUND;

Since I need to a create a 'Positions' view, I don't give the nri::BufferDesc a structure stride.

nri::BufferDesc bufferDesc = {};
bufferDesc.size = indicesSize + positionsSize;
bufferDesc.usage = nri::BufferUsageBits::INDEX_BUFFER | nri::BufferUsageBits::SHADER_RESOURCE;

nri::Buffer* buffer = nullptr;
NRI_ABORT_ON_FAILURE(NRI.CreateBuffer(*device, bufferDesc, buffer));

nri::BufferViewDesc bufferViewDesc = {};
bufferViewDesc.buffer = buffer;
bufferViewDesc.size = positionsSize;
bufferViewDesc.offset = indicesSize; // Alignment omitted for brevity.
bufferViewDesc.format = nri::Format::RGB32_SFLOAT;
bufferViewDesc.viewType = nri::BufferViewType::SHADER_RESOURCE;

nri::Descriptor* bufferView = nullptr;
NRI_ABORT_ON_FAILURE(NRI.CreateBufferView(bufferViewDesc, bufferView));

However, the subsequent call to nri::CoreInterface::UpdateDescriptorRanges() results in a VK error:

Validation Error: [ VUID-VkWriteDescriptorSet-descriptorType-00331 ] Object 0: handle = 0xe88693000000000c, type = VK_OBJECT_TYPE_BUFFER; | MessageID = 0xde8c9ed7 | vkUpdateDescriptorSets(): pDescriptorWrites[0].pBufferInfo[0].buffer was created with VK_BUFFER_USAGE_2_TRANSFER_SRC_BIT|VK_BUFFER_USAGE_2_TRANSFER_DST_BIT|VK_BUFFER_USAGE_2_UNIFORM_TEXEL_BUFFER_BIT|VK_BUFFER_USAGE_2_INDEX_BUFFER_BIT|VK_BUFFER_USAGE_2_SHADER_DEVICE_ADDRESS_BIT, but descriptorType is VK_DESCRIPTOR_TYPE_STORAGE_BUFFER.
The Vulkan spec states: If descriptorType is VK_DESCRIPTOR_TYPE_STORAGE_BUFFER or VK_DESCRIPTOR_TYPE_STORAGE_BUFFER_DYNAMIC, the buffer member of each element of pBufferInfo must have been created with VK_BUFFER_USAGE_STORAGE_BUFFER_BIT set (https://vulkan.lunarg.com/doc/view/1.4.304.0/windows/1.4-extensions/vkspec.html#VUID-VkWriteDescriptorSet-descriptorType-00331)

It seems not providing a structure stride causes the nri::BufferVK to be created without the VK_BUFFER_USAGE_STORAGE_BUFFER_BIT flag.

I don't really know much about Vulkan. So I can't say whether or not NRI is doing anything wrong.
Is my strategy not viable under Vulkan? Is there another way I should handle this?

Full example code here.
(You can just paste it into Sample::Initialize() in any of the sample projects...)

[Marlax0]

I see ultimately I'm going to need to call nri::CoreInterface::CmdSetVertexBuffers(), which I am not using at all under D3D12.

Validation Error: [ VUID-vkCmdDrawIndexed-pStrides-04913 ] Object 0: handle = 0x23393ae5680, type = VK_OBJECT_TYPE_COMMAND_BUFFER; Object 1: handle = 0xbfab3700000000c7, type = VK_OBJECT_TYPE_PIPELINE; | MessageID = 0x994fe9d9 | vkCmdDrawIndexed(): VK_DYNAMIC_STATE_VERTEX_INPUT_BINDING_STRIDE state is dynamic, but the command buffer never called vkCmdBindVertexBuffers2.
The Vulkan spec states: If the bound graphics pipeline was created with the VK_DYNAMIC_STATE_VERTEX_INPUT_BINDING_STRIDE dynamic state enabled, but without the VK_DYNAMIC_STATE_VERTEX_INPUT_EXT dynamic state enabled, then vkCmdBindVertexBuffers2 must have been called and not subsequently invalidated in the current command buffer prior to this draw command, and the pStrides parameter of vkCmdBindVertexBuffers2 must not be NULL (https://vulkan.lunarg.com/doc/view/1.4.304.0/windows/1.4-extensions/vkspec.html#VUID-vkCmdDrawIndexed-pStrides-04913) [Line: 1424] [File: DeviceVK.hpp]

I guess this is what I should be doing instead?

This is quite a big change from my bindless system, but I suppose if it's necessary...

Update:
I also have per-object OBJECT_DATA and MATERIAL_DATA, stored together in a single nri::Buffer, with multiple views.
Nearly identical setup as the example buffer, minus nri::BufferUsageBits::INDEX_BUFFER. Same VK error.
So regardless of switching to nri::CoreInterface::CmdSetVertexBuffers(), I don't understand this error, and what I am doing wrong.

[dzhdanNV]

I'm very busy today and about to leave (karting race). Quick and incomplete response:

• in VK VK_BUFFER_USAGE_STORAGE_BUFFER_BIT is mandatory at creation time if a buffer will be used as a structured buffer
  (https://github.com/NVIDIA-RTX/NRI/blob/main/Source/VK/DeviceVK.hpp#L39)
• do you really need structured? It's not needed if a buffer is used as vertex input only (still can be used as STORAGE/UAV as a typed buffer)
• could you briefly explain your bindless system, please?

(I will answer only tomorrow)

[Marlax0]

I'm currently only using bindless for vertex data and material textures.
The actual shader is like this:

NRI_RESOURCE(StructuredBuffer<float3>, Positions[], t, 0, 3);
NRI_RESOURCE(StructuredBuffer<VERTEX_ELEMENTS>, Elements[], t, 0, 4);
NRI_RESOURCE(Texture2D, Textures[], t, 0, 5);

I'll describe the bindless Textures[]. It's all the same idea anyway.

I basically just create a big nri::DescriptorSet with an arbitrary large size like 4096 descriptors.
Then I create a free list of descriptor indices, 0-4095.
When I create a texture/view, I pop an index from the free list and update the set range using the index as the BaseDescriptor.
The index is stored alongside some other stuff in a higher level "Texture" resource.
When I destroy a texture/view, I add its descriptor index back to the free list.
This is a very simple way to manage bindless descriptors, but it works well enough for now.

Per-object OBJECT_DATA and MATERIAL_DATA are not accessed bindlessly.
These are updated once per frame for all objects in the draw list.
The object's index in the draw list is used to access the OBJECT_DATA and MATERIAL_DATA of that object in the structured buffers.
Since the sizes are determined by the number of objects, it made sense to me to put them both in 1 nri::Buffer with 2 views.
(Any time one of them needs to be resized, both of them do.)

NRI_RESOURCE(StructuredBuffer<OBJECT_DATA>, ObjectData, t, 0, 1);
NRI_RESOURCE(StructuredBuffer<MATERIAL_DATA>, MaterialData, t, 1, 1);

inline OBJECT_DATA LoadObjectData(uint objectIndex)
{
    return ObjectData[objectIndex];
}

inline MATERIAL_DATA LoadMaterialData(uint objectIndex)
{
    return MaterialData[objectIndex];
}

The object index is passed to the shader right before nri::CmdDrawIndexed() using push constants:

struct SCENE_PUSH_CONSTANTS
{
    uint ObjectIndex; // This is the index of the object in the draw list.
    uint EntityID;

    // Used to access Positions[] and Elements[] bindlessly.
    uint PositionBufferViewIndex;
    uint ElementsBufferViewIndex;
};

NRI_ROOT_CONSTANTS(SCENE_PUSH_CONSTANTS, Push, 0, 0);

An object's MATERIAL_DATA contains the texture descriptor indices, for bindless access to them:

MATERIAL_DATA materialData = LoadMaterialData(Push.ObjectIndex);
Texture2D<float4> baseColorTexture = Textures[materialData.GetBaseColorTextureViewIndex()];

Bindless Positions[] and Elements[] behave more or less the same way.
(Create mesh buffer/views -> pop indices from free lists -> update set ranges.)

inline float3 LoadPosition(uint vertexID)
{
    return Positions[Push.PositionBufferViewIndex][vertexID];
}

inline VERTEX_ELEMENTS LoadElements(uint vertexID)
{
    return Elements[Push.ElementsBufferViewIndex][vertexID];
}

Accessed using object index from push constants and NRI_VERTEX_ID:

OBJECT_DATA objectData = LoadObjectData(Push.ObjectIndex);
float4 position = float4(LoadPosition(NRI_VERTEX_ID), 1.0f);
float4 worldPosition = mul(objectData.World, position);

I hope that wasn't all too confusing!

VERTEX_ELEMENTS is a struct that is defined before compilation, depending on what elements the vertex has.
(Stuff like color, normal, tangent, UV, etc.)
This makes switching from a structured buffer to vertex buffers a pretty big rewrite.
Another benefit of bindless structured buffers for vertex data is they only need to be set once per frame, regardless of draw list size.
Oh and pipeline creation is a lot simpler when you don't have to provide a nri::VertexInputDesc.

Obviously I can solve the OBJECT_DATA/MATERIAL_DATA problem by giving them each an exclusive nri::Buffer.
I'll be able to set the structure strides and they'll get the VK_BUFFER_USAGE_STORAGE_BUFFER_BIT flag.
Is using 1 buffer for multiple views with different strides and sizes an unusual practice?
It's just something I've always done in D3D12, and I never questioned it.

Just to prove any of this works at all, here's a really crappy image of my unfinished renderer using NRI with D3D12:

[Marlax0]

I spotted another "problem" with Vulkan requiring nri::CoreInterface::CmdSetVertexBuffers() for all graphics pipelines.
For my final compositing pass I draw a fullscreen triangle without a vertex buffer, since I can just calculate the vertices in the shader:

PS_INPUT main(NRI_DECLARE_DRAW_PARAMETERS)
{
    PS_INPUT output;
    const float2 uv = float2(uint2(NRI_VERTEX_ID, NRI_VERTEX_ID << 1) & 2);
    output.Position = float4(lerp(float2(-1.0f, 1.0f), float2(1.0f, -1.0f), uv), 0.0f, 1.0f);
    output.UV = uv;

    return output;
}

nri::DrawDesc drawDesc = {};
drawDesc.vertexNum = 3;
drawDesc.instanceNum = 1;

NRI.CmdDraw(commandBuffer, drawDesc);

This is used to "copy" an intermediate texture to the swap chain back buffer:

struct PS_INPUT
{
    // Do not perform perspective-correction during interpolation.
    noperspective float4 Position : SV_POSITION;
    noperspective float2 UV       : TEXCOORD;
};

NRI_RESOURCE(Texture2D, ColorTexture, t, 0, 0);

float4 main(in PS_INPUT input) : SV_Target
{
    return float4(ColorTexture[input.Position.xy].xyz, 1.0f);
}

Since VK_DYNAMIC_STATE_VERTEX_INPUT_BINDING_STRIDE is always set, VK always expects a vertex buffer to be set.
(Calling nri::CoreInterface::CmdSetVertexBuffers() with 0/null parameters does not bypass this.)

Validation Error: [ VUID-vkCmdDraw-pStrides-04913 ] Object 0: handle = 0x21349017410, type = VK_OBJECT_TYPE_COMMAND_BUFFER; Object 1: handle = 0x4b7df1000000002f, type = VK_OBJECT_TYPE_PIPELINE; | MessageID = 0x7790eb4c | vkCmdDraw(): VK_DYNAMIC_STATE_VERTEX_INPUT_BINDING_STRIDE state is dynamic, but the command buffer never called vkCmdBindVertexBuffers2.
The Vulkan spec states: If the bound graphics pipeline was created with the VK_DYNAMIC_STATE_VERTEX_INPUT_BINDING_STRIDE dynamic state enabled, but without the VK_DYNAMIC_STATE_VERTEX_INPUT_EXT dynamic state enabled, then vkCmdBindVertexBuffers2 must have been called and not subsequently invalidated in the current command buffer prior to this draw command, and the pStrides parameter of vkCmdBindVertexBuffers2 must not be NULL (https://vulkan.lunarg.com/doc/view/1.4.304.0/windows/1.4-extensions/vkspec.html#VUID-vkCmdDraw-pStrides-04913)

Creating and setting a dummy vertex buffer will probably work fine, but it feels like a little bit of a hack.
Any suggestions for alternative techniques that play nicely with both VK and D3D12 are appreciated!

[Marlax0]

After applying the tweaks I mentioned I am able to get everything working under Vulkan.
I did have to use a dummy vertex buffer, which Vulkan and D3D12 both give warnings for.
One more oddity I came across is:
RWStructuredBuffer also needs to be given a structure stride for Vulkan.
Under D3D12 this prevents the buffer from being cleared with nri::CoreInterface::CmdClearStorageBuffer().

D3D12 ERROR: ID3D12CommandList::ClearUnorderedAccessViewUint: ClearUnorderedAccessView* methods are not compatible with Structured Buffers. StructuredByteStride is set to 4 for resource 0x0000019506794390:'Unnamed ID3D12Resource Object'. [ RESOURCE_MANIPULATION ERROR #1156: CLEARUNORDEREDACCESSVIEW_INCOMPATIBLE_WITH_STRUCTURED_BUFFERS]

[dzhdanNV]

Thanks for all info. I see 2 problems:

• inability to use CmdDrawXxx without a bound vertex buffer in Vulkan since VK_DYNAMIC_STATE_VERTEX_INPUT_BINDING_STRIDE is always set (you have indices for VERTEX INPUT stage, all other data comes from shader resources)
• inability to clear a structured buffer with a CmdClearStorageBuffer in D3D12/VK

Correct?

[UPDATED]

[dzhdanNV]

What I gonna try and check:

• for problem 1 (solution) - do not enable VK_DYNAMIC_STATE_VERTEX_INPUT_BINDING_STRIDE if GraphicsPipelineDesc::vertexInput is NULL (no vertex inputs provided)
• for problem 1 (bonus) - enabling VK null descriptor feature (if supported) should add even more flexibility (but it's not mandatory needed)
• for problem 2 - I need to create a unit test...

Please, try this:

• find this line of code: https://github.com/NVIDIA-RTX/NRI/blob/main/Source/VK/PipelineVK.hpp#L252
• change this:

    dynamicStates[dynamicStateNum++] = VK_DYNAMIC_STATE_VERTEX_INPUT_BINDING_STRIDE;

to:

    if (vi)
        dynamicStates[dynamicStateNum++] = VK_DYNAMIC_STATE_VERTEX_INPUT_BINDING_STRIDE;

• try it! (it should remove the requirement to call CmdSetVertexBuffers)

[dzhdanNV]

RWStructuredBuffer also needs to be given a structure stride for Vulkan.

Yes. It's by VK spec. The corresponding buffer usage must be set at creation time.

but... Under D3D12 this prevents the buffer from being cleared with nri::CoreInterface::CmdClearStorageBuffer()

Yes. It's by D3D12 spec. ClearUnorderedAccessViewUint can be used with typed buffers only. I think that creating a bonus storageBuffer descriptor, which is not structured, just for clearing ops should solve your problem. I will improve embedded comments for CmdClear stuff to clarify usage.

Also, it looks like in this code:

NRI_INLINE void CommandBufferD3D12::ClearStorageBuffer(const ClearStorageBufferDesc& clearDesc) {
    DescriptorSetD3D12* descriptorSet = m_DescriptorSets[clearDesc.setIndex];
    DescriptorD3D12* resourceView = (DescriptorD3D12*)clearDesc.storageBuffer;
    const UINT clearValues[4] = {clearDesc.value, clearDesc.value, clearDesc.value, clearDesc.value};

    m_GraphicsCommandList->ClearUnorderedAccessViewUint({descriptorSet->GetPointerGPU(clearDesc.rangeIndex, clearDesc.descriptorIndex)}, {resourceView->GetPointerCPU()}, *resourceView, clearValues, 0, nullptr);
}

Uint of Float clear should be selected based on Descriptor type, like it's done for texture clears...

[dzhdanNV]

And no... D3D12 ClearUnorderedAccessView[Uint/Float] can't be used with STORAGE_STRUCTURED_BUFFER. Moreover, a typed STORAGE view can't be created for a structured buffer resource. The only way to clear a structured buffer is to use a compute shader in D3D12. It's silly, yes, but it's what we have. Vulkan uses copies under the hood, it seems to be doable in D3D12, but again it seems to be very artificial.

[dzhdanNV]

I'm thinking to merge Core.CmdClearStorageBuffer and Core.CmdClearStorageTexture to Core.CmdClearStorage:

• D3D12 code is the same for both, but there are potential issues not covered by the spec (what if the descriptor is RGBA16f but ClearUint is used?)
• VK uses fill commands, which can be distinguished (buffer/texture) internally using the descriptor type (already here)
• improve embedded documentation for the new function

Future:

• add Core.CmdFillZero (at least for buffers)
• VK will continue to use vkFillBuffer
• D3D12 will mimic same functionality by using a bunch of copies (from, say, a 1 Mb 0-ed resource)

[dzhdanNV]

As it turned out, ClearUnorderedAccessView[Uint/Float] is barely compatible with vkCmdFillX. Should match with the following constrains (updated NRI, not committed yet):

// Clear (potentially slow)
// For any "SHADER_RESOURCE_STORAGE" descriptors (D3D: structured buffers are unsupported)
void (NRI_CALL *CmdClearStorage)(NriRef(CommandBuffer) commandBuffer, const NriRef(ClearStorageDesc) clearDesc);

// Clear storage
NriStruct(ClearStorageDesc) {
    // For any buffers and textures with integer formats:
    //  - Clears a storage view with bit-precise values, copying the lower "N" bits from "value.[f/ui/i].channel"
    //    to the corresponding channel, where "N" is the number of bits in the "channel" of the resource format
    // For textures:
    //  - Clears a storage view with float values. It only works on float, UNORM, and SNORM formats
    // For buffers:
    //  - To avoid discrepancies in behavior between GAPIs prefer using "R32f/ui/i" formats for views
    const NriPtr(Descriptor) storage;
    Nri(Color) value; // avoid overflow
    uint32_t setIndex;
    uint32_t rangeIndex;
    uint32_t descriptorIndex;
};

[dzhdanNV]

TL/DR:

• This comment should eliminate the requirement to use CmdSetVertexBuffers with dummy inputs in VK (D3D12 seems to be unaffected according to your words)
• This comment shows improvements (demystification) around CmdClearStorage
• Additionally, I will add CmdFillBuffer with a bit uncommon implementation for D3D11/D3D12 :)

Hope to commit everything tomorrow.

[Marlax0]

Thanks for taking the time to go through all this. I know it's a lot!
I'll try all these suggestions ASAP.

Real quick I just want to say that overall I'm having a really good experience using NRI.
Yeah, it's confusing when something works in one API but not another, but that's just the nature of these beasts.
You and the community are doing a great job anyway, managing all that chaos.

[Marlax0]

* This comment should eliminate the requirement to use `CmdSetVertexBuffers` with dummy inputs in VK (D3D12 seems to be unaffected according to your words)

This is working perfectly. 👍

I'd like to just ask one more clarification:
To get my bindless vertex data working under Vulkan I had to separate my index/positions/elements data into 3 separate buffers.
(From 1 giant buffer with 3 views to 3 smaller buffers, 1 view each.)
This way each buffer can be created with the appropriate structure stride.
I am required to do this if I want to use the positions and elements views for structured buffers.
Is my understanding correct, that there's no way to use 1 giant buffer?

I noticed adding SCRATCH_BUFFER to the usage does give it the necessary VK_BUFFER_USAGE_STORAGE_BUFFER_BIT flag.
This completely bypasses the need to provide a structure stride, and any created views can be used for structured buffers.
Of course that doesn't seem intended and I don't know what other side effects SCRATCH_BUFFER will cause, in VK or D3D12.

From:

nri::BufferDesc bufferDesc = {};
bufferDesc.size = 768;
bufferDesc.structureStride = 12;
bufferDesc.usage = nri::BufferUsageBits::SHADER_RESOURCE;

nri::BufferViewDesc bufferViewDesc = {};
bufferViewDesc.buffer = buffer;
bufferViewDesc.size = 768;
bufferViewDesc.viewType = nri::BufferViewType::SHADER_RESOURCE;

To:

nri::BufferDesc bufferDesc = {};
bufferDesc.size  = 768;
bufferDesc.usage = nri::BufferUsageBits::SHADER_RESOURCE | nri::BufferUsageBits::SCRATCH_BUFFER;

nri::BufferViewDesc bufferViewDesc = {};
bufferViewDesc.buffer = buffer;
bufferViewDesc.size = 768;
bufferViewDesc.format = nri::Format::RGB32_SFLOAT;
bufferViewDesc.viewType = nri::BufferViewType::SHADER_RESOURCE;

This really does work and I don't notice any rendering issues.
Of course it may be against VK spec/undefined behavior.

[dzhdanNV]

Real quick I just want to say that overall I'm having a really good experience using NRI.

Thanks (づ｡◕‿‿◕｡)づ ! Kinds words are enlightening for further improvements :)

Additionally, I will add CmdFillBuffer with a bit uncommon implementation for D3D11/D3D12

Unfortunately, it will be CmdZeroBuffer due to D3D...

Is my understanding correct, that there's no way to use 1 giant buffer?

I don't think so. Maybe adjustments are needed. Let's discuss this in order:

• don't artificially add SCRATCH_BUFFER, it's for raytracing. Not dangerous, but confusing at least
• (as a rock solid solution) if buffers are suballocated from a large memory heap (via AllocatorInterface or manually via Core) from API point of view they are different, but actually become "closer" to being a single entity
• please, note: a structured buffer works best at leasst on NV HW with sizes 4, 8, 16 and alignment 16 if size >= 16
• yes, current NRI API doesn't allow to create 3 different "structured buffer" views for 1 buffer because of... D3D11, sorry. I have just realized it. I think it can be improved.

=> Please:

• close this issue, if you are happy with updated CmdSetVertexBuffers behavior
• create a new issue: add CmdFillBuffer since CmdClearStorageBuffer can't be used with structured buffers in D3D11/D3D12 (I will explain that only CmdZeroBuffer is possible because of wonky D3D)
• create a new issue: impossible to create a multiple structured buffer views from a single buffer (I will explain that the common denominator is D3D11, but really it's not the 1st class citizen anymore)

Feel free to adjust wording.

[dzhdanNV]

Fix for CmdSetVertexBuffers has been committed as well as added CmdZeroBuffer.

[Marlax0]

All completely reasonable. No further issues.