 | Adding Feedback Loop to your Ray-Tracer (Path-Tracer) |  |
Path-tracers take many frames to converge on a final image. How do you take the output from the ray-tracer, store it and pass it to the next frames output?
You start with a ray-tracer which renders the output to the screen (single frame) - with no feedback, here is an example code:
Link for path-tracer code with no feedback (code).
The output for the ray-tracer is stored in a compute shader image (image2D ) so we'll use the same format for the stored texture that we'll pass to the next frame. The input from the last frame is the same as the one we write out.
<?php
layout(binding = 3, set = 0, rgba8) uniform image2D lastFrameImage;
| Add Output Storage Image (and View) | |
We add a couple of variables to store and hold the data from the output which will be passed to the next frame.
<?php
VkImage outputImage;
VkImageView outputImageView;
| Inititilize Output Image Storage | |
Allocate the memory and setup the storage - make sure it's the same size as the output image.
<?php
void CreateOutputImage(VkDevice device, VkPhysicalDevice physicalDevice, VkImage& outputImage, VkImageView& outputImageView)
{
VkImageCreateInfo image{};
image.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO;
image.imageType = VK_IMAGE_TYPE_2D;
image.format = swapChain.colorFormat;
image.extent.width = width;
image.extent.height = height;
image.extent.depth = 1;
image.mipLevels = 1;
image.arrayLayers = 1;
image.samples = VK_SAMPLE_COUNT_1_BIT;
image.tiling = VK_IMAGE_TILING_OPTIMAL;
image.usage = VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_STORAGE_BIT;
image.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
VK_CHECK_RESULT(vkCreateImage(device, &image, nullptr, &outputImage));
VkMemoryRequirements memReqs;
vkGetImageMemoryRequirements(device, outputImage, &memReqs);
VkMemoryAllocateInfo memoryAllocateInfo = {};
memoryAllocateInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
memoryAllocateInfo.allocationSize = memReqs.size;
memoryAllocateInfo.memoryTypeIndex = getMemoryType(memReqs.memoryTypeBits, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT);
VkDeviceMemory outputImgMemory;
VK_CHECK_RESULT(vkAllocateMemory(device, &memoryAllocateInfo, nullptr, &outputImgMemory));
VK_CHECK_RESULT(vkBindImageMemory(device, outputImage, outputImgMemory, 0));
VkImageViewCreateInfo colorImageView = {};
colorImageView.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO;
colorImageView.viewType = VK_IMAGE_VIEW_TYPE_2D;
colorImageView.format = swapChain.colorFormat;
colorImageView.subresourceRange = {};
colorImageView.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
colorImageView.subresourceRange.baseMipLevel = 0;
colorImageView.subresourceRange.levelCount = 1;
colorImageView.subresourceRange.baseArrayLayer = 0;
colorImageView.subresourceRange.layerCount = 1;
colorImageView.image = outputImage;
VK_CHECK_RESULT(vkCreateImageView(device, &colorImageView, nullptr, &outputImageView));
VkCommandBuffer cmdBuffer = createCommandBuffer(VK_COMMAND_BUFFER_LEVEL_PRIMARY, true);
setImageLayout(cmdBuffer, outputImage,
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_GENERAL,
{ VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1 });
flushCommandBuffer(cmdBuffer, queue);
}
| Command Buffers | |
In the same function that copies the ray-tracer output from the storage buffer to the swap chain output - we also copy the ray-tracer output to our output image buffer.
...
{
// Copy the image to the output image for the next frame
// Prepare output image as transfer destination
setImageLayout(
drawCmdBuffers[i],
outputImage,
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
subresourceRange);
// Copy the current swap chain image to the output image
vkCmdCopyImage(drawCmdBuffers[i], swapChain.images[i], VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, outputImage, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, ©Region);
// Transition output image to shader read-only layout
setImageLayout(
drawCmdBuffers[i],
outputImage,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
subresourceRange);
}
...
At this point, we have the storage buffer with a copy of the output - but we need to 'link' it into the pipeline (i.e., bindings).
| Bindings | |
We'll add the last frame image to binding 3 (matches what we said at the start when we added the image2d to the shader).
<?php
VkDescriptorSetLayoutBinding outputImageLayoutBinding{};
outputImageLayoutBinding.binding = 3;
outputImageLayoutBinding.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE;
outputImageLayoutBinding.descriptorCount = 1;
outputImageLayoutBinding.stageFlags = VK_SHADER_STAGE_RAYGEN_BIT_KHR;
Link it to the write descriptors as well:
...
VkDescriptorImageInfo outputImageDescriptor{};
outputImageDescriptor.imageView = outputImageView;
outputImageDescriptor.imageLayout = VK_IMAGE_LAYOUT_GENERAL;
VkWriteDescriptorSet resultImageWrite = writeDescriptorSet(descriptorSet, VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, 1, &storageImageDescriptor);
VkWriteDescriptorSet uniformBufferWrite = writeDescriptorSet(descriptorSet, VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER, 2, &ubo.descriptor);
VkWriteDescriptorSet outputImageWrite = writeDescriptorSet(descriptorSet, VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, 3, &outputImageDescriptor);
std::vector<VkWriteDescriptorSet> writeDescriptorSets = {
accelerationStructureWrite,
resultImageWrite,
uniformBufferWrite,
outputImageWrite
};
...
| Done - Simple? | |
In fact, looping the output to the next frame is simple with the ray-tracing extension - as the output uses a storage image for the images (don't need to load a sampler). Write directly to the image (raw pixel colors with no modifications or sampling).
As the feedback (or should I say 'feedforward') loop for the image - will be used for a path-tracer - it means 'probablity' and 'random' number calculations. So you also need to pass a 'timer' uniform to the shader - as the random number seed needs to change each frame - if you're using the texture coordinates for the seed - this will not change if the same scene is rendered multiple times - to fix this - a 'timer' counter is passed which is added to the texture coordinates - so the seed changes for each frame update.
| Resources & Links | |
• Path-Tracer (without Feedback Loop)
• Path-Tracer (with Feedback Loop)
 |
Other Related Texts You Might Find Interesting |
 |
|
