 | Adding Feedback Loop to your Ray-Tracer (Path-Tracer) |  |
Path-tracers take many frames to converge on a final image. How do you take the output from the ray-tracer, store it and pass it to the next frames output?
You start with a ray-tracer which renders the output to the screen (single frame) - with no feedback, here is an example code:
Link for path-tracer code with no feedback (code).
The output for the ray-tracer is stored in a compute shader image (image2D ) so we'll use the same format for the stored texture that we'll pass to the next frame. The input from the last frame is the same as the one we write out.
layout(binding = 3, set = 0, rgba8) uniform image2D lastFrameImage;
| Add Output Storage Image (and View) | |
We add a couple of variables to store and hold the data from the output which will be passed to the next frame.
VkImage outputImage;
VkImageView outputImageView;
| Inititilize Output Image Storage | |
Allocate the memory and setup the storage - make sure it's the same size as the output image.
void CreateOutputImage(VkDevice device, VkPhysicalDevice physicalDevice, VkImage& outputImage, VkImageView& outputImageView)
{
VkImageCreateInfo image{};
image.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO;
image.imageType = VK_IMAGE_TYPE_2D;
image.format = swapChain.colorFormat;
image.extent.width = width;
image.extent.height = height;
image.extent.depth = 1;
image.mipLevels = 1;
image.arrayLayers = 1;
image.samples = VK_SAMPLE_COUNT_1_BIT;
image.tiling = VK_IMAGE_TILING_OPTIMAL;
image.usage = VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_STORAGE_BIT;
image.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
VK_CHECK_RESULT(vkCreateImage(device, &image, nullptr, &outputImage));
VkMemoryRequirements memReqs;
vkGetImageMemoryRequirements(device, outputImage, &memReqs);
VkMemoryAllocateInfo memoryAllocateInfo = {};
memoryAllocateInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
memoryAllocateInfo.allocationSize = memReqs.size;
memoryAllocateInfo.memoryTypeIndex = getMemoryType(memReqs.memoryTypeBits, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT);
VkDeviceMemory outputImgMemory;
VK_CHECK_RESULT(vkAllocateMemory(device, &memoryAllocateInfo, nullptr, &outputImgMemory));
VK_CHECK_RESULT(vkBindImageMemory(device, outputImage, outputImgMemory, 0));
VkImageViewCreateInfo colorImageView = {};
colorImageView.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO;
colorImageView.viewType = VK_IMAGE_VIEW_TYPE_2D;
colorImageView.format = swapChain.colorFormat;
colorImageView.subresourceRange = {};
colorImageView.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
colorImageView.subresourceRange.baseMipLevel = 0;
colorImageView.subresourceRange.levelCount = 1;
colorImageView.subresourceRange.baseArrayLayer = 0;
colorImageView.subresourceRange.layerCount = 1;
colorImageView.image = outputImage;
VK_CHECK_RESULT(vkCreateImageView(device, &colorImageView, nullptr, &outputImageView));
VkCommandBuffer cmdBuffer = createCommandBuffer(VK_COMMAND_BUFFER_LEVEL_PRIMARY, true);
setImageLayout(cmdBuffer, outputImage,
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_GENERAL,
{ VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1 });
flushCommandBuffer(cmdBuffer, queue);
}
| Command Buffers | |
In the same function that copies the ray-tracer output from the storage buffer to the swap chain output - we also copy the ray-tracer output to our output image buffer.
...
{
// Copy the image to the output image for the next frame
// Prepare output image as transfer destination
setImageLayout(
drawCmdBuffers[i],
outputImage,
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
subresourceRange);
// Copy the current swap chain image to the output image
vkCmdCopyImage(drawCmdBuffers[i], swapChain.images[i], VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, outputImage, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, ©Region);
// Transition output image to shader read-only layout
setImageLayout(
drawCmdBuffers[i],
outputImage,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
subresourceRange);
}
...
At this point, we have the storage buffer with a copy of the output - but we need to 'link' it into the pipeline (i.e., bindings).
| Bindings | |
We'll add the last frame image to binding 3 (matches what we said at the start when we added the image2d to the shader).
VkDescriptorSetLayoutBinding outputImageLayoutBinding{};
outputImageLayoutBinding.binding = 3;
outputImageLayoutBinding.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE;
outputImageLayoutBinding.descriptorCount = 1;
outputImageLayoutBinding.stageFlags = VK_SHADER_STAGE_RAYGEN_BIT_KHR;
Link it to the write descriptors as well:
...
VkDescriptorImageInfo outputImageDescriptor{};
outputImageDescriptor.imageView = outputImageView;
outputImageDescriptor.imageLayout = VK_IMAGE_LAYOUT_GENERAL;
VkWriteDescriptorSet resultImageWrite = writeDescriptorSet(descriptorSet, VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, 1, &storageImageDescriptor);
VkWriteDescriptorSet uniformBufferWrite = writeDescriptorSet(descriptorSet, VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER, 2, &ubo.descriptor);
VkWriteDescriptorSet outputImageWrite = writeDescriptorSet(descriptorSet, VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, 3, &outputImageDescriptor);
std::vector<VkWriteDescriptorSet> writeDescriptorSets = {
accelerationStructureWrite,
resultImageWrite,
uniformBufferWrite,
outputImageWrite
};
...
| Done - Simple? | |
In fact, looping the output to the next frame is simple with the ray-tracing extension - as the output uses a storage image for the images (don't need to load a sampler). Write directly to the image (raw pixel colors with no modifications or sampling).
As the feedback (or should I say 'feedforward') loop for the image - will be used for a path-tracer - it means 'probablity' and 'random' number calculations. So you also need to pass a 'timer' uniform to the shader - as the random number seed needs to change each frame - if you're using the texture coordinates for the seed - this will not change if the same scene is rendered multiple times - to fix this - a 'timer' counter is passed which is added to the texture coordinates - so the seed changes for each frame update.
| Resources & Links | |
• Path-Tracer (without Feedback Loop)
• Path-Tracer (with Feedback Loop)
 |
Other Related Texts You Might Find Interesting |
 |
|
