Volume Ray Casting
- Volume Ray Casting
- Volume ray casting involves shooting view rays through pixels in a virtual screen. The color of the pixel a given ray passes through will reflect the color of the object it passes through.
The overall idea of volume ray casting is as follows: for each pixel in the 2D rendering, a ray is cast from the camera, through the pixel, and finally through the volume being rendered. Figure 1 displays this basic concept, showing how the basic color of the points in the volumes each ray passes through is displayed in the final rendering. Of course, there is much more to it than just checking the color at several points, but that will be explained in detail later on. For now, just understand that where each ray goes through the volumes determines the color of each ray's pixel.
Why It's Interesting
Volume ray casting is part of a set of techniques classified as volume rendering. All these techniques involve creating 2D images of 3D data sets. These data sets are often acquired through the use of various scanners, such as CTs and MRIs, which take regular 2D "slices" of objects and store them as data. All the slices put together to form a 3D data set. CT and MRI scanners are most commonly used for medical purposes, so many of the volume rendering images found online are of bones and organs in the human body.
A More Mathematical Explanation
The process of performing volume ray casting can be a bit overwhelming at first, so let's break it down into bite-sized chunks.
Note: You should have a good understanding of vectors and vector math in order to understand the math explained below. Specifically, cross product, dot product, unit vector, and others.
This process can be broken into the following steps:
- Create the camera base
- Create the virtual screen
- Cast the rays
- Sample along each ray
- Combine all samples on each ray into the final colors of each pixel
Step 1: Creating the Camera Base
The camera is the point where the 3D data is being viewed from. In Figure 2, you can see a bunch of different vectors pointing in different directions from a sphere. First, notice the three rays that are faded out. They represent the X, Y, and Z axes. The Y axis is the height in the 3D space, while the X and Z axes are the width and the depth. Now, in order to create the rays, a camera base must be calculated. This base is made up of X, Y, and Z axes that are local to the camera. The vectors that make up the base are shown in Figure 2 as U, -W, and V. You can think of this base as the axes rotated in such a way that the Z axis is pointing in the opposition direction that the camera is facing.
To calculate this camera base, start by looking at what you are given.
- From - the location of the camera
- To - the location the camera is looking at
- Up - a vector representing which direction is up
First, calculate the vector from the camera to the point it is facing. This vector is -W, and is found by subtracting the From point from the To point.
The vectors in the camera base need to be unit length, so divide -W by it's length, then take the negative of it to produce the first vector in the base, W.
Next, calculate the cross product of W and the Up vector.
This gives you the U vector. Lastly, calculate the cross product of W and U, which results in V.
Example Calculate -W. Calculate the length of -W, |-W|. Calculate W. Calculate U. Calculate V. And now you have W, U, and V.
Step 2: Creating the Virtual Screen
The virtual screen is a grid located a little bit in front of the camera, that represents the pixels of the final image that will be rendered. You can think of the virtual screen as the film in a camera, because it is where the image is imprinted. Figure 3 shows this virtual screen as a grid between the camera and the volumes that are being rendered. Each grid square represents a single pixel in the final image. Each ray will pass through the middle of one of the grid squares before passing through the volumes.
You are given two pieces of data that are needed for calculating this grid.
- Resolution of Image - the height and width, in pixels, of the final image
- Angle - the angle between the vectors going from the camera to the top and bottom of the grid
The distance, d, from the camera to the center of the grid can be any value you choose, so it is easiest to just use 1, so you don't need to worry about it in the calculations. The first step is to calculate the top, bottom, left, and right edges of this grid. Divide the given angle by 2, giving the angle represented in Figure 3 by the letter A. This is a right triangle, and the adjacent side is known, so
Rearrange this to give
If you chose to use 1 as your d value, then the equation is just
The rest of the edges are simple to compute once the top is known.
Now that you have the edges, you must find the individual pixel locations. The generic formulas for the X and Y coordinates of each pixel are
The X coordinate of a given pixel is the left edge plus a portion of the total width of the grid. The total width is right - left, which is then multiplied by the number of the current pixel, i, in that row, with zero being the first pixel. It is then divided by the resolution width. Also, the rays pass through the middle of the grid squares, so the pixel number, i, is offset by adding 0.5 to it. The Y coordinate is calculated in the same way, except it uses bottom and top instead of left and right, respectively. It also uses the current row, j, instead of the current location in the row.
Example If we were to calculate the pixel in grid position (15,72), i = 15, j = 72, As you can see, 15 grid positions to the right of the left edge is equal to -0.285798. The same is done for the Y position. In the same way, 72 grid positions above the bottom edge is equal to 0.18639. So,
Step 3: Casting the Rays
This third step, casting the rays, puts the results of the first and second steps to use. The rays can be represented by the equation
e is the camera location, d is a vector representing the direction of the ray, and t is some scalar value that represents the distance along that ray. We already know e, but we don't know d, the direction. This is calculated using the camera base vectors and the pixel location. First, multiply X by U.
Then, do the same with Y and V, adding the result to d.
Last of all, add W.
This gives the direction vector for the ray that passes through that specific pixel. This process is repeated for each pixel.
Example Using the values calculated in the previous two examples, calculate d. So, the ray through pixel (15,72) has a direction of (0.995, 0.253, 0.357)
Volume Ray Casting is continued in Volume Ray Casting (part 2).
A similar technique, Ray Tracing, which uses the same 3 steps described above, is explained in Ray Tracing.