Ray Marching Over a React Three Fiber (R3F) Scene

A-little disclaimer - Im still learning so don’t take everything i say as gospel. But its a good way to overlay a raymarched scene on top of a three.js scene.

A while back I wanted to get some fundamentals about raymarching and it crossed my mind… its all well and good raymarching but how can you overlay this onto a three.js or R3F scene??

Have a play around with the codeSandBox below 🙂

This is a very simple sand box, its a POC of a white raymarched sphere onto a three.js scene.

Its very cool raymarching a sphere with postprocessing but what you soon realise, is that a simple raymarched sphere wont rotate with the camera if you don’t account for certain things.

I hear ya, why would you raymarch a sphere. Its boring and useless..

Well sort of 😛

The thing is Im starting to learn the basics of implementing 3D scenes or effects without meshes and and laying the fundamentals to do more complex cool things.

Heres a very high level over view of what we are doing with raymarching:

The first thing to understand is the range of

Copy 
1vec2 screenPos = gl_FragCoord.xy

for example:

Now to explain these two lines:

Copy 
1vec2 uv =( gl_FragCoord.xy * 2.0 - uResolution ) / uResolution;
2vec2 p = gl_FragCoord.xy / uResolution.xy;

sudo code for the first line:

Copy 
1vec2 topLeft = (vec2(0.0,0.0) * 2.0 - vec2(8.0, 8.0)) / vec2(8.0, 8.0);
2vec2 topLeft = vec2(-1.0, -1.0)
3
4vec2 topRight = (vec2(8.0,0.0) * 2.0 - vec2(8.0, 8.0)) / vec2(8.0, 8.0);
5vec2 topRight = vec2(1.0, -1.0)
6
7vec2 bottomRight = vec2(1.0, 1.0)
8
9vec2 bottomLeft = vec2(-1.0, 1.0)

Essentially we are putting the screen coordinates in the range -1.0 - 1.0.

And the second line is getting coordinates taking into account screen resolution, for acquiring the background texture - NOT the rayMarched scene.

Once you have the ray we want to rotate the rays and this is how you would achieve this:

Copy 
1vec3 ray = (cameraWorldMatrix * cameraProjectionMatrixInverse * vec4( screenPos.xy, 1.0, 1.0 )).xyz;

When you can rotate the ray around the scene with camera movements, you can do a simple raycasting algorithm:

Copy 
1// Ray Marching Variables
2vec3 rayOrigin = cameraPosition;
3vec3 rayDirection =ray;
4vec3 sphereOirgin =  vec3(0.30);
5
6// This is the R3F scene texture passed to
7// this pass in the effect composer tDiffuse
8// uniform.
9vec3 sum = texture(tDiffuse, p).xyz;
10float rayDistance = 0.0;
11float MAX_DISTANCE = 2000.0;
12
13for (int i = 0; i< 1000;i ++) {
14    vec3 currentStep = rayOrigin + rayDirection * rayDistance ;
15
16    float dist = sphereSDF(currentStep,sphereOirgin, .10);
17
18    if (dist < 0.00001 ) {
19        sum = vec3(1.0);
20    }
21    if (rayDistance > MAX_DISTANCE) {
22        break;
23    }
24    rayDistance += 0.09;
25}
26
27gl_FragColor = vec4(sum.xyz, 1.0);

For the keen eyed among you if you position the camera so that the white raymarched sphere is behind the R3F cube it still appears as if its in front of the cube.

To fix this you would need a depth texture and another if block - so that if a fragment has a depth then you would not raymarch that particular ray/fragment and just show the colour from tDiffuse.

As i said at the start this is a really basic example with a few artefacts but gives you the basic idea of how to raymarch on top or onto a R3F or three.js scene.

Final Thoughts

Hope you enjoyed this little tutorial and if you do anything more advanced than this let me know! would love to see it 🙂