Neverending Prototype Mode

npm

@davepagurek/glsl-autodiff 0.0.15 • Public • Published

glsl-autodiff

Tired of doing math to get normals in your vertex shader? Same. Use this library to write your function once and generate derivatives automatically!

Demo

Some example distortion functions with autodiff used to make normals:

Why?

Sometimes I want to dynamically distort meshes in shaders. If I want to preserve shading, this means I need to figure out the change to the surface normals in addition to the vertex displacement. If we have two different tangents vectors on the surface at a point, the normal is normalize(cross(tan1, tan2)). So far so good! Now how do we get those tangents?

If my displacement is based on two variables (e.g. if I am displacing z based on x and y), I can get those tangents by calculating the change in displacement in each axis:

vec3 normal = normalize(
cross(
vec3(1.0, 0.0, dz_by_dx),
vec3(0.0, 1.0, dz_by_dy)
)
)

Calculating these by hand involves doing a lot of derivatives and chain rules. I'm probably going to mess up my algebra somewhere and spend hours tracking down a minus sign I forgot. Instead, how about I calculate z using a domain specific language that can automatically compute dz/dx and dz/dy?

e.g. 1D Plane Displacement

In the simplest case, our input mesh is only in 2D, and we displace it in only the third dimension (x and y are independent; z is dependent.) Because of this setup, we can directly compute surface tangents: given a unit vector pointing in a surface direction, we set its z component to be the offset derivative for that axis. The cross product of these tangents then gives us a normal.

import { gen } from '@davepagurek/glsl-autodiff'
const vert = `
varying vec3 vVertPos;
void main() {
vec3 position = vVertPos.xyz;
// Generate z displacement based on x and y
const displace: (x: Op, y: Op) => Op = (x, y) => { /* Fill this in */ }

const offset = displace(position.x(), position.y())
offset.output('offset')
offset.outputDeriv('dodx', position.x())
offset.outputDeriv('dody', position.y())
})}

// Use the offset
vec3 outPosition = position;
outPosition.z += offset;

// The normal is the cross product of two surface tangents, which are
// the changes in z given a unit change in our two surface directions
vec3 slopeX = vec3(1., 0., dodx);
vec3 slopeY = vec3(0., 1., dodx);
vec3 normal = normalize(cross(slopeX, slopeY));

// TODO do something with outPosition and normal
}
`

e.g. 1D Mesh Displacement

Even if we are still only distorting our mesh in the z axis, it complicates things if we have a mesh without constant normals of its own (meaning it isn't flat, e.g. a sphere).

Like before, we generate a displacement in z based on x and y, and use the displacement derivatives to compute a normal. However, this normal is the normal for a displaced plane, not a sphere. Instead of using it directly, we calculate the rotation from an undisplaced plane to this new normal, and then apply that same rotation to the mesh's original normal.

import { gen } from '@davepagurek/glsl-autodiff'
const vert = `
varying vec3 vVertPos;
varying vec3 vVertNormal;

mat4 axisAngleRotation(vec3 axis, float angle) { /* ... */ }

void main() {
vec3 position = vVertPos.xyz;
// Generate z displacement based on x and y
const displace: (x: Op, y: Op) => Op = (x, y) => { /* Fill this in */ }

const offset = displace(position.x(), position.y())
offset.output('offset')
offset.outputDeriv('dodx', position.x())
offset.outputDeriv('dody', position.y())
})}

// Use the offset
vec3 outPosition = position;
outPosition.z += offset;

// Compute a normal like before, as if the mesh were a plane
vec3 slopeX = vec3(1., 0., dodx);
vec3 slopeY = vec3(0., 1., dodx);
vec3 displacedPlaneNormal = normalize(cross(slopeX, slopeY));

// The un-displaced normal for our hypothetical plane. This should be
// the cross product of the two surface vectors.
vec3 originalPlaneNormal = vec3(0., 0., 1.);

// Find the rotation induced by the displacement
float angle = acos(dot(noDisplacementNormal, originalPlaneNormal));
vec3 axis = normalize(cross(noDisplacementNormal, originalPlaneNormal));
mat4 rotation = axisAngleRotation(axis, -angle);

// Apply the rotation to the original normal
vec3 normal = (rotation * vec4(vVertNormal, 0.)).xyz;

// TODO do something with outPosition and normal
}
`

e.g. 3D Mesh Displacement

Now what do we do if we want to full generalize our displacement, so that we can displace using a 3D vector based on the entire 3D position input? Rotating the original normal will still work, but we no longer have two obvious surface tangents to use to come up with a displacement normal.

Our previous tangents were vec3(1., 0., dodx) ad vec3(0., 1., dody). These can be re-expressed as a unit vector added to the offset induced by a change in that direction: vec3(1., 0., 0.) + vec3(0., 0., dodx) and vec3(0., 1., 0.) + vec3(0., 0., dody). We can do the same thing now to come up with tangents, but where our induced change is a whole vector instead of just the z component! Our induced changes are no longer linearly independent from the input vectors, so this does place us in academically dubious territory, but in practice it seems to work decently.

The other issue is that we have three tangents to worry about now, x, y, and z, but we need only two vectors to produce a normal. Instead of getting a normal from cross(slopeX, slopeY), we can make one of our tangents span two axes, e.g. cross(slopeX, slopeYZ) where slopeYZ = vec3(0., 1., 1.) + dody + dodz).

import { gen } from '@davepagurek/glsl-autodiff'
const vert = `
varying vec3 vVertPos;
varying vec3 vVertNormal;

mat4 axisAngleRotation(vec3 axis, float angle) { /* ... */ }

void main() {
vec3 position = vVertPos.xyz;
// Generate x,y,z displacement based on x,y,z input
const displace: (position: VectorOp) => VectorOp = (pos) => { /* Fill this in */ }

const offset = displace(position)
offset.output('offset')
offset.outputDeriv('dodx', position.x())
offset.outputDeriv('dody', position.y())
offset.outputDeriv('dodz', position.z())
})}

// Use the offset
vec3 outPosition = position;
outPosition.z += offset;

// Compute a normal with a surface tangent in X and a surface tangent in YZ to
// cover all three axes
vec3 slopeX = vec3(1., 0., 0.) + dodx;
vec3 slopeYZ = vec3(0., 1., 1.) + dody + dodz;
vec3 displacedPlaneNormal = normalize(cross(slopeX, slopeYZ));

// The un-displaced normal for our hypothetical plane. This should be
// the cross product of the two surface vectors.
vec3 originalPlaneNormal = normalize(cross(vec3(1., 0., 0.), vec3(0., 1., 1.)));

// Find the rotation induced by the displacement
float angle = acos(dot(noDisplacementNormal, originalPlaneNormal));
vec3 axis = normalize(cross(noDisplacementNormal, originalPlaneNormal));
mat4 rotation = axisAngleRotation(axis, -angle);

// Apply the rotation to the original normal
vec3 normal = (rotation * vec4(vVertNormal, 0.)).xyz;

// TODO do something with outPosition and normal
}
`

API Usage

This exposes a gen function that generates GLSL that you can splice into your shader code. Inside a call to gen(), you provide a callback that takes in ad, the AutoDiff engine, and uses it to build shader code. We build shaders by essentially creating a graph of operations that we pass inputs through to arrive at an output. Every function call creates an object representing such an operation. Once we have a graph, we can automatically compute derivatives using it!

The AutoDiff engine's methods are designed to look similar to how you would normally write a shader, but using method chaining instead of operator symbols, similar to how p5's Vector class works. It looks something like this in action:

import { gen } from `@davepagurek/glsl-autodiff'

const vert = `
void main(void) {
vec4 objSpacePosition = vec4(aPosition, 1.0);

float x = objSpacePosition.x;
float y = objSpacePosition.y;
const speedX = 1.5
const speedY = 2.8

for (let i = 0; i < 3; i++) {
time.mult(0.002),
)
))
}
offset = offset.mult(0.1)
offset.output('z')
offset.outputDeriv('dzdx', x)
offset.outputDeriv('dzdy', y)
})}
objSpacePosition.z = z;
vec3 slopeX = vec3(1.0, 0.0, dzdx);
vec3 slopeY = vec3(0.0, 1.0, dzdy);
vNormal = uNormalMatrix * normalize(cross(slopeX, slopeY));

vec4 worldSpacePosition = uModelViewMatrix * objSpacePosition;
gl_Position = uProjectionMatrix * worldSpacePosition;
}
`

Inputs and Outputs

Parameters are variables that you create outside of AutoDiff but that you want to reference inside AutoDiff. Depending on their variable type, use one of the following methods to create them:

interface AutoDiff {
param(name: string): Param
vec2Param: (name: string) => VecParam
vec3Param: (name: string) => VecParam
vec4Param: (name: string) => VecParam
}

Outputs are used to store computation results from within AutoDiff into names that are accessible outside AutoDiff. You can also output the derivative of a computation with respect to an input parameter.

interface Op {
output(name: string);
outputDeriv(name: string, param: Param);
}

As an example, we can use param to reference a uniform, do some math with it, and output the result:

const vert = `
// A uniform we want to reference
uniform vec3 position;

void main(void) {
// Get a computation graph node representing the uniform

// Do stuff with it
const offset = ad.sum(position.x(), position.y(), position.z())

// Output it so the rest of the shader can reference it
offset.output('offset')
})}
vec3 worldSpacePosition = position;
worldSpacePosition.z += offset; // Use offset, an output, outside of AutoDiff
gl_Position = uProjectionMatrix * vec4(worldSpacePosition, 1.0);
}
`

Operations

Given any scalar graph node (Op), use any of the following methods to generate a new node corresponding to the GLSL operation of the same name:

interface Op {
neg(): Op; // Multiply by -1
sub(val: Input): Op;
mult(...params: Input[]): Op;
div(param: Input): Op;
pow(param: Input): Op;
sqrt(): Op;
sin(): Op;
cos(): Op;
tan(): Op;
mix(b: Input, amt: Input): Op;
clamp(min: Input, max: Input): Op;
min(...params: Input[]): Op;
max(...params: Input[]): Op;
ifElse(thenOp: Input, elseOp: Input): Op; // Ternary with the receiving Op being the condition
vecIfElse(thenOp: VectorOp, elseOp: VectorOp): VectorOp; // Ternary but where the if/else values are vectors
}

Note that an Input is either another node, or a number literal.

Given a vector graph node (VectorOp, which extends Op), use any of the following methods to again generate a new node with the computation result:

interface VectorOp {
neg(): VectorOp;
scale(k: Input): VectorOp; // When multiplying by a scalar
mult(...params: VectorOp[]): VectorOp; // When multiplying by another vector
mix(other: VectorOp, amt: Input): VectorOp;
clamp(min: VectorOp, max: VectorOp): VectorOp;
min(...params: VectorOp[]): VectorOp;
max(...params: VectorOp[]): VectorOp;
dot(other: VectorOp): Op;
length(): Op;
dist(other: VectorOp): Op;
}

You can also construct nodes directly from the AutoDiff engine passed to you in the gen callback:

interface AutoDiff {
// Create new values
val: (n: number) => Op;
vec2: (x: Input, y: Input) => Vec;
vec3: (x: Input, y: Input, z: Input) => Vec;
vec4: (x: Input, y: Input, z: Input, w: Input) => Vec;

// Create a new operation with scalar inputs
sum: (...params: Input[]) => any;
prod: (...params: Input[]) => any;
sqrt: (param: Input) => any;
sin: (input: Input) => any;
cos: (input: Input) => any;
tan: (input: Input) => any;
mix: (a: Input, b: Input, amt: Input) => any;
clamp: (val: Input, min: Input, max: Input) => any;
min: (...params: Input[]) => any;
max: (...params: Input[]) => any;
ifElse: (ifOp: Input, thenOp: Input, elseOp: Input) => any;

// Create a new operation with vector inputs
vecSum: (...params: VectorOp[]) => VectorOp;
vecProd: (...params: VectorOp[]) => VectorOp;
vecMix: (a: VectorOp, b: VectorOp, amt: Input) => VectorOp;
vecClamp: (val: VectorOp, min: VectorOp, max: VectorOp) => VectorOp;
vecMin: (...params: VectorOp[]) => VectorOp;
vecMax: (...params: VectorOp[]) => VectorOp;
dot: (a: VectorOp, b: VectorOp) => Op;
length: (val: VectorOp) => Op;
dist: (a: VectorOp, b: VectorOp) => Op;
}

On some very rare occasions, you write code that Just Works™! For the other 99% of the time, AutoDiff includes debug options that you can change by passing an additional settings object into the gen function:

gen: (
) => string;

maxDepthPerVariable: number;
debug: boolean;
};

When you set debug: true, each line in the outputted GLSL that comes directly from an code you wrote gets a comment referencing where it came from:

Input Output
const vert = `
void main(void) {
vec4 objSpacePosition = vec4(aPosition, 1.0);
float origZ = objSpacePosition.z;
float x = objSpacePosition.x;
float y = objSpacePosition.y;
const speedX = 1.5
const speedY = 2.8

for (let i = 0; i < 3; i++) {
time.mult(0.002),
)
))
}
offset = offset.mult(0.1)
offset.output('z')

}, { debug: true, maxDepthPerVariable: 4 })} objSpacePosition.z += z; vec4 worldSpacePosition = uModelViewMatrix * objSpacePosition; gl_Position = uProjectionMatrix * worldSpacePosition; } `

void main(void) {
vec4 objSpacePosition = vec4(aPosition, 1.0);
float origZ = objSpacePosition.z;
float x = objSpacePosition.x;
float y = objSpacePosition.y;

objSpacePosition.z += z; vec4 worldSpacePosition = uModelViewMatrix * objSpacePosition; gl_Position = uProjectionMatrix * worldSpacePosition; }

Sometimes too much or too little ends up on one line, so you can also supply maxDepthPerVariable to supply a maximum number of nested operations before creating an intermediate variable. If unspecified, we use maxDepthPerVariable: 8. To condense into as few lines as possible, use Infinity. To make every operation its own variable, use 1.

Importing in JS/TS builds

Then, import the gen method:

import { gen } from '@davepagurek/glsl-autodiff'

// Use autodiff here
})}
`

Importing in the Browser

<script type="text/javascript" src="autodiff.js"></script>

Then, use the global AutoDiff variable which includes the gen function:

// Use autodiff here
})}
`

Contributing

Want to help out? There are a number of things I'd like to get done at some point that are written down for posterity in the Issues tab that I'd love help on!

Setting up the repo:

git clone git@github.com:davepagurek/glsl-autodiff.git
cd glsl-autodiff
yarn install

Compiling build/autodiff.js and build/autodiff.d.ts:

yarn build

Pushing all compiled code to Github Pages for demos (if you are me or have write permissions):

yarn deploy

Publishing a new version to NPM:

yarn publish

Keywords

none

Install

npm i @davepagurek/glsl-autodiff

170

0.0.15