Oil paint

Oil paint

Draws a three.js scene as thick oil paint, stroke by stroke: every leaf, trunk, drift of snow, ray of the aurora and fold of cloth is a dab of the brush pinned to the world, so the paint stays on things as you move. The demo is two walks with the same wanderer: a ridge of snow under the aurora, with a cabin at the end of it, and an autumn wood.

In the demo: click to look with the mouse, WASD to move, Shift to sprint, Space to jump, the wheel to step the eye back or in, H hides the hint. A gamepad works too. The two buttons in the corner change the picture. There is no sound.

• Aurora. You start among spruces under snow, come out onto a knoll under the whole sky, and follow the crest north to a cabin at the foot of the horn. V looks through his eyes, and from the cabin's doorstep on the demo does that for you. Look up.
• Autumn wood. A worn way through beech, birch and oak to a clearing.

It is a starting point, not a filter. You can't lay it over a finished game: each thing in your scene is given three mixes of colour and a way for its strokes to run. Start from one of the two scenes, change it, replace it.

The idea

Nothing is filtered after the picture is drawn. Everything is a dab: one touch of a loaded brush, in one mixed colour, pinned to a point of the world and turned to lie along a direction of the world (the grain of a trunk, the fall of a cloak).

• Colour is paint, not light. A thing has three mixes (light, half-tone, shade). Each dab picks one from how lit its spot is, then is mixed a little differently from its neighbours. Nothing is tone-mapped, so you choose colours as they should look on the canvas.
• The brush is a sheet of eight marks made in code: where there is paint, how thick it stands, and a streak of a second colour. A light from the upper left of the canvas rakes across the thickness.
• Distance simplifies. Dabs keep between a least and a most size on the canvas, so far things are painted with fewer, larger touches.
• The ground stays put. Its dabs lie flat on it, real patches of the surface, each at its own small height, so they overlap in one fixed order and never turn with the eye. Every dab belongs to one small square of ground for good. With distance, one in four is drawn larger to stand for its neighbours; as you come nearer it shrinks back where it lies and the others grow in round it. Nothing is swapped for something else.
• Nothing is put in order by the eye. Which of two overlapping strokes is on top must never depend on where the camera is, or the paint crawls as you breathe. Near things are given a depth each. Far things (ranges, water, sky) are too far apart for a depth to tell them from each other, so they are laid in one fixed order, far to near.
• Shade. The still world's shade is drawn once from the sun (or the moon), and a dab takes it whole, so a shadow's edge is a ragged line of whole dabs. A moving thing's shadow would make dabs flick as it passed, so it goes over the ground as a soft glaze, pixel by pixel.
• Light that is light. A few things are painted in strokes that add up instead of covering: the aurora, a flame, the glow round a candle. A carried lamp (uLamp) turns what is near it toward that thing's own warm colour.
• Mist puts distance between one row of things and the next. In the wood, the one thing that is not a dab is the sun standing in that mist, found by walking each line of sight through the shade.

Use it in a game

It owns the picture: a three.js WebGLRenderer (WebGL2), its own render target and final pass, and every material. Your scene, camera and game loop stay yours. Built on three 0.186.

import * as THREE from 'three';
import { createShared, createBrushSheet, dabMaterial, dabGeometry, dabMesh, groundRings, pinDabs, createShade, createFrame } from './src/oilpaint/index.js';

const renderer = new THREE.WebGLRenderer({ antialias: false });
renderer.autoClear = false;
renderer.outputColorSpace = THREE.LinearSRGBColorSpace; // colours are paint: written as they are

const shared = createShared();              // sun, wind, mist, maps: every material reads these
shared.tBrush.value = createBrushSheet();
// shared.tHeight / shared.tLand: your ground's height and a map of what lies where (see src/world/land.js)

// 1. dabs of your own: positions in buffers, colour and direction decided in GLSL
const mesh = dabMesh(dabGeometry(count, { aPos: [positions, 3] }), dabMaterial(shared, {
  head: `in vec3 aPos;`,
  body: `void makeDab(inout Dab d) {
    float lit = sunAt(aPos);
    vec4 r = hash4(float(gl_InstanceID));
    d.pos = aPos;
    d.dir = vec3(0.0, 1.0, 0.0);                       // the way the stroke runs
    d.size = vec2(0.2, 0.5);                           // across and along, metres
    d.c1 = hazed(jitter(paint3(LIGHT, HALF, SHADE, lit, r.x), r.yzw, 0.5), aPos);
    d.c2 = d.c1 * 1.15;                                // the colour streaked into it
  }`,
}));

// 2. a ground with no end (see src/world/floor.js for the leaf carpet, src/night/snow.js for snow)
const carpet = groundRings(shared, { body: carpetGLSL, per: 3, n: 192, cell0: 0.14, levels: 6, minPx: [5, 9], maxPx: [4000, 4000], cells: [0, 8] });

// 3. paint on any mesh, rigged or not: pins spread over its surface and carried with it
const paint = pinDabs(shared, [[light, half, shade]], [{ mesh: model, count: 500, style: (p, n) => ({ pal: 0, size: [0.06, 0.12], dir: UP }) }]);
scene.add(mesh, carpet.group, paint.mesh);

// 4. shade and the canvas
const shade = createShade(renderer, shared, stillThingsScene);   // the still world, once
const frame = createFrame(renderer, shared);

function tick() {
  shared.uTime.value = time;
  shared.uPixel.value = (2 * Math.tan(THREE.MathUtils.degToRad(camera.fov) / 2)) / drawingBufferHeight;
  carpet.follow(camera.position.x, camera.position.z);
  paint.update();                    // after the model has moved
  shade.update(scene, heroPosition); // the moving thing's shadow: draws the meshes on layer 1
  frame.render(scene, camera);
}

In makeDab you have paint3 (the three mixes), jitter (broken colour), hazed (mist), sunAt / woodSunAt (shade), lampAt (a carried light), gustAt (wind), heightAt / normalAt (the ground) and hash4. Set d.lie = 1.0 with d.nrm and d.height for a dab that lies on a surface, d.glaze = 1.0 for one the moving shadow passes over, and d.base = 1.0 for one pinned by its foot (a blade of grass, a ray of the aurora). createShade takes the sun's direction and how much of the world its picture of the shade must hold.

For made things (a cabin, furniture, pots) there is a kit in src/night/kit.js: give it flat faces or any geometry with the name of a palette and it keeps the faces as an underpainting and lays strokes on them along the grain, with logs and planks in courses. src/night/cabin.js, roof.js and details.js are a whole house built with it.

What is where

Limits

• three.js on WebGL2 only. It is not a node material and does not run on WebGPU.
• Almost everything is opaque: there is no glass, and water is strokes that give back the sky, not a mirror of the world.
• Textures on your models are ignored; colour comes from the palettes you give.
• Measured only on an RTX 4090 at 1080p: about 4.5 ms a frame in the aurora, about 4.6 in the wood. ?quality=low, or Patch's performance mode, draws less (in the aurora: fewer pixels, no veil behind the curtains, less snow in the air; in the wood: no sun in the mist, fewer falling leaves).
• The demo downloads about 26 MB, most of it the wanderer's clips. A first visit is ready in about 9 seconds on a fast machine.
• Each scene is one hour of one day. Walk off the ridge into the fjord, or more than 78 m from the way in the wood, and you are put back at the start.
• Through his own eyes you see no body or hands.

Build

npm install, then npm run build: the demo is in dist/ with index.html at the top. node tools/serve.mjs dist 5405 serves it. The tools in tools/ drive it headless with Playwright: set CHROME to a Chromium of your own if Playwright has none. They show the wood unless given --query "scene=aurora".

Credits

Code MIT, art CC0 1.0. The wanderer is Patch's starter man (a MakeHuman figure, CC0) with clips from Quaternius' Universal Animation Library (CC0) and Patch's own (CC0). Physics by Rapier (Apache-2.0). Everything else in both scenes is made in code. Every part and its source is in LICENCE-NOTES.md.