Drawing convincing curtains on an Android tablet
This summer I’ve been teaching an Android tablet to draw curtains. Not a still picture of a curtain, but one you can open and close, which bunches up into folds as it goes and hangs back down again. There’s a Roman blind as well, which pulls up into pleats.
It’s for my side project: I run the website for Kurtinz.com, a soft-furnishings shop. The site already shows off its curtains in a Flash player, and this is the start of a prototype that goes a step further. You stick a fiducial marker (a printed pattern that software can easily spot and measure) on your window, and tell it the window’s dimensions. Then the tablet draws the curtains over the live camera view, so you can see them hanging in your own room before you buy.
The result is a small OpenGL ES 2.0 engine, now on GitHub as android-ogles2-engine. It needs Android 3.0 or later.
The problem with fabric
Curtains are all folds, and folds are all detail. If you model every ripple as real geometry you need a very fine mesh, and a tablet will struggle to animate that smoothly. So the engine cheats in two layers:
- A deliberately coarse mesh does the physics. A curtain has one column of vertices every 5 cm and just four rows down its whole drop. That’s enough to get the overall shape right: how far it’s bunched, and how it swings.
- A normal map does the detail. A normal map is a texture that, instead of colour, stores which way the surface is facing at each pixel. The lighting shader reads it and shades the flat mesh as if it were rippled. Here, it’s the only thing the lighting looks at.
The interesting part is that the normal map isn’t a fixed image. It’s regenerated from the physics every frame. Each segment of the mesh gets one half-cycle of a cosine ripple, and the strength of the ripple depends on how deeply that segment is folded. That’s measured from how far apart neighbouring vertices along the hem are, front to back. Draw the curtain open and the fabric bunches, the folds deepen, and the ripples get stronger. Pull it closed and they flatten out again.
On top of that goes a second, fixed normal map for the pleated heading tape along the top, which fades out a few centimetres down.
Here’s a toy version you can play with. It’s a rewrite in the browser, not the Android code, but it uses the same recipe. Drag the curtain’s edge, or use the slider. Then untick normal-map ripples to see what the coarse mesh looks like by itself:
The physics
The cloth is a grid of particles joined by invisible sticks. It’s the classic game-physics approach:
- Verlet integration. Each particle remembers where it was last frame instead of storing a velocity. Its next position is its current one plus (a little less than) however far it moved last time, plus gravity. The “little less” is a drag of 0.97, which stops it swinging for ever.
- Constraints. After every step, each pair of neighbouring particles is nudged back towards its rest distance. Do that a few times per frame and the grid behaves like fabric. I use a well-known approximation that avoids a square root per stick, which adds up when you’re doing it for every stick, several times a frame, on a tablet.
- Pinning. The top row is fixed along the curtain rail. To open the curtain, the free corner slides along the rail, easing an eighth of the remaining distance each frame, and the rest of the top row spreads out evenly behind it.
The fabric is 1.5 times wider than the rail, as real curtains usually are. That’s what makes it fold at all: the particles along the top can never lie flat, so the sticks between them push the fabric out into zigzags. To get it started I set every other column back slightly, as if it had been pleated.
The Roman blind uses the same particle system with a different set of rules. Its fabric is divided into sections, every 20 rows. Raising the blind lifts the bottom of each section in turn, and stacks the raised ones a fixed distance apart, so the cloth between them has to fold. Its physics works, but it doesn’t have a normal map yet, so for now it looks rather flatter than it should.
A nice side effect of all this is that the engine knows when it can stop. If no particle has moved more than a tiny amount, it skips the physics and the normal-map update altogether until someone touches the curtain again. That’s good news for the tablet’s battery.
The engine
Around the curtains there’s a small general-purpose engine: meshes, a plain textured material, and a Phong material with normal mapping, plus the usual vector classes. The shaders are written out as Java strings, as seems to be the way with Android’s GLES20 examples. The curtain’s hem also gets four times as many vertices as the rest of each row, bent into a gentle wave, so the hem curves in and out between the folds instead of zigzagging.
The physics code started life elsewhere. There’s still a commented-out function in ActionScript syntax in the Java, which rather gives the game away: it came over from the Flash version on the Kurtinz site.
Where it’s got to
Curtains open, close, fold and swing, and they’re lit convincingly enough to pass for fabric. The Roman blind is working but still waiting for its ripples, and the augmented-reality part, putting all this over a live camera view of your window, isn’t in this repo.