A browser viewer for an MLX90640 thermal camera

The MLX90640 is a thermal camera on a chip: a grid of 32 × 24 infrared sensors, 768 pixels in all, that reads the temperature of whatever it’s pointed at over I²C. That’s not many pixels. It’s enough to pick out a warm mug, a radiator or a person, though.

I had one on Pimoroni’s breakout board, wired to a Raspberry Pi, and wanted to see its picture live in a browser, on whatever device was to hand. This was a one-day job: the files all date from a single Sunday in January, from early afternoon to about half past nine at night.

How it fits together

There are three pieces:

1. A program that reads the camera. rawrgb is one of the examples in Pimoroni’s mlx90640-library, so I didn’t have to write this bit. It runs at 2 frames per second and writes each frame to its standard output as 768 red–green–blue triples, already coloured with a heat-map palette. It runs under sudo, since it wants direct access to the hardware.
2. A Python server using the websockets library. When a browser connects, it starts rawrgb as a subprocess, reads frames from its output (2,304 bytes each), packs every pixel into a single integer, and sends the frame down the websocket as a JSON array of 768 numbers.
3. A web page with a 320 × 240 canvas. Each time a frame arrives, it unpacks the integers back into red, green and blue, and paints each camera pixel as a 10 × 10 block.

One neat trick: the same Python process serves the web page too. The websockets library lets you hook into the HTTP request before the websocket upgrade happens, so a plain GET / gets index.html back and anything else gets a 404. That means one port and no separate web server.

The upscaling is done by hand, pixel by pixel, into an ImageData buffer. So the picture comes out as honest, chunky 10 × 10 squares rather than being smoothed by the browser. With only 768 pixels to play with, you can see exactly how little the camera knows.

Test patterns first

Before plugging the camera in, I wanted to be sure the plumbing worked: a subprocess writing bytes, asyncio reading them, and a websocket passing them on. So the first version of the server started a tiny fake camera instead: a one-line Python program that wrote a red pixel, then a green one, then red again, switching eight times a second. The server painted every other pixel of the frame in that colour and the rest blue, so a working pipeline showed up in the browser as a flickering checkerboard.

There’s also a scrap of a script that just reads three bytes at a time from a subprocess and prints them. It’s the smallest possible test of “can asyncio read binary from a pipe”, which it can.

Where it stands

As of this evening, the Pi streams the thermal camera to any browser on the network at 2 frames per second, as a grid of big glowing squares. Next, I’d like to interpolate those squares into something smoother, and maybe lay the heat map over an ordinary camera image, the way a proper FLIR camera does.