showstack

The shape of it

Capture, find structure, decide what it means

Every machine-reading system is these three steps. When one fails, it fails at one of them, and knowing which saves an enormous amount of guessing.

The one in everybody's pocket

Why a QR code survives being damaged

A QR code is engineered almost entirely around the fact that a person will point a phone at it badly, from an angle, in poor light, after something has been spilled on it.

The three big squares in the corners are finder patterns. They exist so that a scanner can locate the code anywhere in a frame, work out its rotation, and correct for the perspective of holding a phone at an angle — all before reading a single data bit. A fourth, smaller alignment pattern corrects for the paper being curved.

The data itself is stored with Reed–Solomon error correction — real forward error correction, of the kind on the encoding page. At the highest correction level roughly 30% of the code can be missing or wrong and the original data is still recoverable, not guessed. That is why a code with a logo pasted over the middle of it still scans, and why a torn poster still works.

The pattern of light and dark is also deliberately masked — XORed with one of several fixed patterns, chosen at encoding time — so that the result has no large blank areas that would confuse the scanner. None of this is about storing more data. All of it is about a human being holding a camera.

The older one

What OCR is actually doing

Turning a photograph of a page into text is four separate problems, and modern systems still do them in roughly this order even when a single neural network is handling several at once.

Straighten and find the text. Correct the skew and perspective, separate text from images and background, identify blocks and their reading order. A two-column page read as one column produces perfect character recognition and unusable output.

Segment. Cut blocks into lines, lines into words, words into candidate glyphs. Touching or broken characters are where this goes wrong, which is why photocopies are harder than scans.

Classify. Decide what each shape most likely is. Older systems compared extracted features against templates; modern ones run a trained network. Either way the output is a ranked list with confidences, not an answer.

Use language. This is the step people forget and it does most of the heavy lifting. rn and m are nearly identical shapes; so are 0 and O, 1 and l, 5 and S. The shape genuinely is ambiguous. What resolves it is a model of what words and number formats are possible — which is why OCR on a normal sentence is far better than OCR on a serial number, where there is no language to lean on.

How much of a code can you lose?

Reed–Solomon redundancy is a dial, not a constant. Higher correction survives more damage and leaves less room for data in the same physical size.

Distance

Every rangefinder is one of four principles

Anything that measures how far away something is uses one of a small number of tricks, and knowing which one a device uses tells you immediately where it will fail.

Two sensors worth pulling apart

A pressure mat and an image sensor

A pressure mat is usually one of three things. A resistive mat has two conductive layers separated by a spacer; standing on it presses them together and the resistance drops. A piezoelectric element generates a voltage when it is deformed — which means it senses the change, not the standing weight, so it detects a footfall and then reports nothing. A capacitive mat senses a body altering an electric field, works through a floor covering, and is the same physics as a touchscreen. Which one you have decides whether it can tell you somebody is still standing there.

An image sensor is a grid of photodiodes, each accumulating charge in proportion to the light landing on it during the exposure. Two things about it matter constantly on a show. First, it is colour-blind: colour comes from a Bayer filter, a mosaic of red, green and blue filters over the pixels, with twice as many green because human vision is most sensitive there — the full-colour image is then interpolated. That is the same three-channel approximation of human vision behind metamerism.

Second, the shutter. A rolling shutter reads the sensor row by row, so different parts of the frame are captured at different instants — which is why fast pans skew and why LED lighting and screens produce banding. A global shutter captures every pixel at the same moment and costs more. On a camera stage this is not a photography detail, it is the whole reason genlock exists.

The hard one

Depth cameras, point clouds, and staying aligned while you scan

A depth camera returns a distance per pixel, by one of three methods. Stereo uses two lenses and triangulates from the disparity between them — cheap, and it needs texture to match on, so a blank white wall defeats it. Structured light projects a known pattern and reads how the pattern deforms across the surface — very precise up close, and useless in sunlight. Time of flight measures the return time of modulated light per pixel — robust and longer range, with lower spatial detail.

The output is a point cloud: a large set of 3D coordinates, and nothing else. No surfaces, no objects, no meaning. Turning it into a mesh is a separate step, and turning that into something a CAD package can use is another.

And here is the problem nobody warns you about. A single scan only sees what is in front of it. To capture a room you take many, from many positions — and every one of them is in its own coordinate system, with its origin at the scanner. Making them into one model means finding the transform that puts each into a shared frame. That is registration, and it is the whole job — and it is the same coordinate-agreement problem that PSN and RTTrPM carry the results of.

How registration is actually done in the field

Targets. Place spheres, checkerboards or reflective markers around the space and make sure several appear in every scan. The software finds them, matches them between scans, and solves the transform exactly. Slow to set up, and the most reliable method there is — which is why survey-grade work still does it.

Cloud-to-cloud (ICP). Iterative Closest Point: guess a transform, pair each point with its nearest neighbour in the other cloud, compute the move that reduces the total distance, repeat. It converges beautifully when the scans overlap well and there is real geometric variety, and it fails in exactly the environments this industry works in — long featureless corridors, a black box studio, a flat empty floor. A shape with nothing distinctive in it has many equally good alignments.

Feature matching. Find distinctive local structures — corners, edges, planes — and match those instead of raw points. Faster and more robust than raw ICP, and still dependent on there being features to find.

SLAM. On a handheld or moving scanner, the device is building the map and tracking its own position in it at the same time — simultaneous localisation and mapping — usually fusing an inertial sensor with the visual or lidar data so that fast movement does not lose the thread. This is the same machinery as an inside-out tracked headset.

Drift, and the moment it gets fixed

Every registration is slightly wrong. Chain a hundred of them together walking around a building and those small errors accumulate — the far end of the scan is metres out of place, and nothing in the data itself has flagged a problem. This is drift, and it is the characteristic failure of any system that builds a map by adding to it.

The cure is loop closure. Return to somewhere already scanned and recognise it. Now there are two versions of the same place that ought to coincide, and the mismatch between them measures the total accumulated error. That measurement can be distributed backwards across the whole path, correcting everything at once.

Which gives the single most useful piece of field advice about 3D capture: close your loops. Walk a circuit and come back through somewhere you have already been rather than scanning out and stopping. And in a feature-poor space — an empty black box, a bare arena floor — put targets out, because neither ICP nor SLAM can align geometry that has nothing to distinguish one part from another.

The point

All of it is built around us

It is tempting to read this page as a list of clever machine tricks. It is closer to the opposite.

A QR code has 30% redundancy because a person will damage it. It has finder patterns because a person holds a phone crooked. An image sensor has three colour channels because we have three. It has more green pixels because we are most sensitive to green. OCR leans on a language model because the shapes alone are genuinely ambiguous — and the ambiguity is only resolvable because human writing is predictable.

And a 3D scanner needs registration and loop closure because a person carries it around a room on foot, in an order that made sense to them at the time.

None of these are machines perceiving the world. They are tools shaped around a person perceiving it — which is the same claim the perception and neuro pages make from the other direction, and the reason this stage of the site sits after that one rather than before it.