All Models are Wrong

We get asked this question a lot: how accurate is that model?

The short answer is: as accurate as it needs to be. Whenever we have measurements and/or dimensions available, our model reproduces them to the same degree of precision as the measurement came to us. Often, however, our models are based on a range of source info, including photographs, satellite imagery and police diagrams. We use a combination of visual judgement, knowledge of standard measurements, and, when the task demands it, a scientific process called photogrammetry to position objects in the model. Depending on the source material, that usually puts our level of accuracy somewhere between millimeters and a few meters.

So does that mean our models are wrong? Yes and no. Yes, because as British statistician George Box quipped, “All models are wrong, but some are useful.” No, because the flip side of that statement is that a model must be “wrong” in order to be useful.

For instance, a map is a two-dimensional scale model of a place. Its usefulness can be characterized by all the information it leaves out, all the ways that it is inaccurate and “wrong.” What gets included in a map are only the things the mapmaker thinks we need to know, the answers to the questions the mapmaker is trying to answer (What is the shape of the world? How do places relate to each other? How does one get from point A to point B?). A map is necessarily a reduction of reality.

Which leads us to another wonderful quote (credited to scholar Alfred Korzybski): “the map is not the territory.” (If you’re enjoying these quotes, read more here.) Well, obviously, you might say. But it’s an important precept to keep in mind, because it’s a distinction that easy to lose track of. That’s why our first task in any project is to clearly identify the objective(s) for any maps (or models) that we are being asked to create. A map (or model) is only as good as the clarity of its intention.

So back to the original question: how accurate is that model?

If the purpose of the model is to establish the path of a bus based on a photogrammetric process involving video footage and a survey of the scene, the model is very accurate (within a few centimeters across the area of an urban intersection).

If the purpose of the model is to show how an injury on a balcony occurred, the model might be very precise in some places (the railing that failed, which led to the injury), and approximate in others (the precise size and placement of windows, for example).

If the purpose of the model is to determine if muzzle flashes seen from a helicopter could have come from a homeowner’s porch, the camera position may be unknown (due to a lack of available information), but its line of sight can be deduced with a degree of accuracy sufficient to definitively answer the question.

As Dr. Box says, “all models are wrong.” The useful ones are only as “right” as their objectives need them to be.

Ady Leverette was a designer and a principal at Fat Pencil Studio between 2011 and 2018.