The map and the territory

The tech industry continues to surprise me in its ability to totally ignore irony. It was bad enough, a decade ago, when a meal replacement drink for bros got named Soylent. As we know from cultural osmosis of classic science fiction, Soylent Green, the most delicious of the Soylent varieties (in the eponymous 1973 film, Soylent Green) is made of people. Why in the world would anyone want to risk having their product be even tangentially associated with such an idea? In the intervening time, there’s been much discussion about the “hold my beer” attitude of a little company called Palantir. The surveillance technology firm is saying the quiet part out loud by naming itself after an extremely morally- and technically-questionable scrying stone from The Lord of the Rings series. Surely, you’d think, Soylent and Palantir have already brought us to peak irony deficiency. But no, they have not.

Last week, I read a broadly very fine article published by MIT Technology Review (this one). It was about how the activities of Pokémon Go and Ingress players are being used to improve the precision of delivery robots. For your context, these two games involve going around and pointing your phone camera at things in order to have an augmented reality experience in a certain location. Apparently this has resulted in a huge trove of granularly geotagged imagery of locations where there's an Ingress portal, a Pokémon training gym, or another feature which makes the location popular for players. These photos are now being used to improve positioning of delivery robots in areas where GPS has relatively poor performance, but getting a location exactly right makes a big difference. All well and good. Did we ever really expect that games which harness the cameras and location data of players wouldn’t be used for something more profitable than catching imaginary monsters? But here’s the kicker: at the end of the article, the author describes the goal of Niantec Spatial (a spin-out from Niantec, the company behind Ingress and Pokémon Go) as “capturing everything.” The CTO of the company is then quoted as saying, “We’re not there yet, but we want to be there. I’m very focused on trying to re-create the real world.”

Now, maybe I’m over-eagerly jumping on a little journalistic flourish. After all, “capturing everything” could be a sly nod to Pokémon and the imperative to “Catch ‘em all.” Or it could be the desire of the journalist to put the CTO’s statement into a broader context. But we have here a direct quote from someone who is apparently striving to “re-create the real world” in a representation of the world which will be used for, among other things, making the spatial placement of delivery robots more precise than they’re currently capable of being. This is being done with the help of a vast cache of geotagged images of the real world. Images gleaned from Pokémon Go and Ingress players will inform a hyper-precise representation of the world they have captured. Sounds useful enough. Where’s the irony deficiency? It's in the knowledge that this idea is a clear refutation of the old axiom that the map is not the territory. Instead, the business model appears to exactly be making the map into something as close to the territory as technically, humanly, and financially possible.

On exactitude in mistaking the map for the territory

Why does the idea of a near-perfect visual representation of the world get up my nose so badly? I have spent the last ten or more years trying gently to argue against this idea, when possible. I’m of course not the only one trying to bat away the dream of perfectly representing reality. Innumerable others have gotten there first, and some have been doing the good work for over a century. Lewis Carroll wrote amusingly about it in Sylvie and Bruno Concluded, with the end game of the 1:1 scale map being that it would risk, if overlaid on the territory completely, blocking out the sun and preventing the crops from growing. Similarly, Jorge Louis Borges wrote the most wonderful of extremely short short stories about the risk of trying to map the territory in such detail (it is here, and worth the five minutes it takes to read). In the end, the ambitious country trying to make a perfect map gives up on the discipline of cartography entirely. The axiom that the map is not the territory is not just about fantasy, but about the fallacy of trying to create perfect representation, and the risk that if you achieve something close to that 1:1 mapping, you might even believe your own representation. The now-common way of expressing this concern, that the map is not the territory, is attributed to the philosopher Alfred Korzybski, and has by now become axiomatic when discussing the risks of believing models or representations instead of looking to what they're representing ((for a primer on this, check out the relevant Wikipedia entry).

Thanks to recent technical and business developments, the need to challenge the idea of perfect reproduction of spatial reality feels more urgent than ever. Niantec Spatial is not the only company banking on the idea of more-perfectly representing some aspect of the world. The data-hungry technologies currently gaining hype, both in the popular imagination and in certain fields, have the idea of more-perfect representation through mass-data slurping at their core. We could talk all day about the risks of believing that language is thought (one of the core fallacies in the context of imagining large language models as a stepping stone to some kind of notional artificial general intelligence), but I won’t do that here. Instead, I want to stick with the impetus towards representations of things, places, and people.

Increasingly granular maps

The easy availability and capturability of data about an increasing number of phenomena (whether it’s a constant log of your heart rate thanks to a smartwatch, or real-time information about the location of every vehicle in a city’s public transit fleet) creates new opportunities to believe in the reality of the map. Just committing the logical fallacy of mistaking the map for the territory isn’t a crime, but the risk is in the combination of increasing the scope of the map while simultaneously using it to do more. By this I mean that it’s one thing to know that fifty people have used their transit passes to tap into a given subway station in the last ten minutes, and another thing to assume that the number of taps exactly reflects the number of riders present. The number of taps tells us nothing about how many extra riders tailgated in order to catch a free ride, for example. This is a small and comparatively simple example of how an apparent reality reflected in data can prove to be an incomplete picture of what is actually happening. With something as simple as decoupling the number of taps from our assumption about rider numbers, it could be relatively easy to cross-reference the data with another source (say, crowd counting cameras in the station) or with historical information representing what’s known about number of paid versus unpaid riders. The taps do not need to be the only data. A conscientious public transit authority interested in how many riders are present (and perhaps more concerned with that than with other issues) might decide to put all its efforts into smashing its different data streams together in order to more accurately represent the number of riders in the station at a given moment.

The risk, in this toy example of tailgating transit riders, is that the more data we add, the more we may feel confident that we have the full picture. With every new way of cross-referencing, we feel more confident in the belief that we have a grasp of reality. As the map approaches 1:1, we begin to mistake it for the reality, and we feel increasingly confident that the representation is the truth. And while the transit riders may be a comparatively low-stakes example, innumerable more dangerous and risk-heavy cases are playing out. False positives in facial recognition systems fall into this category. When a match between two images is presented as authoritative, and when we do not approach such computational certainty with the care and criticality it deserves, real people come to harm. I could also mention incidents like the Dutch childcare benefit scandal (the toeslagenaffaire, a word that has become shorthand for egregious harm when talking about algorithmic injustice), in which factors attached to historical fraud cases were taken as credible indicators of future fraud, leading to the re-marginalization of people who were already at risk. When the representation is used for and in place of the reality, actions can be taken which wildly diverge from those which would be reasonable in real life, but feel right on our maps.

In essence, as the sensors, quantities of data, and computational power needed to record and correlate everything become increasingly pervasive, we’re making an old mistake at turbo-speed. As with many fields at the moment, there’s a tendency to mistake having a lot of data for being able to reproduce reality. While there are fields where modelling works, it works when the use case is suitably scoped, and the goal is not the complete replication of reality. The greater the belief in the reality of the representation, the greater the risk of scoping outside what the representation is capable of accurately representing. When we mistake the map for the territory at the expense of examining the territory itself, we risk believing our own stories and representations at the expense of understanding what is in front of us.