Colosseo: Reimagining the Roman Coliseum with type (Canon 7D) from Cameron Moll on Vimeo.

Cameron Moll has created a poster that depicts the Coliseum using type. The Colosseo is a gloriously hybrid entity, digitally produced but mechanically reproduced. The prints are these beautiful objects, but the Colosseo is also data. You can buy parts of the data as vector art glyphs, while a low-resolution digital copy flies around the Internet.

The artwork is great but I’m sharing this for the video, which first lovingly depicts and then explicitly discusses the fetishistic craftsmanship of printing the posters. In fact, the video devotes far more time to the process of reproducing the work than to the time spent creating it, which was done on a computer and much more time-consuming. (Moll has released other videos focussing the act of digital creation.)

Here’s why this fascinates me:

An analysis of art in the age of mechanical reproduction must do justice to these relationships, for they lead us to an all-important insight: for the first time in world history, mechanical reproduction emancipates the work of art from its parasitical dependence on ritual. To an ever greater degree the work of art reproduced becomes the work of art designed for reproducibility. From a photographic negative, for example, one can make any number of prints; to ask for the “authentic” print makes no sense. But the instant the criterion of authenticity ceases to be applicable to artistic production, the total function of art is reversed.

Walter Benjamin The Work of Art in the Age of Mechanical Reproduction

And yet, here are Cameron Moll and Bryce Knudson managing to impart all kinds of aura and ritual to the reproduction. The reproductions are weirdly more authentic than the original which is just a file with dubious forward-compatibility.

I enjoy this alchemy, made possible by the presence of easier reproduction techniques. It transmutes the time needed to make a letterpress work into painstaking labour when, at the moment of invention, it was labour-saving. Imagine the salespeople and inventors of these machines learning that their long term legacy would be assured by how difficult they are to use, compared to their displacing successors (yes, yes, I know there are special features of the resulting print that are unique to the process but the video is all about the process itself).

What I’m deeply curious about is what comes next. At what point will the techniques have morphed and changed to that point that lovingly submitting PDFs to be printed “by hand” on colour printer feels more authentic than whatever’s replaced it? I suppose we’re about due for dot-matrix nostalgia.

I think we’re already seeing some glimpses of that sentiment in essays like this one:

I want to make things, not just glue things together.

Mike Taylor Whatever happened to programming?

In the summer of 2009, I was in San Francisco for the first time and on my way to meet Alexis Madrigal and Sarah Rich for a drink. Equipped with only a photocopied map and a dumb cellphone, I got off at the appointed BART stop with instructions to head south and no idea which way that was. Ever the intrepid explorer, I worked out the solution using the phone’s clock, the map, and the location of the sun. That’s so remarkable that it’s worth saying a second time: In 2009 in a major metropolitan area, confused and disoriented I resorted to navigation by the sun.

photo credit: teresawer

Here’s how that story goes in Edmonton, a city with which I am equally unfamiliar: I get off at the appointed stop, pull out my smartphone, put in the address, and the phone works out where I am and points me to my destination.

The difference? International roaming charges haven’t crippled me.

A fair number of future-facing writers like to call various aspects of our connected world their outboard brains. It’s a cute conceit but also an aspirational statement. It looks forward to the implanted memories and off-loaded cognition promised by cybernetics.

We already have prototypical versions of that to some degree. Just about everyone uses a calculator for simple math, many of us offload scheduling memory to a physical or digital calendar – that’s all elementary “everyone’s a cyborg” stuff. Expect this to intensify. The promise of intelligence in the cloud is that we get access to terabytes of data as needed, and that this access will make us better whatever it is we are trying to be.

Here’s how the San Francisco story goes in 2002: I get off the BART and my hosts are waiting to meet me, because they know it’s easy to get turned around OR I get off the BART and see the local landmark that I was careful to ask my hosts about, so that I could situate myself when I emerged from the station. In 2002 this is a natural part of the flow of planning. In 2009, it doesn’t enter into consideration until it’s too late. There’s an assumption by everyone involved in the planning process that getting from the exit to the bar is a solved problem, so it isn’t discussed.

Here’s why this is interesting: As knowledge and information move further and further away from being something we have towards being something we process, we become increasingly reliant on the machines that enable this relationship. Having knowledge becomes an increasingly contingent and fragile state. As this stuff advances, there comes a point when the connectivity becomes mandatory instead of optional and unconscious instead of controllable.

This is the wild extreme of the transformation of intelligence documented in Lyotard’s The Post Modern Condition.

We may thus expect a thorough exteriorisation of knowledge with respect to the “knower,” at whatever point he or she may occupy in the knowledge process. The old principle that the acquisition of knowledge is indissociable from the training (Bildung) of minds, or even of individuals, is becoming obsolete and will become ever more so. The relationships of the suppliers and users of knowledge to the knowledge they supply and use is now tending, and will increasingly tend, to assume the form already taken by the relationship of commodity producers and consumers to the commodities they produce and consume.

Jean François Lyotard The Post Modern Condition: A Report on Knowledge

Looking ahead to a time when these machines are more thoroughly integrated, we end up with some profoundly weird consequences. Travellers become literally less intelligent when they leave their coverage area, relative to their connected hosts. A civil emergency occurs because a brief service outage leads to a poor decision by a plant manager. Data corruption causes a segment of customers to suffer a kind of patchwork amnesia. Rumours abound of hackers able to execute man-in-the-middle attacks that allow them to lift and shift memories. Parents and school administrators spar over what constitutes fair or unfair augmentation when it comes to state testing. Augmented students stripped of their connections fare far worse than their have-not peers. When the machines are active, the scores are very different.

For a glimpse into your connected future, consider the case of Steve Mann, wearable computing pioneer. He’s been connected to various devices for the past 20 years and has become used to a computer-mediated relationship with the world. At the height of post 9/11 security paranoia, some overzealous airport guards decided they needed to see his rig removed. In the process of the inspection, some equipment was damaged and all of it torn off his body.

Without a fully functional system, he said, he found it difficult to navigate normally. He said he fell at least twice in the airport, once passing out after hitting his head on what he described as a pile of fire extinguishers in his way. He boarded the plane in a wheelchair.

By Lisa Guernsey At Airport Gate, a Cyborg Unplugged for the New York Times

If misplacing your cellphone gives rise to a panic beyond what would be reasonably expected for a few hundred dollar expenditure, you are beginning to get there. If you no longer remember addresses, you simply refer to a slip of paper, you are well on your way.

There will be outages. There will be coverage problems. There will be billing issues.

On January 12th, Google announced that they’d been the target of a series of cyber attacks aimed at accessing the Gmail accounts of Chinese human rights activists. Their response was to stop censoring Google.cn effective immediately. In the same week, Google was granted a patent that would enable them to replace real billboards with virtual ones in Google Street View.

Leaving aside the utter surprise at finding myself on the pro-corporate side of the government v. megacorp sovereignty wars, both of these stories are about the same thing: filtering data streams to match one set of aims or another.

photo credit: Barabeke

The China story is a story of the filtration with which we are most familiar. Censorship is old, well understood and for the most part opposed by people of conscience. Though, Google hasn’t yet announced a unilateral end to their censoring practices in France and Germany and there are plenty of things that you aren’t allowed to say all over the world.

The replaceable ads feel like something else entirely. If they get implemented, it probably won’t be a wholesale replacement on a political agenda. It’s more likely to be a patchwork replacement of the real. There’s an appealing elegance to the move. They were ads anyway, now they are just different ads. (Weirdly, many of the real billboards are there illegally. There is a troubling chemistry at work when virtual advertising space is being made available on the basis of improperly placed real advertisements.)

So much for the governmental and corporate.

There is a third layer of filtration, the personal layer. By putting up a set of filters, individuals can customize their experiences; cutting away the unwanted and subbing in whatever they please.

Consider the ways that you can remake YouTube in your image: you can make videos downloadable; you can filter out all the comments that are algorithmically determined to be stupid; you can even filter out the whole site. Sites like Give me back my Google attempt to remove affiliate and other spam-type links that Google hasn’t filtered themselves.

Subtractive filtering isn’t the only approach. Add Art is a Firefox plugin that replaces online advertising with curated art shows. The shows consist of images formatted to the standard ad dimensions which replace real advertising on the fly. (I tried Add Art for awhile but had to uninstall it when a show consisting of rapidly strobing GIFs rotated in.) If Google implements Street View virtual ads, maybe Add Art can implement Street View art shows on top of that.

Having a good set of filters is crucial. We are bathed in a sea of insistent data and augmented reality is only going to make it worse. As Bruce Sterling points out, an issue with augments is that they don’t scale. If you’re looking at any given square of surface, there’s only room for so much text, video or images. Very, very quickly, too many people will be trying to get your attention at once. You’ll need heavy filters to handle it all. What if they get too powerful?

Overly powerful personal filters is a recurring fear. We worry that people watch the news, go to church and generally build communities that confirm their opinions. Self-imposed censorship is probably the hardest to fight as there isn’t curiosity or fascination with the forbidden to turn leaks into torrents.

Take the moment in John Carpenter’s They Live when Roddy Piper puts on the glasses.

Now, imagine that this guy got together the tools to make a set of glasses that highlighted occult symbols automatically. Imagine a set of glasses that filtered out objectionable images for whatever value of “objectionable images” you care to substitute.

It’s pre-augmented reality but the 2004 film Epic 2014 presents a compelling vision of a world where the filters have won in a big way. The resulting output is both the best curation of the news-you-need-to-know and the worst set of bias-confirming pablum, depending on the preferences and habits of the user.

The bias confirmation fear has been around for a long time. It was the main theme of my grade 11 “Media Literacy” class. The key has shifted from centralized media control to fears about distributed extremist groups, but the chorus is the same.

It wasn’t until this past week that I worked out the problem with the analysis. The trajectory assumed is of increasingly powerful and impregnable filters. If that trajectory holds, then one expects an increasingly balkanized culture, full of isolated groups that think they have nothing in common. But there’s a second set of actors in play, the ones being filtered out.

As the first group works harder to filter out unwanted messages, the second works harder to break through. We see it in the arms race around advertising. We see it in politicians struggling to find new ways of reaching their audience. We see it in Google’s need to constantly change and update their pagerank algorithms as black hat SEOs learn to game the system.

So long as the arms race continues, the filters will get better without becoming perfect. And in those cracks, reality (or at least an alternate viewpoint) can intrude. Insofar as we believe that people can’t know in advance what is best for them or what information they should receive, we should celebrate inefficiencies in filters.

In every successfully delivered spam message, there is a ray of hope.

When I was working on the idea of the pocket-device model of augmented reality versus the lanyard model, I realized something about conference badges that didn’t really fit into the piece.

photo credit: jdlasica

Conference badges are little ancient proto-augments.

They are a way for people to carry around and display metadata about themselves. The basic bit of data is “I am allowed to be here”. Most conference passes add to that details such as who you are and who you work for. Some conference badges add in slots for customization such as press-clippings, your business card or whatever else you feel like tossing in there.

Everywhere you go, there’s the badge, broadcasting who you are and whether you are authorized. Anyone around can take a look and grab your data (such as it is). They are – as in all the lanyard augments – public, passive, and always-on.

I’d like to talk about augmented reality. Specifically, I’d like to talk about augmented reality, the things you carry in your pockets, and the things you wear around your neck.

Primer

AR basics: It’s an extension of the mobile Internet (that’s the one where people are walking around with computers disguised as phones in their pockets). The mobile Internet is what happens when devices become untethered from power, ethernet, and phone cords.

photo credit: tranchis

Let’s talk about finding a restaurant.

From the perspective of the Internet, it’s really not that interesting to know where my desktop is. It’s the same place it was yesterday and generally this fact cancels itself out. When I’m at home, a search for “Italian food” looks about the same whether I’m looking for a spot to meet a client for lunch downtown tomorrow or walking distance from my home tonight. So I have to put extra effort into telling the Internet my intentions (using other keywords and the like).

When a mobile like a Blackberry goes online, something important happens. You can’t assume that it’s in the same place that it was last time it connected. As location becomes variable it can become meaningful.

Equip a pocket computer with an Internet connection and GPS and it can start filtering “Italian restaurant” results by how close they are to the searcher. Give it a compass and a video camera and you can point your screen at things and get a video-game like overlay of the world. It can show you all kinds of stuff, such as which restaurants in your view have a 4 star rating or better. (I should note that I hate the overlayed reviews example, but it gets used a lot to illustrate the idea. More on this later.)

The notion is that there is a lot of metadata about the world which is only really relevant to people who are in a particular time and place. If the information superhighway was about bringing vast quantities of data to our fingertips in the home, augmented reality is about carrying that data back out and embedding it in the world. It’s about further blurring the line between the physical and digital. (For more, check out this overview by Will Wiles of Icon magazine.)

This is all in its very early stages. There’s a lot of excitement and money floating around, but we’re still very weak on actual useful applications running on usable devices. The eventual realization of these dreams will require a combination of hardware and software to really work. It’s not yet clear how that will look or behave.

So much for the basics.

Techno-Evolution

As the early engi-pioneers in flooding the physical world with metadata try to sort out how it’ll all work, I’ve noticed a curious process by which these devices are evolving into two distinct branches.

Let’s call branch 1: Things You Keep In Your Pocket. These are the iPhones, Blackberries, Androids and other smartphone-type things. The apps are stuff like Layar and Nearest Tube. Implicitly, I’ve been acting as if these are the only devices. Not so.

When I started writing this, it was a joke post. The title was Lanyards: Official accessory to your augmented reality and I was going to open with with this quip from Bruce Sterling:

I wish designers would stop imagining that customers really really want to carry imaginary geek-junk around their necks on lanyards. We all know that 99.995% of designers would rather be killed than wear a dorky lanyard, so why inflict that on the rest of us?

Bruce Sterling Beyond the Beyond

The joke would have been a string of photos from promotional material of people looking fabulous with near future tech toys hanging from their necks. Stuff like this:

Maptor by Jin-Sun Park and Seon-keun Park, featured on Yanko Design

So branch 2 is: Things You Wear Around Your Neck, AKA Technology on Lanyards. I’m fascinated by this branch.

(There is also a branch 3: Things You Wear On Your Head, but it’s gone dormant for some reason. It’s possible that through convergent evolution, branch 2 will come to look very much like what branch 3 might have been. Branch 4 is, I suppose, Things Embedded In Other Things. This’ll be when your car’s windshield has data displays glowing at you. Imagine a fighter jet’s HUD and you get the idea. The hold an image up to a webcam apps fall into this as well.)

Pockets

Right now, branch 1 apps are the most successful. They spread parasitically by running on top of devices that are already popular. There’s a lot of gee-whizery here. They make your boring old phone/computer/thing act like magic. It’s something you can pull out and show your friends. They are socially acceptable, because everyone you know already has a mobile phone.

They are also, for the most part, party tricks.

Take a look at the Nearest Tube demo again. Leaving aside the jittery icons, imagine yourself trying to use that thing to get around. It has arrows pointing you to the nearest stations but those arrows point as-the-crow-flies. You are in a city – in particular, you are in London. If there is one thing I can tell you about the route from your current location to the nearest tube station, it’s that it’s not a straight line.

View Larger Map

You are left holding your phone in front of you, constantly adjusting for the turns in the road. Sometimes you have to point the thing up into the sky, depending on where the icon stacking has seen fit to put your particular destination. Compare that to the (suddenly) old-fashioned Maps technique of showing you a birds-eye view of the surrounding area and a turn-by-turn suggested route.

Lanyards

The lanyard devices are quite different from the pocket ones. They are not even a little socially acceptable. No one looks good in a lanyard and if you are the kind of person who moves around much, they are get in the way all the time. They’re uncomfortable. They chafe the neck and if the thing on the end is at all heavy, they cause soreness. They evoke high school coaches with stopwatches and perky tour guides with clipboards.

They also imply a very different mode of interaction than the pocket devices.

Pocket augmented reality tends to being active, burst-use, and private. It uses tiny screens on a personal device. No one else is meant to see what it shows beyond the person holding it. You have to actively decide to take it out and make use of it, and it spends most of its time in sleep mode in your pocket.

In contrast, lanyard augmented reality is passive, always-on, and public. These are fuzzy boundaries, but take a look at a few prototypes and you’ll see what I mean.

I’ve already pointed to Maptor, a GPS device that projects a map of your immediate surroundings onto a nearby surface. Yanko’s pitch for it talks about how useful it is because no one need ever know you are lost. They gets this exactly backwards.

Lost and don’t want anyone to know? Projecting a glowing map will probably blow your cover. You are much better off pulling out your Droid and pretending you are looking at text messages while you find your way.

On the other hand, being able to unfurl a map where everyone in the group can see? That’s something missing from tiny screens. With Maptor, a group of people can gather around and point to the things they want to talk about, taking navigation back to the shared collaborative experience that it was when we had paper maps.

The Vicon Revue. Image from Reportages Photos

The Vicon Revue is a fish-eyed camera that hangs around your neck and takes photos every 30 seconds. This is perfect for reconstructing whatever happened after tequila number 5 or for self-incrimination at trial.

It’s also perfect for illustrating the passive part of lanyard augmented reality. With a pocket device, you need to pull it out of storage to start using it. You need to decide in advance that there’s something you want to capture. The Vicon reverses that. It’s always recording. Your decision is when to turn it off and, when you get home, what to edit out.

MIT’s Sixth Sense. Image from Dustin Kirk’s blog

Sixth Sense combines the camera and projector into a kind of mega-lanyard hybrid. Watch the demo here and see if you can feel the difference between that and a pocket device.

In its current state it’s clunky and socially clueless (the best moment is when it projects metadata onto the hapless acquaintance). It’s the kind of thing that only experimental engineers and conference-goers could love. But it’s also a fundamentally different attitude about the world and how the digital should bleed into it.

It’s always running, always scanning, always recording, and always waiting for instructions. Even picture-taking is changed. The act of framing the image you want isn’t taking a picture, not really. It’s telling the device which pictures to keep.

The projector is also very different from the private screen for all the reasons that Maptor is different from Maps on your phone. Other people can interact with the data. When he looks at the toilet paper and his rig tells him it’s OK, his roommate can be standing next to him and seeing the same conclusion. The digital isn’t being edited onto a video of the world, it’s being literally projected into it.

The name of the application speaks to the ambitions of the project. They want something that is so wholly integrated into your life that it feels like part of you.

It’s early days here in the augmented reality field and if the early developers keep making dorky devices, let’s cut them a little slack. After all, the Apple ][ became the Mac.