I’d like to talk about augmented reality. Specifically, I’d like to talk about augmented reality, the things you carry in your pockets, and the things you wear around your neck.

Primer

AR basics: It’s an extension of the mobile Internet (that’s the one where people are walking around with computers disguised as phones in their pockets). The mobile Internet is what happens when devices become untethered from power, ethernet, and phone cords.

photo credit: tranchis

Let’s talk about finding a restaurant.

From the perspective of the Internet, it’s really not that interesting to know where my desktop is. It’s the same place it was yesterday and generally this fact cancels itself out. When I’m at home, a search for “Italian food” looks about the same whether I’m looking for a spot to meet a client for lunch downtown tomorrow or walking distance from my home tonight. So I have to put extra effort into telling the Internet my intentions (using other keywords and the like).

When a mobile like a Blackberry goes online, something important happens. You can’t assume that it’s in the same place that it was last time it connected. As location becomes variable it can become meaningful.

Equip a pocket computer with an Internet connection and GPS and it can start filtering “Italian restaurant” results by how close they are to the searcher. Give it a compass and a video camera and you can point your screen at things and get a video-game like overlay of the world. It can show you all kinds of stuff, such as which restaurants in your view have a 4 star rating or better. (I should note that I hate the overlayed reviews example, but it gets used a lot to illustrate the idea. More on this later.)

The notion is that there is a lot of metadata about the world which is only really relevant to people who are in a particular time and place. If the information superhighway was about bringing vast quantities of data to our fingertips in the home, augmented reality is about carrying that data back out and embedding it in the world. It’s about further blurring the line between the physical and digital. (For more, check out this overview by Will Wiles of Icon magazine.)

This is all in its very early stages. There’s a lot of excitement and money floating around, but we’re still very weak on actual useful applications running on usable devices. The eventual realization of these dreams will require a combination of hardware and software to really work. It’s not yet clear how that will look or behave.

So much for the basics.

Techno-Evolution

As the early engi-pioneers in flooding the physical world with metadata try to sort out how it’ll all work, I’ve noticed a curious process by which these devices are evolving into two distinct branches.

Let’s call branch 1: Things You Keep In Your Pocket. These are the iPhones, Blackberries, Androids and other smartphone-type things. The apps are stuff like Layar and Nearest Tube. Implicitly, I’ve been acting as if these are the only devices. Not so.

When I started writing this, it was a joke post. The title was Lanyards: Official accessory to your augmented reality and I was going to open with with this quip from Bruce Sterling:

I wish designers would stop imagining that customers really really want to carry imaginary geek-junk around their necks on lanyards. We all know that 99.995% of designers would rather be killed than wear a dorky lanyard, so why inflict that on the rest of us?

Bruce Sterling Beyond the Beyond

The joke would have been a string of photos from promotional material of people looking fabulous with near future tech toys hanging from their necks. Stuff like this:

Maptor by Jin-Sun Park and Seon-keun Park, featured on Yanko Design

So branch 2 is: Things You Wear Around Your Neck, AKA Technology on Lanyards. I’m fascinated by this branch.

(There is also a branch 3: Things You Wear On Your Head, but it’s gone dormant for some reason. It’s possible that through convergent evolution, branch 2 will come to look very much like what branch 3 might have been. Branch 4 is, I suppose, Things Embedded In Other Things. This’ll be when your car’s windshield has data displays glowing at you. Imagine a fighter jet’s HUD and you get the idea. The hold an image up to a webcam apps fall into this as well.)

Pockets

Right now, branch 1 apps are the most successful. They spread parasitically by running on top of devices that are already popular. There’s a lot of gee-whizery here. They make your boring old phone/computer/thing act like magic. It’s something you can pull out and show your friends. They are socially acceptable, because everyone you know already has a mobile phone.

They are also, for the most part, party tricks.

Take a look at the Nearest Tube demo again. Leaving aside the jittery icons, imagine yourself trying to use that thing to get around. It has arrows pointing you to the nearest stations but those arrows point as-the-crow-flies. You are in a city – in particular, you are in London. If there is one thing I can tell you about the route from your current location to the nearest tube station, it’s that it’s not a straight line.

View Larger Map

You are left holding your phone in front of you, constantly adjusting for the turns in the road. Sometimes you have to point the thing up into the sky, depending on where the icon stacking has seen fit to put your particular destination. Compare that to the (suddenly) old-fashioned Maps technique of showing you a birds-eye view of the surrounding area and a turn-by-turn suggested route.

Lanyards

The lanyard devices are quite different from the pocket ones. They are not even a little socially acceptable. No one looks good in a lanyard and if you are the kind of person who moves around much, they are get in the way all the time. They’re uncomfortable. They chafe the neck and if the thing on the end is at all heavy, they cause soreness. They evoke high school coaches with stopwatches and perky tour guides with clipboards.

They also imply a very different mode of interaction than the pocket devices.

Pocket augmented reality tends to being active, burst-use, and private. It uses tiny screens on a personal device. No one else is meant to see what it shows beyond the person holding it. You have to actively decide to take it out and make use of it, and it spends most of its time in sleep mode in your pocket.

In contrast, lanyard augmented reality is passive, always-on, and public. These are fuzzy boundaries, but take a look at a few prototypes and you’ll see what I mean.

I’ve already pointed to Maptor, a GPS device that projects a map of your immediate surroundings onto a nearby surface. Yanko’s pitch for it talks about how useful it is because no one need ever know you are lost. They gets this exactly backwards.

Lost and don’t want anyone to know? Projecting a glowing map will probably blow your cover. You are much better off pulling out your Droid and pretending you are looking at text messages while you find your way.

On the other hand, being able to unfurl a map where everyone in the group can see? That’s something missing from tiny screens. With Maptor, a group of people can gather around and point to the things they want to talk about, taking navigation back to the shared collaborative experience that it was when we had paper maps.

The Vicon Revue. Image from Reportages Photos

The Vicon Revue is a fish-eyed camera that hangs around your neck and takes photos every 30 seconds. This is perfect for reconstructing whatever happened after tequila number 5 or for self-incrimination at trial.

It’s also perfect for illustrating the passive part of lanyard augmented reality. With a pocket device, you need to pull it out of storage to start using it. You need to decide in advance that there’s something you want to capture. The Vicon reverses that. It’s always recording. Your decision is when to turn it off and, when you get home, what to edit out.

MIT’s Sixth Sense. Image from Dustin Kirk’s blog

Sixth Sense combines the camera and projector into a kind of mega-lanyard hybrid. Watch the demo here and see if you can feel the difference between that and a pocket device.

In its current state it’s clunky and socially clueless (the best moment is when it projects metadata onto the hapless acquaintance). It’s the kind of thing that only experimental engineers and conference-goers could love. But it’s also a fundamentally different attitude about the world and how the digital should bleed into it.

It’s always running, always scanning, always recording, and always waiting for instructions. Even picture-taking is changed. The act of framing the image you want isn’t taking a picture, not really. It’s telling the device which pictures to keep.

The projector is also very different from the private screen for all the reasons that Maptor is different from Maps on your phone. Other people can interact with the data. When he looks at the toilet paper and his rig tells him it’s OK, his roommate can be standing next to him and seeing the same conclusion. The digital isn’t being edited onto a video of the world, it’s being literally projected into it.

The name of the application speaks to the ambitions of the project. They want something that is so wholly integrated into your life that it feels like part of you.

It’s early days here in the augmented reality field and if the early developers keep making dorky devices, let’s cut them a little slack. After all, the Apple ][ became the Mac.

((Hi there, how’s your weekend going? This is slightly off-topic for Quiet Babylon, but it’s about the future of journalism which is one of my side-obsessions.

It concerns this hilarious opinion piece to which Jay Rosen linked on Twitter. Douglas Rushkoff is afraid that journalism (and by extension all content creation) can’t survive what he sees as Google’s parasitism and sees in Microsoft and Rupert Murdoch a glimmer of hope. There is so much wrong with the argument.

Hope you enjoy it. I promise that Monday will be about lanyards and augmented reality.)) Read the rest of this entry »

In the 1960s, the legend goes, NASA spent millions developing a pen that could write in zero gravity. The Russians used a pencil.

photo credit: christian.korsager

I couldn’t help but think of this story when I came across the following pair of headlines.

From WIRED:
Humans, Shmumans: What Mars Needs Is an Armada of Robots and Blimps

From the CBC:
Russia hopes nuclear ship will fly humans to Mars

In fairness to NASA, it’s worth pointing out that the legend is totally false. Fischer developed the Space Pen on spec and sold it to both the Americans and the Russians. A pen that can write is space in much better than a pencil – pens don’t have lead tips to break and leave dangerous particles floating around a spaceship and they aren’t flammable. After the Apollo 1 disaster, non-flammable objects seem pretty useful.

Move along, no pithy lessons about technology here.

1.

In 2008, Australian show A Current Affair broadcast an episode that included a brief hypnotherapy session. The segment was called Think Slim and the idea was that it would help viewers lose weight. This was found to be in breach of the Commercial Television Industry Code of Practice which specifically forbids broadcasting shows “designed to induce a hypnotic state in viewers”.

photo credit: rheanvent

2.

Take a look at this video of an augmented-reality physics app that Bruce Sterling linked to on Beyond the Beyond. You sketch objects in 2d and then the app figures out what they should look like and renders them in 3d. This results in simulated 3d objects that interact with one another and gravity. It’s all very cool, but pay special attention to how it’s controlled.

Did you see that? They use a scrap of paper marked with play or pause, which they slide into the frame when they want the simulation to start or stop. This strikes me as a big deal. It transforms the electronic eye from a data gathering device into something that can receive instructions. A control device.

When this thing is running, images become executable. You can send the computer commands visually. You can hack it with pictures.

Computers you can hypnotize.

3.

The thing that I find so appealing about retinal scanners is that it’s a technological re-imagining of the salt-of-the-earth gut-check folk wisdom of the need to look someone in the eyes. The machine peers into the depths of your soul and decides if you are who you really say you are and whether you should be allowed in.

Unless, of course, you are a guard rendered unconscious by the super-agent and dragged up to the scanner. Or you are a super-agent in possession of a scan of someone’s eye.

One way or another, the door gets opened.

4.

In 1997, 685 Japanese children were taken to hospitals because of seizures induced by an episode of Pokemon. The seizures were caused by a screen-filling pattern of flashing red and blue lights. When the news media picked up the story, some stations showed clips, which in turn caused more seizures.

The full episode was never aired again.

5.

The G60 grenades doesn’t kill, but they do produce a flash bright enough to blind targets and a bang loud enough to deafen, stun and even dizzy them. They are also known as flashbangs. Some version of the grenade use multiple detonation to further disorient opponents.

6.

The advent of 2d bar codes promises to make an increasingly machine-readable world. (It was meant to be RFID, but for my money, in an environment where every mobile device already has a camera, visual codes seem likely to win out.) Mix that with various augmented reality applications and the inconvenience of hitting “accept” all the time, and you have a powerful new vector for virii.

What?

Did you think that in a world where people happily install any old .exe that promises free porn or cat pictures and hand over banking details to sites that more or less look like their bank’s that mobile devices would somehow be free of this kind of problem?

7.

I was sitting in on a virtual meeting using Second Life. The sim was dressed up to look as much like a regular meeting as possible. It featured a stage, presenter, and a projection screen with powerpoint slides.

“Please stop clicking on the projection screen,” they begged us, “the permissions haven’t been set right and anyone can advance the presentation by accident.”

8.

Picture someone wearing a Sixth Sense rig or some other always-on smart-camera gear with overly trusting settings wandering into the wrong alley with the wrong images on the wall. Picture some jerk with a laser pulsing at a rate known to cause buffer overflows in unpatched rigs, carefully aiming at rich-looking patrons from across the dance floor.

9.

Visual input is messy and low-fi. It has to be, because of the wide variety of lighting and orientation situations with which the viewer is likely to be presented. Our brains automatically reorient and abstract the world. It’s a skill that lets us get by but it’s also a hurdle that artists need to unlearn in order to draw precise, verisimilar pictures of the world.

Putting control systems in optic inputs means teaching computers to reorient and abstract the world and in that translation lies vulnerability. Passwords work because there is only one way to enter your password. You either type in the correct string of characters or you don’t. Retinal scanners try to control the input by insisting on a narrow range of acceptable positions from the eyeball being scanned, but there is a lot of image processing required to turn that into a yes or no.

Facial recognition remains a nightmare, easily defeated by tilted heads, sunglasses, and bright smiles.

So when visual controls and codes are made they have to be highly error tolerant. Which means easily forged inputs. How do you protect against that? When we automate inputs, we aim for convenience, but we set ourselves up for a whole host of problems. Ambiguity breeds vulnerability.

Imagine QR code phishing, where a malicious user pastes a new sticker over a trusted one in an official location. Imagine your brat sister dropping her own play and pause scraps into the scene when you’re trying to put together your 3d drafting assignment. Imagine your jerk co-worker painting his nails your signature sixth sense colours. Imagine elite teams of special forces tossing spaz grenades, flashing their unique patterns that disable or reprogram the other guy’s auto-turrets.

10.

Imagine a digital sleight-of-hand artist.

“The hand is quicker than the camera…”

photo credit: Spanish Flea

I remember the first moment, a few years ago, at which I began to take the Internet seriously. It was a very, very silly thing. There was a guy, a computer research student at Carnegie Mellon, who liked to drink Dr Pepper Light. There was a drinks machine a couple of stories away from him, where he used to regularly go and get his Dr Pepper, but the machine was often out of stock, so he had quite a few wasted journeys.

Eventually he figured out, “hang on, there’s a chip in there and I’m on a computer and there’s a network running around the building, so why don’t I just put the drinks machine on the network, then I can poll it from my terminal whenever I want, and tell if I’m going to have a wasted journey or not?” So he connected the machine to the local network, but the local net was part of the Internet – so suddenly anyone in the world could see what was happening with this drinks machine.

Now, that may not be vital information but it turned out to be curiously fascinating; everyone started to know what was happening with the drinks machine. It began to develop, because the chip in the machine didn’t just say, “The slot which has Dr Pepper Light is empty,” but had all sorts of information; it said, “There are seven Cokes and three Diet Cokes, the temperature they are stored at is this and the last time they were loaded was that.” There was a lot of information in there, and there was one really fabulous piece of information: it turned out that if someone had put their fifty cents in and not pressed the button, i.e., if the machine was pregnant, then you could, from your computer terminal wherever you were in the world, log on to the drinks machine and drop that can! Somebody could be walking down the corridor when suddenly, bang! – there was a Coca-Cola can! What caused that? Well, obviously somebody five thousand miles away!

Now that was a very, very silly but fascinating story, and what it said to me was that this was the first time we could reach back into the world. It may not be terribly important that from five thousand miles away you can reach into a university corridor and drop a Coca-Cola can, but it’s the first shot in the war of bringing to us a whole new way of communicating.

Douglas Adams Q&A after a speech entitled Is There an Artificial God?

Adams is gave this speech in 1998 about something that happened a ‘few years earlier’. So let’s say 1996 or 1995.

When you get excited about things like @towerbridge, @lowflyingrocks, Google energy monitors, and Spimes, remember: the idea and the means was already there, we just had our attention elsewhere for awhile.