Get set for the cameraphone-enabled Internet Of Things

Next up speaking in the More Vision session at Nokia World is Dr Gaetano Borriello from the University of Washington. His session is called the Internet Of Things, but he’s focusing on using a cameraphone to overlay information onto the real world, as seen through your mobile screen.

It’s come from work with people with cognitive impairments – helping them get from A to B in unfamliar environments by overlaying directions onto a mobile screen. He’s showing a Nokia phone with a corridor viewable through the camera, but with satnav-style direction arrow overlaid on top, and a message saying ‘Take the next right at the end of this hallway’.

The way it works, is you start with a camera image, and the system calculates your location based on your phone’s Wi-Fi signal, and maps that against a floorplan of your location – to then figure out where you need to go, and paint that information onto the camera image. So it is kinda like an indoor GPS navigation system (without the GPS, obviously) that overlays directions onto a live image captured using your cameraphone.

On a purely practical level, I’m wondering how long the average Nokia Nseries handset’s battery would last if someone’s using the camera AND Wi-Fi for a sustained period of time.

Borriello is explaining how it all works in more detail, and how the system analyses your camera image and works out where you are in a building, by tracking key points in the image (corners, for example, or doorways in a corridor). It pulls these out of the image, and compares them to the floorplan to figure out where you are.

Flippant blogger remark No. 1: i could do with this when trying to navigate my house while tipsy after a big night out. Especially if they can tweak it to always lead me away from the fridge or takeaway menus. It’s a killer app!

Currently, there’s a latency time of 5-10 seconds – the time it takes to calculate your position and serve up the directions. But the researchers are working on that. They’re also figuring out how to make it work with curved corridors, and ones where the floor changes texture from carpet to, say, concrete.

There’s also obstructions. It works fine in an empty hallway, but what if people are walking down it towards you? It’s another challenge that the research team are tackling.

So, what’s the wider relevance of this project? “We really see this developing into a new paradigm for phone applications, of enhancing the physical world with information,” says Borriello.

Potential uses include a car engine application, providing you with a mobile app when you buy your car, with instructions on how to, say, replace your fan belt – and then you point your camera phone at the engine, and it overlays info on which bit is what, and arrows to show what you need to do.

Next example: home improvement – if you had the architectural details of how your home was built, you could have the wiring or pipes overlaid onto the walls as you look at them through a cameraphone, so you know where not to hammer a nail in.

And another one: ‘micro-finance – which has been designed for self-help groups in India, which involves putting barcodes on their forms and documents, and then a user points their cameraphone at the barcode, and it reads out instructions for people filling them in, telling them what fields need filling out – good for semi-literate users.

Borriello finishes up by looking to the future. Its hallway navigation system could expand to outdoor environments (wouldn’t GPS be the alternative here though?). Anyway, that’s it from him, aside from questions.

What about battery life issues? Borriello says that for most of the apps he’s demoed, power isn’t a problem – for example, in the car example, you’d just use it when you needed to fix the car, and you wouldn’t need Wi-Fi. “But clearly, the power equation could be an issue if you take this technology in certain directions,” he says.

Another question – could the real world adapt to make this easier – buildings putting markers in corridors for example, to help this kind of application calculate a user’s location more easliy? Yes, says Borriello. The example of barcodes in Japan shows that this can happen, “although in other cases, you might not want to put a weird marking on things.”

For the latest posts from the show, check our Nokia World 2007 category