Globe and Mail

It's all in the wrist for playing cellphone games

Ontario company's technology turns your cellphone's camera into a wireless joystick - perfect for becoming one with virtual realities

Theresa Ebden
October 30, 2007

For most of us, shaking our cellphones is an activity reserved for those times when we're desperately trying to get better reception. But for a growing number of people in Japan and the United States, rocking and shaking mobile phones is a way to take aim at invading space aliens, hit tennis balls or go bowling. Thanks to the proliferation of cameras in phones, a bit of virtual reality is literally at hand. The newest technology from GestureTek Inc., started by two students 21 years ago in Canada, uses a cellphone's camera to detect movement, which essentially turns the phone into a joystick with no wires.

Called EyeMobile®, it's the latest development for a company that has, for two decades, used cameras as tools to interact with on-screen computer content from a distance. The Japanese mobile provider, NTT DoCoMo Inc., is GestureTek's largest client. "It became very natural to look at the cellphone when they were released around the world with cameras built into them," says Vincent John Vincent, 48, who cofounded GestureTek in 1986 with Francis MacDougall after they worked on a project together at the University of Waterloo. The Toronto-based company now has offices in Ottawa and the United States.

The new technology is similar to that of the company's cameras, used in popular gaming consoles such as Sony's EyeToy products running on the PlayStation2. A camera records the image around it and uses it as a reference point to measure movement experienced by the device, whether it's a 10-year-old boy holding a Nintendo wireless controller or a businessman holding his phone to navigate a drop-down menu.

For example, in some games developed in part by DoCoMo, users can play tennis, waving their hands like paddles in front of their phones. With bowling games, users view the lane on the screen and swing the phone as if they're rolling the ball - and hope they don't throw the phone by accident. In other applications, users can control the phone's menus using gestures instead of pressing buttons.

"It's meant to be an enabling technology as well as a fun technology," Mr. John Vincent says. DoCoMo, which has created 70 games that use the technology, recently released its first phones with GestureTek's technology. The phones sold out quickly and the technology will be embedded in the next generation of phones as well, Mr. John Vincent says. "They expect our technology to be on somewhere between five to 10 million phones by year end," he adds. The technology has taken off quickly in Japan, where "consumers really engage with the technology in a big way and are very interested," Mr. John Vincent says. "It's one of those communities that seems to be willing to adopt new forms of entertainment. They have an affinity for everything technology." Demand is lower in the United States, but GestureTek's technology is nonetheless included on 24 cellphone models in the Verizon network. It should be available in Canada within a year, Mr. John Vincent says.

To start a game, the user shakes the phone. Rocking it back and forth makes characters jump up or down, or lets a user move through menus. Rolling the phone around mimics traditional joystick commands. With traditional video-game navigation, players "use tiny little buttons - most of the games are about navigating a 3-D world and moving a character forward," Mr. John Vincent says. "With this, you can just move the [phone] forward, dip it down to the floor, and the character will move through the world in that way."

The system has its limitations. Because the camera uses the visual background to get its bearings, it might not work in a dark room or near the window of a moving train, Mr. John Vincent says. So far, the most popular game in the U.S. market is 3D Tilt-A-World, in which players roll a marble through a three-dimensional landscape. In Japan, the biggest hit is a game in which players roll a tiny ball that picks up pieces of lint and begins growing. Eventually the ball gets so large that it picks up larger items such as houses. Bowling and boxing games are also popular, Mr. John Vincent says.

GestureTek has about 75 employees, most of them in Canada, with the Toronto office running sales, administration and distribution, and Ottawa as its research and development base. It also has offices in New York City and near San Jose, Calif. Cellphones now figure prominently in the company's business plan, but it wasn't always that way. Until about seven years ago, most of GestureTek's applications were used in museums, science centres and corporate promotion events, Mr. John Vincent says. Using patents procured in 1991, GestureTek was able to control the market for the camera technology. Unlike other companies, however, it didn't use virtual-reality masks or data gloves.

"Our point of view was letting people make natural motions, not hold or touch anything," Mr. John Vincent explains. Now the future of the company is changing yet again, he says. GestureTek is working with physical and cognitive health professionals to create devices that will guide and measure patients' physiotherapy exercises, or lead them through exercises designed to stimulate the brain after a stroke or head injury. The company's technology also allows people to use their bodies to wave or point at billboards and displays, manipulating the image on it and making advertising interactive. Software giant Microsoft is looking at "surface" computing, which makes the keyboard obsolete by allowing users to manipulate images using gestures on a large computer screen that sits on a work table. It's this sort of computing that GestureTek is already marketing. "We see a big opportunity opening to us because the camera technology is everywhere," Mr. John Vincent says. "We really have this opportunity to keep developing the technology."

Theresa Ebden is an associate producer for Business News Network.

About GestureTek Mobile
GestureTek Mobile is a world leader in gesture-based user interface for mobile devices and the inventor of the patented, award-winning EyeMobile Engine.  EyeMobile Engine is the world’s first software-only solution that uses the existing camera on a cell phone or mobile Internet device to provide people with the ability to interact with their device using gestures. By shaking, rocking or rolling the phone (or making hand motions in front of the phone) users can answer a call, play mobile games, scroll menus, navigate maps, view images and documents, browse the web, enter text messages and do anything else they would normally do on their mobile device, without pressing buttons.  Licensees of GestureTek’s patents or technologies include Sony for the EyeToy, Microsoft for the XBOX 360 and Hasbro for the ION Educational Gaming System.  GestureTek Mobile’s award wins include the 2008 Mobile Innovation Global Award, the LAPTOP Magazine ‘Best of CTIA’ Award and the NATPE++ Award for the Hottest Mobile Application.  Games powered by the EyeMobile Engine have been recognized by the BREW Game Developer Awards, the International Mobile Gaming Awards and the IGN Editors Choice Awards.  GestureTek Mobile developed the first gesture-recognition software to be embedded in NTT DoCoMo phones in Japan and provided the software for the first gesture-controlled mapping application on a cellphone.  EyeMobile supports many handsets on the JAVA, BREW, SYMBIAN, WINDOWS MOBILE and DOJA platforms. Applications are available for over the air download on the Verizon network.  Full developer tools are available on the Qualcomm website.   A catalogue of turnkey games and applications for multiple platforms are available from the GestureTek mobile site. GestureTek Mobile is a business unit of GestureTek Inc., pioneer, patent-holder and world-leader in computer vision control for presentation and entertainment systems.