Varney's Place

Scroll to Info & Navigation

On Predicting the Future: Roku’s Reward and Augmented Reality

Predicting the future is a tricky business. It’s difficult to know what’s going to happen, and you never know whom you might inspire.


Kenwood has a long history of predictive technology projects: vision videos that provide a glimpse into a future that might—or might not—happen. Among others, these include:

For Apple, the granddaddy of all vision videos, which foreshadowed future touch-screen technology, mobile computing, face-to-face communications, Internet connectivity, and others—Knowledge Navigator at 25: Many Predicted Technologies Are Now in Place

For Sun Microsystems, a piece on data centers moving to cold-weather regions to take advantage of cheap power and natural cooling—Server Farms: Early Kenwood Video Foreshadows Big Data’s Impact on the Environment


Jim Samalis, who joined Kenwood as Executive Creative Director on April 1, was reminded recently of a visionary film he made years ago, and was rewarded by seeing the fruit of some seeds he helped to sow.

The story starts seven years ago. Jim worked for another company, charged with creating a video for Hewlett-Packard that tapped the new field of Augmented Reality. Phil McKinney, the client from HP, explained  to Jim: “Imagine you hold your device up to a landmark where a battle happened. With AR, you might be able to see in front of it, a scene from the battle that took place there.

“Now imagine playing a video game, where you’re interacting with your environment, and clues are attached to the environment. You’re on a scavenger hunt that’s taking you through San Francisco, and you’re gaining points and gaining information as you go from place to place. As you point your device up at the buildings, you get more information.”

Jim found the idea exciting. He created and produced a video for McKinney and HP called “Roku’s Reward.” (No relation to Roku, the current streaming-video service).

“We literally had to devise a game with little knowledge, and think about what the rules of the game would be, and how the game was played, and how you gain points,” says Jim. “But there was no game, right?  It was completely concocted by the director and me. So we came up with this game video idea, introduced a bit of a storyline, and pitched it to Phil.”

The video depicts a teenage boy physically running around San Francisco with a small gaming device that he uses to point at real buildings, streets, and landmarks, and to put himself visually into a virtual 3D gaming space. He collects points and bonuses and competes against other gamers also speeding through the streets of the City.

HP’s interest was in publicizing their “back end” telecommunications products and support services, not making video games. But this concept video had legs. “They ended up posting it online,” says Jim, “and we got 200,000 hits. I don’t know how people even knew it was there, at that point, or what they were searching for, or how they got to it. But it started to gain some momentum. Every time Phil gave a talk, people would say, ‘Oh, I saw that video, are you guys actually making that game?  When’s it coming out?’”

Roku’s Reward was seven years ago. “Fast-forward to four months ago,” says Jim. “I was picking up my four-year-old twins from school, and we got invited to a play-date. I met the other parents, we chatted, and I asked what they did for work.”

The other dad, whose name was Matt Miesnieks, said, “Well, I’m developing an Augmented Reality solution that works with iPads and iPhones.”

Jim was intrigued. “I told him I made a video many years ago about AR, and asked if that was the kind of thing he was talking about.”

Says Matt, “when I mentioned Augmented Reality, Jim instantly knew what I was talking about. This surprised me, as it’s not a widely understood domain. As we talked further he mentioned Roku’s Reward, which I’d seen and knew well. It was one of the first AR concept videos, and certainly the first one produced with high production values.

“I actually met with Phil McKinney while I was living in Amsterdam several years ago, working for one of the first mobile AR startups,” he continues. “We were able to pitch him on the current state of the AR industry and what would be needed to bring Roku’s Reward to life.”

Jim says, “That video always stuck with Matt, and now he was creating a game that worked very much like Roku’s Reward. And he pulled out his iPhone, and on his iPhone was this game.

“Matt’s game looked exactly like what we had captured in our fake game! In one of the early scenes in Roku’s Reward, you hold up your phone to the landscape, and the phone does a wire-frame mapping of the landscape, as it learns the 3D space.

“We were in his house, so Matt held up his phone to the chair, and at different angles to the chair. We could see a wire frame cover it, and now the game knew the chair was there. The game he developed has virtual racing cars that can race around the confines of your living room.”


Matt’s current San Francisco-based startup,, has recently secured $3.2 million in first-level, venture capital funding. Dekko recently announced “the launch of its real-world operating system to give developers a platform to build apps that merge mobile technology with the real world in unprecedented ways. Using mobile device cameras, Dekko builds digital grids over physical objects with 3D mapping. This technology allows mobile devices to interpret the real world and unleashes a whole new set of opportunities for mobile developers.”

Matt’s company has a demo of their own game on their website. They’ve made some changes in the basic concept, deciding for now to keep the game room-sized until wearable new mobile devices (such as Google Glass) take hold in the market.

“One of the things we’ve learned about users,” he says, “is that holding a smartphone or tablet up to their face is something they won’t do. They also don’t tend to want to leave their couch to run a few blocks to play a game. This means that based on the current state of the software and hardware, trying to realize Roku’s Reward still wouldn’t work today. The good news is it’s technically possible, but the user behavior changes are too great, in our opinion.

 “Dekko’s real-world OS will serve as the visual layer for wearable computing devices. We think that the concept of Roku’s Reward is strong and it will come about, but the technology to support it needs to come from eyewear, and the gameplay itself needs some deeper design around baking it into our daily lives.”

Matt and Jim enjoy their connection. Says Jim: “We both get a huge kick out of the fact that we had been tied to each other and didn’t even know it, until our kids ended up going to a pre-K program together.

“Sometimes these ideas go out into the ether, and you have absolutely no idea if they hit.  And when you find out that they have hit, it can be rewarding, and it can be nice to hear that you’ve actually captured somebody’s imagination.”

Matt says that Roku’s Reward “certainly had an influence, as the AR space is still mostly concept videos, science projects or marketing gimmicks. It’s the concept videos that spark the imagination, especially for people who are new to the space. Roku’s Reward was influential because it showed an actual credible use-case (outdoor location based gaming) with a level of production quality and narrative that helped people see the experience, not the technology.”

More about and Augmented Reality:

With $3.2M In Funding, ‘Real-World Operating System’ Startup Dekko Refocuses To Build Its Own Augmented Reality Apps

Xbox? More like Xbody: Future game consoles will get under your skin

Google Glass minus glass: Dekko makes the world your OS

Dekko wants to take augmented reality beyond a ‘marketing gimmick’