Making TVs smart: why Google and Netflix want to reinvent the remote control

Making TVs smart is a three-part series that looks at why smart TVs have failed to take off — and what needs to happen for these devices to realize their vast potential. You can read part one here and part two here.

When Google (s GOOG) TV debuted back in 2010, it represented a radical take on smart TVs. Google wanted to combine live TV with internet content, and offer viewers a seamless way to switch from an ABC broadcast to a Netflix stream. The first generation of Google TV devices failed for a variety of reasons.

But all many people remember are those crazy remote controls.

Sony's first-generation Google TV remote control had 80 buttons.
That’s a lot of buttons: Sony’s first-generation Google TV remote control.

Logitech’s (s LOGI) Revue companion box shipped with a full-size QWERTY keyboard better suited for an office desk than a living-room couch, and Sony’s (S SNE) Google TV devices introduced a monster of a remote control that had no fewer than 80 buttons.

Google has learned from these mistakes and, along with a number of other companies, is trying to rethink how people interact with their TVs. Keeping this kind of interaction simple as TVs get smarter turns out to be a big challenge. Cracking this nut could open the door to a whole new range of applications, which is one of the keys to getting users excited and finally turning smart TVs into a success story.

Changing the channel without knowing the number

Google’s initial decision to make Google TV devices with a full QUERTY keyboard was prompted by a problem anyone who has ever tried a streaming box like Apple (s AAPL) TV or Roku knows well: Search, or even entering account credentials, is a royal pain.

Most streaming boxes come with remote controls that are built around a so-called D-Pad — buttons to navigate up, down, left and right. To search, devices use on-screen keyboards, leaving users with the slow, frustrating task of manually jumping from one letter to the next to type.

Companies like Roku have are using auto-suggestions to minimize the need to type, but onscreen keyboards are nonetheless pretty painful.
Companies like Roku are using auto-suggestions to minimize the need to type, but onscreen keyboards are nonetheless pretty painful.

Google TV has always heavily emphasized search, and the D-Pad simply wasn’t the right solution for that approach. After failing with the full keyboard, Google took a different route for its second generation of devices: remote controls with microphones. After pressing a mic button, you just speak up to change the channel, search for a YouTube video or launch an on-demand movie.

“Voice has been around for decades, but for a long time, it wasn’t very good,” Eric Liu, Google TV Product Manager, told me during a recent interview.

[soundcloud url=”” params=”” width=” 100%” height=”166″ iframe=”true” /]

He explained that Google TV uses the same cloud-based voice recognition technology as Android phones. That means that the millions of people who use voice to interact with their smart phones are also helping to improve voice control for Google TV.

Google is trying to make sense of what people want so they’ll be able to change the channel without knowing complicated voice commands, or even channel numbers. Instead of entering channel 47, users simply say “CNN,” and the TV automatically switches to the right channel.  “This is a Google problem to solve because we know what channel CNN is on,” Liu said.

Your TV is not your friend

Google isn’t the only one using voice to control smart TVs and TV-connected devices. Microsoft’s (s MSFT) Xbox gaming console has had voice control for some time, and Samsung is also using voice for its new smart TVs.

Other manufacturers have been experimenting with other ways to move beyond the D-Pad. LG TVs come with a Wii-like remote control that can be used to control a pointer on the TV screen. The Xbox can be controlled by gestures when combined with a Kinect motion sensor, and TV manufacturer Haier even showed off eye tracking at the 2013 CES as a possible way to interact with the smart TVs of the future. It would allow users to simply look to the bottom of the screen to scroll through menus or select menu items.

smart tv feature kinect
Xbox 360 owners can use Microsoft’s Kinect to navigate through apps like Netflix. But do we really need to learn new gestures to skip from one movie to another?

“Remote controls aren’t bad,” Liu said, they’re just not the best solution for every type of situation. Instead of trying to replace the entire remote control with voice or even gestures, Google concentrated on those types of interactions that would take a long time with a traditional remote control.

For example, you can change the volume just fine with a volume button– trying to create a new process for that would just be adding unnecessary complexity. Text input, on the other hand, is something that the traditional remote doesn’t do well.

Another challenge is ensuring that the technology doesn’t make users feel uncomfortable. Speaking to a computer, or even a TV, puts users on the spot. “People have a real nervousness when interacting with technology,” Liu said. That’s why Google TV uses a dedicated button to start voice controls, giving users some time to prepare. Google also consciously skipped any type of Siri-like spoken interaction. It leads the user straight to relevant apps and content sources instead of pretending to do small talk. “This is not a friend,”  Liu said of the remote control.

Almost like AirPlay, but open

Speaking of Siri: The one company that has arguably been the most successful at redefining interaction with the TV is Apple. (s AAPL) AirPlay, which Apple added to its mobile operating system in 2004, allows users to stream audio, video, photos and even a mirror image of their entire desktop straight from their iPhone, iPad or Mac to an Apple TV.

In this scenario, the smart TV once again becomes merely a display — the content itself is on users’ mobile devices. And the best thing about it? No setup is required to make AirPlay work. Devices simply find each other as long as they’re on the same network.


AirPlay works great if you use other Apple products and services, but what if you’re like most consumers, with a patchwork of different devices and platforms? “Homes are complicated places,” said Netflix (s NFLX) Partner Devices Director Scott Mirer. The reality is that your phone may be an Android phone, and your TV may be manufactured by Samsung.

That’s why Netflix has banded together with YouTube and others in the industry to establish an open standard dubbed DIAL, which is shorthand for discovery and launch. The idea: An app on your phone or tablet will automatically discover any DIAL-capable TV in the same network, kind of like iOS devices automatically discover Apple TVs. Once that connection is established, the app on your phone will be able to launch an app or web app on your TV. What the two apps do after that is up to each developer.

Bringing competitors together

Talk to YouTube Product Manager Sarah Ali, and you get the sense that DIAL may have far-reaching applications. “We think this is really powerful,” Ali said. With this technology, users of YouTube’s mobile app can automatically send videos to TV screens without first pairing the devices. The feature first launched on Google TV devices, and has since quietly found its way onto select Blu-ray players and TV sets from manufacturers like LG, Panasonic and Sony. “We want this experience everywhere,” Ali said.

Mirer echoed that sentiment, which also explains why the two competing video services cooperated on the product.

DIAL powers YouTube's multiscreen solution, which makes it possible to send videos straight from the phone to the TV.
New industry standard DIAL makes it possible for YouTube to send videos straight from the phone to the TV.

Indeed, DIAL seems to have struck a nerve: There hasn’t been much talk about the technology, but numerous companies have signed on to participate. An online registry includes entries from the BBC, Hulu, Pandora, (s P) Disney, (S DIS) Turner and others. On the hardware side, there’s participation from Intel, (S INTC) LG, Samsung and Humax.

Netflix also uses DIAL for remote playback, allowing users to select a movie on their tablet and then start playback on the TV screen. The company eventually wants to offer additional second-screen functionality, and Mirer told me that he is “excited by the potential” — but he cautioned that it may take a long time before some of the more advanced functionality makes its way into every living room. “What gets lost a lot in the public discourse is how long this game is,” he said.

Chromecast is Google’s Nexus approach, taken to the living room

That lesson wasn’t lost on Google, which quickly realized that not every publisher had the same resources to develop its own remote playback and mirroring applications on top of DIAL. “It wasn’t enough,” Google product manager Rishi Chandra told me. That’s why Google developed Google Cast, which lets users launch and play content on the TV straight from their mobile devices, on top of DIAL. Google Cast comes with what Chandra called “a much more robust set of APIs,” which should help developers to integrate the technology into their own apps.

Chromecast doesn't have a remote control or an app store - which may make it the most radical smart TV solution yet.
Chromecast doesn’t have a remote control or an app store, which may make it the most radical smart TV solution yet.

And to prove the point that consumers are interested in this, Google launched a new product called Chromecast at the end of July. Chromecast is a $35 dongle that plugs straight into a TV’s HDMI port and that’s capable of playing back content that’s launched on a tablet or mobile device.

It comes without a remote control, and is likely the most minimal smart TV solution so far. Instead of burdening users with yet another app store on TV, or forcing them to navigate through menu items with a remote control, it outsources the entire content discovery and control part to the mobile device, and turns the TV into a canvas that simply plays back the selected content. And unlike AirPlay, which works only with iOS devices, Chromecast works with devices outside the Google ecosystem, too.

Chromecast also reflects the lessons Google learned from its mobile operating system Android, said Chandra: “The best way to define the right user experience is to build it end to end.” The device is “very similar to the Nexus model that you see with Android,” he added.

That approach seems to be working. Chromecast immediately sold out online and in Best Buy retail stores. Google even had to end a Netflix promotion about 24 hours after launching the device due to what the company said was overwhelming demand.

DIAL and Google Cast could make TVs not just smart, but social and fun

Google has already said that it wants to make Google Cast available to other TV manufacturers as well, but Chandra told me that he doesn’t expect devices from other manufacturers to support the technology until next year. He also emphasized that Google is working to expand the functionality, and bring more publishers on board.

What kinds of cool features and services might smart TV consumers get down the road? One hint is in YouTube’s current DIAL-based second-screen implementation, which is also available on Chromecast. YouTube’s mobile apps allow users to build collaborative playlists on the spot, with everyone in the room adding videos to the queue. That basically allows users to band together for instant YouTube parties — and gives the overused buzz phrase “social TV” a whole new meaning.

Ready to cast: Will DIAL and Google Cast enable the future of social TV?
Ready to cast: Will DIAL and Google Cast enable the future of social TV?

Smart TVs and mobile devices, working together, could in the future enable a whole range of new social experiences, with family members competing against each other while Who wants to be a millionaire is on TV, dorm-room buddies acting out real-life versions of, and people singing into their smart phones to participate in impromptu karaoke sessions while American Idol is playing in the background.

DIAL, Google Cast and other emerging technologies could be key to enabling these kinds of apps– and helping smart TVs finally to become fun TVs.