HStreaming ready to show the world its real-time Hadoop

San Francisco-based startup HStreaming has accepted its first venture funding — $1 million from Atlas Venture — and is ready to spread the word about its real-time Hadoop system. The three-person company has actually been around for two years, but CEO Jana Uhlig told me during a phone call that interest is just too high to keep the company self-funded.

It’s not surprising HStreaming would be drowning in interest. Ask anyone how they’d like to see Hadoop evolve beyond its current status as a batch-processing platform, and you’ll likely hear “real-time” as one of the answers. In fact, this is a topic we’ll discussing a lot at Structure: Data next month with companies trying to turn Hadoop into operational databases and various types of OLAP engines, and those companies just generally struggling with unceasing streams of machine data.

Presently, companies trying to incorporate a real-time component into their Hadoop environments, in order to process data as it streams into the system and before it hits the disk, are bolting on open source technologies such as Storm and Kafka. While these certainly aren’t toy technology, Uhlig thinks the open source versions are rudimentary (Storm, for example, can pretty much just classify each piece of data as it hits) and notes that they’re not part of a full-on analytic system.

HStreaming, on the other hand, has built a complete system that incorporates its real-time engine for processing streams of video, server, sensor and other machine-generated data, but also is wholly compatible with Hadoop as an archiving and batch-processing system. It also plugs into a wide variety of existing BI tools for analytics, Uhlig said.


Better yet, for Hadoop users, nothing has to change. HStreaming can do its stream processing by re-using the same MapReduce algorithms and Pig scripts that customers have already written. In practice, Uhlig said, users can move from a batch-only system into a real-time system in just days.

She said major interest thus far has come from governments (especially around video analysis, where HStreaming can stitch together images from thousands of cameras in real-time), telcos and advertising. The company plans to step up its focus on utilities and financial services, too.

Former Vertica (s hpq) CEO and Atlas Venture partner Chris Lynch said telcos should be particularly excited about technologies like HStreaming for the purpose of network arbitrage. Without stream processing and some semi-complex algorithms, he explained, it’s impossible to tell what traffic belongs to which users so network providers can charge other carriers or ensure that premium customers don’t experience degraded service while others are doing fine.

“We tried this unsuccessfully at Vertica,” Lynch said. “… We weren’t fast enough.”

HStreaming certainly appears to have the technical chops to do what it promises. CEO Uhlig is joined by CTO Volkmar Uhlig (Jana’s husband) and Chief Software Architect Jan Stoess, both of whom hold Ph.Ds in computer science. Volkmar was lead architect on the L4 microkernel, has built high-frequency trading systems and spent five years a IBM’s (s ibm) TJ Watson Research Center working on stream-processing technologies.

And while one expects investors to be supportive of their portfolio companies, Lynch is effusive in his praise for what HStreaming is doing. “The technology has no practical limitations, but we have to market the company,” he said. “… Watch what happens in the next six months. … I sold 800 customers at Vertica before I sold the company [to HP], and every one of them is going to want HStreaming.”

Feature image courtesy of Shutterstock user MrJafari.