Hortonworks makes a Hadoop security play and buys XA Secure

Hadoop vendor Hortonworks has acquired XA Secure, a Fremont, California-based startup working on an across-the-board security offering for Hadoop.

Hortonworks VP of product management Tim Hall said some sort of Hadoop-wide security project is necessary now that YARN has caught on and more users are accessing more data via various tools on the same cluster (MapReduce, Spark, Hive and SQL engines, for example). As it stands, many Hadoop projects incorporate their own security protocols but there isn’t a single technology for enforcing security policies and practices across the whole environment. There are also a slew of third-party tools available for addressing various aspects of Hadoop security.

If executives are going to fully embrace Hadoop, Hall added, they’ll demand a security framework that provides a single dashboard that lets them monitor and govern the whole environment. It will have to comport with existing company-wide policies around who can access what, what they can do with it and how that information is audited. XA Secure’s technology does that, which is why he said Hortonworks bought the company.

The goal is to integrate the XA Secure technology into the Hortonworks Hadoop distribution and to release the technology as an Apache project during the second half of this year. Jim Walker, a product manager at Hortonworks, said it will probably be its own separate project, although it will have to have hooks into the gamut of various Hadoop projects with which the technology will work.

The current Hortonworks architecture, complete with security and governance tools.
The current Hortonworks architecture, complete with security and governance tools.

Hall added that Hortonworks will likely build a software development kit for the XA Secure technology and generally try to open it up to partners such as Red Hat, Microsoft and Teradata, whose technologies integrate tightly with the Hortonworks platform.

Terms of the deal are undisclosed, although it probably wasn’t a huge investment for Hortonworks. XA Secure is a relatively new company, having just launched in January 2013, with much of its development team based in India. The company has at least one customer in telecommunications provider Sprint.

Having a security story as deep as XA Secure is certainly good for Hortonworks and its ecosystem, but one has to wonder how broadly adopted the open source project based on XA Secure will be. The company was already technology partners with Cloudera and MapR, as well, but as Hadoop has become a more competitive and lucrative market, the vendors selling in it have become less willing to embrace technologies backed by others and more likely to build parallel ones. Lately, that goes for security as well as most other things.

It’s possible everyone could rally around a security project like they did around various components early on (for example, Hive as a data warehouse framework), but it’s also possible there’s too much money at stake and too much technological divergence at the other layers to make that a realistic option.