What the world needs now is not “love, sweet love,” but faster ways to analyze large amounts of data without having to tap a database that will slow things down. The solution is increasingly in-memory databases where the data is stored right next to the processor doing the computing.
Many of the current offerings for in-memory databases that can offer lightning-fast analytics are housed in expensive appliances such as SAP’s (s sap) HANA or Oracle’s (s orcl) TimesTen product. But as of Wednesday, people with can download Druid, an open-source in-memory database offered by Metamarkets (see disclosure) for free.
Metamarkets provides a data analytics service that offers real-time analytics, visualizations and other products. Metamarkets Co-Founder and CEO Mike Driscoll said it made sense to open source Druid, because the world needs an open source in-memory database. He said if Metamarkets didn’t offer Druid, someone would build a different option and Metamarkets might eventually have to support it. And since Metamarkets wants to deliver a service (or several services), as opposed to the raw technology, open sourcing Druid doesn’t cut into the firm’s ideal customer base.
The rise of in-memory databases has been documented elsewhere as the need for faster analytics and improvements in hardware have goosed both the demand for faster databases and the ability to deliver them as an appliance or as Metamarkets does — on cores running in the cloud. At the moment Metamarkets runs Druid on an 800-core system running on Amazon (s amzn) EC2. Others have done a decent job explaining what Druid seems good for and where the tradeoffs might be.
At a high level the availability of a powerful, real-time analytics capable in-memory database that doesn’t come with licensing fees and can run on the cloud’s commodity hardware, should help tech-savvy startups handle the influx of online data coming at them. Not everyone has giant chunks of data stored someplace ready for a Hadoop batch-process. Some firms catch their data on the fly and want to use it for decions made in real-time. After all, who doesn’t like things fast?
Disclosure: Metamarkets is a portfolio company of True Ventures, which is also an investor in GigaOM. Om Malik is also a venture partner at True.
