Amazon’s Import/Export Service, which allows folks with large data sets to mail their files to the company’s (s amzn) storage and compute clouds, has moved from beta to general availability. We wrote last year when the service launched about how our broadband infrastructure isn’t robust enough to handle terabytes of information quickly and affordably, which meant that certain AWS clients with large data sets would find the ability to put their data on storage drives and pop them in the mail to Amazon both faster and a better deal than sending it via the web.
A year later Amazon has opened the service up to everyone, and offers some use cases that drive home both the value of cloud computing and the amazing masses of data that are being processed on the cloud. The company’s Open Solutions Group, for example, uses AWS to parallel process more than 40 terabytes of geospatial data, yielding results in hours instead of years. “Within just days our customers have disks and data in hand, and can begin modeling, analyzing, and visualizing the results of our geo-processing work,” Brian Levy, president and CEO of Open Solutions Group, is quoted as saying.
What will really be cool is when customers of Open Solutions no longer bring that data in-house for analysis. At that point we’d have a broadband network robust enough to send such information out to customers, but also a culture at enterprises that finds it acceptable to take the data and perform the modeling within a compute cloud. But first it will have to be easier to move the data from one cloud to another than it is currently (and ideally not involve the U.S. Post Office), and would likely require a special-purpose cloud dedicated to data analytics.
I think providing a faster way to move the data (such as Aspera does), ensuring open standards for the data so that it’s portable, and possibly special purpose high-performance computing clouds are emerging business opportunities for startups and established companies. I also think processing big data on compute clouds is leading to very interesting business opportunities, which I outlined in a GigaOM Pro research note a few weeks back (sub req’d). Anyone interested in either of these should really come to our Structure 2010 conference in San Francisco later this month where we’ll be discussing exactly these sorts of opportunities. Or you can talk about it in the comments below.
