Amazon (s amzn) today said it would now also offer high-performance computing (HPC) through Cluster Compute Instances, which use more powerful processors and clustered nodes to help get around some of the latency issues associated with distributed computing. HPC has long taken advantage of many nodes working in tandem, but has typically connected those processors using Infiniband or low-latency networks. However, as the need for HPC becomes more popular, and the next generation of faster supercomputers grows more difficult to build (GigaOM Pro sub req’d), the cloud may offer some hope.
The rise of cloud computing owes a bit to HPC, but for the most part demanding workloads couldn’t take advantage of the cost benefits and flexibility associated with the cloud because of the latency within the cloud and the cost and time associated with moving terabytes of data back and forth from a lab to the cloud. Amazon has solved the data problem by using the postal service and also by acting as a free repository for some types of scientifically valuable information. From the Amazon release:
Cluster Compute Instances can be grouped as cluster using a “cluster placement group” to indicate that these are instances that require low-latency, high bandwidth communication. When instances are placed in a cluster they have access to low latency, non-blocking 10 Gbps networking when communicating [with] the other instances in the cluster. Next, Cluster Compute Instances are specified down to the processor type so developers can squeeze optimal performance out of them using compiler architecture-specific optimizations. At launch Cluster Computer Instances for Amazon EC2 will have 2 Intel Xeon X5570 (also known as quad core i7 or Nehalem) processors.
Since enterprises are also turning to HPC for certain jobs, from analytics to making movies, it seems that Amazon may also help draw those jobs into the cloud. Of course, there are companies out there providing access to an actual supercomputer in an on-demand fashion, but Amazon’s decision could shake up the world of HPC. If Amazon succeeds, it proves the value of offering different hardware platforms for different workloads in a compute cloud. Perhaps we’ll see other heterogeneous cloud architectures follow suit.
