Is Cloud Computing and Big Data a “Must Have” Strategic Asset?
Amazon keeps beating their chest about the role that AWS took in the last Presidential Election. Specifically, about the technology that powered President Obama’s successful reelection campaign, according to Jeff Barr, Amazon’s Web Service Evangelist, in a recent blog post.
Jeff states: “The campaign used AWS to avoid an IT investment that would have run into the tens of millions of dollars. Along the way they built and ran more than 200 applications on AWS, scaled to support millions of users. One of these apps, the campaign call tool, supported 7,000 concurrent users and placed over two million calls on the last four days of the campaign.”
The campaign leveraged a database running on Amazon RDS, which was the primary registry of voter file information. The RDS database consumed data from a number of sources, including www.barackobama.com and donor information from the finance team. This provided the campaign managers with the right data visualization to analyze what exactly was going on, and thus how to allocate and re-allocate resources.
Of course, data is one problem to solve, analytics is another. They leveraged an analytics system running on EC2 Cluster Compute instances, which was coupled with the database. “Another database cluster ran Leveldb on a trio of High-Memory Quadruple Extra Large instances.”
The use of the database provided the campaign workers with information required to target voters, including focusing and re-focusing marketing resources. For example, they could understand the effectiveness of certain ads on polling and donations, and step up or step down the ads to reach the desired outcome.
The applications made use of virtually every AWS service, including EC2, Route 53, SQS, DynamoDB, SES, RDS, VPC, EBS Provisioned IOPS, SNS, ElastiCache, Elastic Load Balancing, Auto Scaling, and CloudFront.
No matter if you voted for Red or Blue, you have to be impressed by the quick scaling up of systems, including applications, processes, and data to support the massive needs of the campaign. This included spinning up the instances, operating them during the spikes of use, and spinning them down when not needed. This is a great case study for the power and value of cloud computing.
The larger lesson here is one of business intelligence, and leveraging the right technology. Or, the ability to leverage most of the data you have in hand to understand just what is going on within the business, in this case, the business of campaigning. It’s also the ability to quickly correct processes, or reallocate resources to optimize the business opportunities.
The existing state of data leveraging within most of the Global 2000 is dismal at this time. Most don’t understand where their data exists, let alone what the data means in the larger holistic context of the business. Instead, they have small snapshots of instances of data, such as data warehouses and data marts, that have proven to be limiting and cost prohibitive to implement.
So, the movement to cloud computing, and further on to Big Data systems, solves a few problems:
First, it’s finally cost efficient. As in the case study presented by Amazon, you’re spinning up compute and database resources as you need them, only paying for what you need. This is vastly less expensive than paying for whole new sections of the data center to support the management of massive amounts of data.
Second, the ability to see current data. In many instances, data warehouses provided access to old aggregated data. While it could answer a few strategy questions, it was typically worthless for operational applications. Thus, those running the business did not have the insights required to solve emerging problems, or the ability to take advantage of emerging opportunities, as they did in the Amazon case study.
Finally, there is the ability to make quick work of analyzing current data. Just because I’m storing a few terabytes of data does not mean I can make any sense of it in a timely manner. Cloud computing, along with new approaches to database processing (such as map reduce), allows you to spin up as many instances of resources as you need to take a divide and conquer approach to culling through millions or billions of records. What used to take 2 days, now takes 2 minutes.
Moving forward there will be two kinds of companies. There will be companies that understand how to take advantage of the data they have, and make critical adjustments to the business with near perfect information. The other kind will not have good visibility into their current business data and have to react to perceptions or data fragments. Who do you think will be in business in 10 years?