It could be my advancing years – I seriously doubt that – but every morning I dread turning on my MacBook Pro, fearful of the data deluge that it will bring and the day-long struggle to find the information I need to get things done that will ensue. And I’m not the only one caught in this quicksand-like avalanche of digital data.
According to market research firm comScore, in May the total number of Internet searches conducted in the U.S. alone was about 10.7 billion — up nearly 20 percent from 9.1 billion searches in May 2007. Those numbers make clear that we’re all searching for more information. What they don’t make clear is that often we don’t find what we’re looking for, and so end up trying again and again.
The problem is that there’s too much data coming online too quickly, and the traditional method of search that involves first finding and then consuming the information is not going to work for much longer. There just won’t be enough time for us to do that and still have a life. It’s a problem, and therefore solving it is an opportunity — a very big opportunity.
Earlier this week I was going through the digital detritus that has accumulated on my computer when I stumbled upon an old slide presentation made by Google back when they were still tiny. One slide estimated that by 2002, there would be about 500 million searches a day and between 3 billion and 8 billion web pages. Now those numbers seem so last century, for every day the amount of information online continues to grow at an exponential rate. It’s nearly impossible to calculate the exact number of web pages that are out there, but a good yardstick would be data from Netcraft, which tracks the number of servers on the Internet and says that the number of active domains almost quadrupled from 2002 to 2007. The total number of web sites at the end of April stood at over 162 million.
Many of these new sites are courtesy of Web 2.0 technologies that have allowed for the easy creation of digital data. Blogs, social networks, RSS feeds, Flickr feeds, Twitter messages, video clips…the data just keeps growing and growing, much like the proverbial Energizer bunny. And the problem of data overload is going to get even bigger as devices such as the 3G iPhone, with their fast wireless connections, make the on-the-go creation and sending of videos, messages and photos to our friends even easier.
If someone can become the Dolby of the web — remove the noise and give us clear sound — then they are going to make a lot of money. And when I say sound, I mean data that is truly useful. But that would just be the start.
Pip Coburn, who runs his own investment firm in New York, recently pointed out that “It’s not data that’s important, but what you do with it.” A good example of that would be a tiny startup called Summize, which is reportedly being acquired by Twitter.
Summize has come up with a clever way of peering through Twitter’s vast data stream and finding out what’s hot, where and how. The results are essentially keywords — topic-, person- or location-based — and thus can be used to show contextual advertising next to the pages that show these results. In other words, Summize has developed an ability to monetize conversations without being intrusive.
The possibilities of what a similar service could do with this data are endless. Imagine a service that would scroll through all the Flickr photos, Twitter messages and marry them to data on the Internet, such as nearby mass transit stations, Starbucks, movie theaters and grocery stories.
All this information would show up on your phone, but you would only see the options in, say, a 100-meter radius that could be increased by zooming out. It would be the ultimate mash-up of various web data sources offered to you as an application, and such applications would make it possible to find, consume and share information — without even trying.
Almost like serendipity! How’s that for a business model?
This post was originally published on BusinessWeek.com.
