Innovation in America: Wrapping Our Heads Around Big, Big Data

This article is part of our Innovation in America series, in which Foolish writers highlight examples of innovation going on today and what they see coming in the future.

Data are all around you. Every time you swipe your credit card at the store, you create a transaction with lots of information about the buyer, the seller, the money transfer, items on your shopping list, and much more. Enormous store chains treat this information as a treasure trove of business fuel, to the point where your local megamart might know that your daughter is pregnant before you figure it out yourself.

That's a pretty disturbing example of the power inherent in free-flowing information, but a relevant one nonetheless. The point is, everything you do in a modern society is worth something to somebody. And if you really want your privacy back, be prepared to make some drastic lifestyle changes:

Cut up all your credit cards.

Insist on getting your paycheck in cash.

That smartphone is dripping with personal information and intimate tracking powers -- throw it away!

Actually, regular cellphones still give away your location every time they connect to a wireless tower. Just go off the grid entirely.

And just in case someone might take a picture and tag you on Facebook (NAS: FB) or Twitter, don't talk to anyone -- ever.

Since most of us don't subscribe to the lifestyle of an extreme hermit, businesses have plenty of data to work with. In fact, they have so much information that it takes very specialized software and systems to make sense of it all. And given the potential value of acting on hidden data patterns, these tools have formed an industry all its own. Welcome to the world of big data and data mining.

$300 billion a year?
Don't just take my word for how enormous an opportunity the data-crunching problem can be. IBM (NYS: IBM) says that 90% of all the data points ever created in the history of the world come from the last two years -- and the pace of new information flows is increasing.

Naturally, Big Blue is working hard to tame this torrential data flow. The Watson computer that destroyed two Jeopardy all-stars without consulting the Internet may seem like a publicity stunt, but actually plays right into the problem of extracting usable conclusions from huge volumes of disorganized data. Lessons learned here will eventually turn into analysis tools that IBM can sell to its customers.

World-renowned consulting firm McKinsey has been sounding the depths of the big-data market, too. In a recent study, the firm said that the U.S. health care sector alone could realize $300 billion in quality and efficiency improvements by applying rigorous analysis to health information. The gains could go even further -- that figure doesn't include "using big data to reduce fraud and errors and boost the collection of tax revenues," for example. And retailers who use these tools to their full potential could boost their operating margins by more than 60%, McKinsey says.

And let's get back to Facebook for a second. The world's biggest social network struggles to make money today, but its users provide a veritable gold mine of highly marketable information. If Facebook can figure out a way to monetize those billions and billions of likes, status updates, and network connections without starting a firestorm of privacy protests, this business could have serious legs. Unfortunately, management appears tin-eared on privacy issues so far. Keep an eye on Facebook's data handling to see if that ever changes.

Cast and crew
IBM is far from the only maker of big data analysis tools. In fact, it's kind of a bit player despite its massive resources.

Some of the more specialized players are already becoming household names. For example, when Wal-Mart (NYS: WMT) needed to organize the world's largest data store two decades ago, Teradata (NYS: TDC) was the only data warehousing specialist that could do the job. The retailer switched to a different solution in 2007 but came back to Teradata again only three years later -- realizing that a penny saved isn't always a penny earned if the cheaper alternative can't do the job.

Then there's TIBCO Software (NAS: TIBX) , which promises to mold usable information out of raw data faster than anyone else. The "2-second advantage" the company promises over slower database-centric solutions can be the difference between a sale and a missed opportunity, or between turning customers into critics or promoters.

The list goes on, and the up-and-coming experts are as varied and colorful as the data they analyze. They do have one thing in common, though: IBM, Tibco, Teradata, and nearly all of these modern-day innovators are as American as baseball and Kansas.

The future is made in America, and we've prepared a special report just for you on three domestic companies whose mind-bending technologies may overturn an entire industry. Click here to read all about it in a special report, free to Fools for a limited time.

The article Innovation in America: Wrapping Our Heads Around Big, Big Data originally appeared on Fool.com.

Fool contributor AndersBylund holds no position in any of the companies mentioned. Check out Anders'holdingsandbio, or follow him on Twitter and Google+. The Motley Fool owns shares of IBM and Facebook. Motley Fool newsletter services have recommended buying shares of TIBCO Software and Teradata. Motley Fool newsletter services have recommended creating a diagonal call position in Wal-Mart. Motley Fool newsletter services have recommended creating a synthetic long position in IBM. The Motley Fool has a disclosurepolicy. We Fools may not all hold the same opinion, but we all believe that considering a diverse range of insights makes us better investors. Try any of our Foolish newsletter services freefor30days.

Copyright © 1995 - 2012 The Motley Fool, LLC. All rights reserved. The Motley Fool has a disclosure policy.