Most are declaring victory for universal big data projects, or those that allow for storing, analyzing and extracting insights from big data. In my opinion, however, that story is pretty much played out – to be replaced by big data as applied to verticals.
The recent news item that got people talking was Cloudera receiving US$ 740 million from Intel. A little over five years old, the company has raised more than one billion dollars. By any measure, this is staggering. The amount of money and excitement funneled towards universal big data is incredible. These projects quite literally provide the capacity for big data analysis. Put another way, if these projects did not exist, there is no way anyone but the largest of corporations could fathom attempting big data analysis.
But from an investment stand-point, I cannot imagine there is much room left there (other than pre-IPO financing, which is a completely different animal). The next stage for this sector is IPOs. Cloudera will most likely have one in the next year or two. And, if Cloudera has one, then Hortonworks and a plethora of other similar companies will either push forward with IPOs or accept buy-out offers. Additionally, I would not be surprised in the least if we see AWS spun-off from Amazon and floated as well.
The point is that these universal plays on big data have matured and the end-game is now visible. There will likely not be many new companies attacking this space. The companies that have made it this far will likely have successful exists.
From an investor’s point of view, the risk has diminished greatly, and so have the returns. The time horizon has also significantly decreased with liquidation events coming in the next few years.
From an operational point of view, these universal projects have done an incredible job. They built the platforms that enable big data analysis. They opened the door for other entrepreneurs to leverage their sector specific knowledge on top of these big data universal projects.
Vertical big data plays appear to be the next sweet spot for investment. Universal platforms are great. Maybe they will get you 80% of the way there. But each sector has its own lingo, peculiarities, growth dynamics, stage of development, metrics, etc. Universal platforms, no matter how good, have extreme difficulty attacking these granular sector-specific problems.
The next large growth phase for big data in general will encompass solving those annoyingly sector specific problems that domain experts have grappled with for years. Leveraging new forms of data and analytics these domain experts potentially can begin to solve these issues. General or universal platforms can be used initially but will over time require tailoring to meet the vertical’s needs.
Domain experts partnering with big data specialists and leveraging the already established big data platforms appears to be the next big thing. Instead of “big data” as applied to everything we can look forward to as applied to healthcare, education, government, agriculture, finance, transportation, etc. These cases have already begun to emerge but will generally accelerate in the near future as entire industries get shaken-up.
The big data universal projects (think cloudera, hortonworks, AWS) generally focused on helping the established players (think Fortune 500 companies) get their toes wet in big data and then leverage this trend. They were not really attacking these companies, their clients or their revenue. This process is and was transformative but not really displacing.
The next phase however has a real chance of displacing already established companies within large sectors and this is where is could get very interesting.