Business Analytics is on the rise again with Big Data leading the way

It is fun to see how some things will just continue being relevant. Business Analytics, Data Warehousing and lately Big Analytics are topping the charts. Based on my own feelings, Big Data really took off the second half of 2012 and we also included that in our business modeling workshops as one optional extension that software vendors (ISVs) should look at. Harvard Business Review brought Big Data to the forefront in its October 1, 2012 magazine with Andrew McAfee and Erik Brynjolfsson (guru whom I followed when I worked on my PhD) with an article “Big Data: The Management Revolution”. According to the authors, Big Data is far more powerful than analytics of the past, specifically in making predictions.

One of the key reasons for the sudden explosion if Big Data has to do with the urge to achieve competitiveness by getting a better understanding of your customer, its behavior and the only way to do this is to enable massive analysis of data and in the past, this has not been possible with on-premise environments due to scalability issues. With new cloud technology such as Azure Big Data, ISVs and end user organizations can scale up the analytics/calculations based on the need (in bursts) and scale down when the calculation is done. There are quite a few new interesting startups in the Big-data-as-a-service domain (Zoomdata, Bidgely, Ginger.io, AgilOne, Continuuity). I expect this trend to continue specifically as cloud platforms enable startups to innovate without having to invest huge amount of capital in hardware and use the elasticity of the cloud instead.

What I expect to happen during 2013 is that you will hear more about real cases of Big Data use and conferences such as BigData TECHCON appear on your radar screen. Big Data is no longer about if there is technology to do it, it is more about finding the people that understand it and how to utilize it. According to McKinsey & Company, there will be a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the data to make effective decisions”. The McKinsey article breaks down the importance of Big Data very nicely, including things such as dealing with policies around privacy, security, intellectual property and even liability. There is a full report that can be downloaded from McKinsey web-site.

How does all this rely back to software vendors that I work with on a daily basis? If you are an ISV that deal with lots of data, you have to have a game plan for Big Data. Even if you do not care about it, your customers will be asking for it going forward. It is the same what has happened with the Cloud. Three years ago, the question about cloud was almost non-existent in many domains and today an ISV can’t really survive without the cloud. How about that as being a guiding factor for Big Data.

Personally I feel this is very exciting to me as Analytics, Data Warehousing, Business Intelligence has been my core domain for more than 20 years. Even my doctoral dissertation Evaluation of a Product Platform Strategy fro Analytical Application Software from 2004 is still relevant and explains the drivers that a software vendors should be looking at from a software product platform and software product line perspective. The link will download the dissertation (in English) and it is in PDF format.

Expect to hear more about Big Data from me during 2013 as it will be even more relevant than during 2012.