Big data – Big deal? What are the trends in big data analysis? Is Big data a key tool for increasing competitiveness and productivity?

comments 0






Kishore kothandaraman's picture

Burgeoning amounts of data are being churned out everyday to the extent that current database technologies are rapidly becoming obsolete. Digitalisation has accelerated the growth in data across every organisation, sector and economy. Companies, government agencies, market researchers and meteorologists are waging a technological battle to comprehend big data and extract some of its value. The ability to store, aggregate and combine data has led to numerous technology firms rushing to build data warehouses to meet rapidly-growing demand.

The first trend I foresee is the rise of the Industrial Internet that will affect the industrial sector dramatically. The next year, machine-to-machine data will grow significantly and continue to do so in the years after. In addition, GE reports that the Industrial Internet could add $10 to $15 trillion to global GDP in the coming years. These sensors will completely change the way companies, factories and supply chains will be operated and managed. Technology is transforming the industrial sector, creating machines that can see, feel, sense and react, so they can be operated far more efficiently. Already there are some great examples of how this will affect companies, ranging from airline companies that can reduce turn-around time with monitoring the plane during the flight, to analyzing many different variables to pick the best places to locate wind turbines around the world to be able to harvest the most energy at the lowest costs. As sensors and storage are becoming cheaper every day, algorithms are becoming better and organisations more and more see the need for smart factories, the Industrial Internet will really soon. Now that the Big Data infrastructure deployment phase is well underway, emphasis will shift to new applications that solve real world problems and deliver specific ROI benefits.

A lot of Big Data startups are creating a Big-Data-as-a-Service solution to help organisations apply Big Data without the heavy costs involved. Especially useful for Small or Medium sized enterprises who do want to develop a data-driven information-centric organisation, but who do not have the capacity to develop and maintain a full-fledge Big Data solution on premises. More experimentation will occur in the Cloud and we will start to see Predictive Analytics “as a service” rather than deployment of larger, expensive analytic platforms. More and more organisations will start to use Big Data techniques to secure their IT infrastructure and prevent from being hacked and have data monitored or stolen. Log data will form an important aspect in this and organisations will start to see the importance in monitoring and analysing their IT infrastructure log data in order to keep their infrastructure and data safe. This will help to restore and keep the trust of their customers. More experimentation will occur in the Cloud and Predictive Analytics will be used as a service rather than deployment of larger, expensive analytic platforms

Consumers are creating massive amounts of data through every click, like, tweet, cell-phone call, purchase and self-tracking applications they use. Companies like Amazon have already used these kinds of data for many years to create a personal online shopping experience with recommendations, personal homepages, personal discounts or personally targeted mass-email campaigns. However, in coming years, more organisations will also start to see the value in such a personalized approach, be it online or offline. A good example is the Australian shoe retailer Shoes of Prey, who have developed an analytics system that enables them to look at individual customer-spend and profitability, and allows them to begin upselling based on the fashion tastes of its clients. Consumers will start to see that their data is valuable and they do want something in return for providing their data. So consumers are willing to cooperate and share their data if it brings them personalized discounts.

Companies can leverage data to design products that better match customer needs. Data can even be leveraged to improve products as they are used. An example is a mobile phone that has learned its owner’s habits and preferences, that holds applications and data tailored to that particular user’s needs, and that will therefore be more valuable than a new device that is not customized to a user’s needs. Capturing this potential requires innovation in operations and processes. Examples include augmenting decision making—from clinical practice to tax audits—with algorithms as well as making innovations in products and services, such as accelerating the development of new drugs by using advanced analytics and creating new, proactive after-sales maintenance service for automobiles through the use of networked sensors. A famous innovation has been to leverage the data collected by loyalty card programs. UK supermarket chains such as Tesco and Sainsbury’s regularly use the past purchase histories on loyalty cards to tailor promotions to individual shoppers in the form of vouchers. Other retailers, such as those in the fashion industry, are also making inroads into big data. Inditex for instance, which owns retail-chain Zara, collects data from its till receipts to identify demand for certain garments. It controls most of it supply chain and employs a just-in-time (JIT) production strategy, which prevents unwanted build-up of inventory. It also means that if it identifies an emerging trend, it takes just weeks to develop a product and get it onto the shop floor, while six months is considered the industry average. A next-generation retailer will be able to track the behaviour of individual customers from Internet click streams, update their preferences, and model their likely behaviour in real time. They will then be able to recognize when customers are nearing a purchase decision and nudge the transaction to completion by bundling preferred products, offered with reward program benefits. This real-time targeting, which would also leverage data from the retailer’s rewards program, will increase purchases of higher-margin products by its most valuable customers.

Manufacturers have been an early and intensive user of data to help drive quality and efficiency during the production process. However, as data continues to grow exponentially and global competition intensifies, they are under pressure to continually improve performance. Big data can significantly accelerate the rate of product development. It allows designers and manufacturers to share data quickly and cheaply, and create simulations that test different designs. Both the aerospace and car industry use big data for this purpose. Car manufacturers in particular have invested heavily in trying to optimise costs across their supply chain using proprietary systems that monitor the price and quality of each part sourced. ToyotaFiat and Nissan claim to have cut development time by 30-50%. While the payoff is certainly large, huge amounts of money are required to invest in such systems. Not every manufacturer has the budget to build such systems in-house but outsourcing them would bring tremendous value.

Not only can companies exploit big data for commercial value, but it can also help improve the business management process. As mentioned, retailers and manufacturers use big data to optimise their supply chain and inventory levels.  It can also be used to help maximise cash flows and minimise the length of the company’s cash conversion cycle - this is the time span between spending cash during the production process and collecting it from customers and also help improve elements of corporate governance by providing certain risk controls. A common problem is that management decisions can be highly ‘ad hoc’ and at times, ill-informed. Companies such as SAP have developed business tools that keep the senior management well informed with real-time data. Making big data available across an entire business has considerable benefits. It can encourage underperforming divisions, for example, to improve without direct management intervention. A common application is by ranking sales targets by division, or even individual employee. Demand is growing for software that can deliver this variety of statistics and performance indicators.

The bottom line is improved performance, better risk management, and the ability to unearth insights that would otherwise remain hidden. As the price of sensors, communications devices, and analytic software continues to fall, more and more companies will be joining this managerial revolution. If big data can be effectively supported, analysed and exploited, it has the potential to enhance the productivity and competitiveness of companies, industries and, ultimately, entire economies. Retailers and manufactures are already using big data to improve supply chain management and speed up the development of new products. Big data is also being harnessed by the senior management of companies to monitor performance and allow effective real-time decision making. The benefits of big data are not exclusive to the private sector – it has the potential to improve public services as well, particularly healthcare. Policy makers who understand that accelerating productivity within sectors is the key lever for increasing the standard of living in their economies as a whole need to ease the way for organizations to take advantage of big data levers that enhance productivity.


1. GE, Industrial Internet Report,
2. Intel, A vision for Big data,
3. IBM, ‘Bringing big data to the enterprise’, <>.
4. P. Zikopoulousa and C. Eaton, Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data, (edn: McGraw-Hill, 2011)
5. McKinsey Global Institute, Big data: The next frontier for innovation, competition, and productivity (June 2011)

Articles you may like