Increased awareness of the power of big data and related analytic capabilities to gain competitive advantage and to improve operational efficiencies has made big data a practical reality.

The explosion of data volume, coupled with the emergence of unstructured and semi-structured data sources, is changing the landscape of business intelligence. The number of devices capable of generating data streams - smartphones, embedded devices, network equipment, and human facing applications - is blowing up. Organizations have increasingly been looking for solutions to monetize 100 percent of their data.

The digital universe is estimated to produce 35 ZB of data by 2020, with 80 percent of the growth coming from unstructured data such as audio, video, and email; machine-generated data from a multitude of sensors; and data from external sources such as the Internet and social media. To retrieve real business value from the humungous amount of data, enterprises need the right tools to capture, organize, process, and analyze the data meaningfully.

Big data analytics breaks down huge amounts of data to uncover hidden patterns, unknown correlations, and other useful information, which can provide organizations with competitive advantages and result in business benefits such as more effective marketing and increased revenues. Until the advent of big data analysis techniques, unstructured and semi-structured information was not considered to be a part of mainstream data analysis, which used to be limited to structured data.

Market Opportunities

The big data market is on the verge of a rapid growth spurt, and is expected to achieve USD 50 billion globally within the next five years, according to Wikibon. It is estimated at USD 5 billion as of early 2012 based on related software, hardware, and services revenue.

Pure-play vendors accounted for USD 480 million in revenue, and despite their relatively small percentage of current overall revenue, vendors such as Vertica, Splunk, and Cloudera are responsible for the vast majority of new innovations and modern approaches to data management and analytics. The major players in the enterprise industry, namely IBM, Intel, HP, Oracle, Teradata, Fujitsu, Verint, Amdocs, Datamatics among others, account for 95 percent of the overall revenue.

Many new pure-plays are expected to enter this flourishing market. Having made significant acquisitions of independent pure-plays in 2011, the big vendors are expected to continue its M&A growth strategy until the market matures and settles down. Each and every big data pure-play is supposed to be potential acquisition target of megavendors IBM, Oracle, HP, and EMC, among others. The market is forecasted to experience significant consolidation within the next three-to-five years. The acquiring vendors are expected to allow pure-plays to continue operating and innovating as largely independent entities, or risk stifling the very innovation that is fueling the big data market's tremendous growth.

For organizations that understand and embrace the new reality of big data, the possibilities for new innovation, improved agility, and increased profitability are nearly endless. Increased interest in and awareness of the power of big data and related analytic capabilities to gain competitive advantage and to improve operational efficiencies, coupled with developments in the technologies and services that make big data a practical reality.

Market Innovations

Hadoop distributions. Cloudera and Hortonworks are responsible for majority of contributions to the Apache Hadoop project that is significantly improving the open source Big Data framework's performance capabilities and enterprise-readiness.

Next generation data warehousing. The three leading next generation data warehouse vendors - Vertica (acquired by HP), Greenplum (acquired by EMC), and Aster Data (acquired by Teradata) are upending the traditional enterprise data warehouse market with massively parallel, columnar analytic databases that deliver lightening fast data loading and near real-time query capabilities.

Big data analytic platforms and applications. Vendors are developing applications and platforms that leverage the underlying Hadoop infrastructure to provide both data scientists and business users with easy-to-use tools for experimenting with big data.

Big-data-as-a-service. It is developing rapidly on account of vendors such as Tresata, 1010data, and ClickFox. Cloud-based big data applications and services have the potential to allow small- and mid-sized business, as well as enterprises that lack internal expertise, to take advantage of big data processing and analytic capabilities without needing to deploy and manage on-premise hardware or software.

Non-hadoop big data platforms. Other non-Hadoop vendors contributing significant innovation to the big data landscape include Splunk, HPCC Systems, and DataStax.

Driving IT Spending

Enterprises are anticipated to begin using their big data experience in an embedded form in their architectures and practices by 2015. The benefits to organizations for adding big data to their information management and analytics infrastructure is expected to force a more rapid cycle of replacing existing solutions.

Big data globally is forecast to drive USD 34 billion of worldwide IT spending in 2013 from USD 28 billion in 2012, according to Gartner. It was pointed out that in 2012, only USD 4.3 billion in software sales will be driven directly by demands for new big data functionality, with most of the current spending being used in adapting traditional solutions to the big data demands -- machine data, social data, widely varied data, and unpredictable velocity.

Enterprises will develop a more thorough and insightful understanding of their business after big data is distilled and analyzed in combination with traditional enterprise data, leading to enhanced productivity, a stronger competitive position, and greater innovation - all of which will have a significant impact on the bottom line. Owing to its pervasive effects, big data will evolve to become a standardized requirement in leading information architectural practices, forcing older practices and technology into early obsolescence.

There is a strong latent demand for big data and analytics platforms in the Indian market. Some of the sectors that are likely to benefit immediately from big data adoption include computer and electronic products, financial and insurance, and government. In addition, healthcare and telecom sectors are also likely to benefit a lot from big data.

"For instance, in healthcare services delivery, management of chronic or long-term conditions is expensive. Use of in-home monitoring devices to measure vital signs and monitor progress is just one way that sensor data can be used to improve patient health and reduce both office visits and hospital admittance.

Manufacturing companies deploy sensors in their products to return a stream of telemetry. Sometimes this is used to deliver services like OnStar that delivers communications, security, and navigation services. Perhaps more importantly, this telemetry also reveals usage patterns, failure rates, and other opportunities for product improvement that can reduce development and assembly costs.

The proliferation of smart phones and other GPS devices offers advertisers an opportunity to target consumers when they are in close proximity to a store, a coffee shop, or a restaurant. This opens up a new revenue for service providers, offering a great chance to target new customers to a number of businesses.

Social media sites like Facebook and LinkedIn simply would not exist without big data. Their business models require a personalized experience on the web, which can only be delivered by capturing and using all the available data about a user or a member."

Mitesh Agarwal
Director-Solution Consulting,
Systems Business, Oracle India