Aa2x9N8Z

EMA Research Report:

Big Data Impacts on IT Infrastructure and Management

Executive Summary

Big data analytics enjoys a great deal of well-earned hype today. The application of advanced analytics

to structured, partially structured, and unstructured data can unearth important insights that were

previously impossible to find. If used correctly, big data can transform a business in countless ways.

This research study by Enterprise Management Associates seeks to understand how big data impacts

and transforms IT infrastructure. Not only does this report examine how the collection and analysis

of vast amounts of data affects IT infrastructure and IT management practices. It also reveals how IT

organizations transform themselves when they start to export IT infrastructure monitoring data to big

data environments for advanced analytics. This report shows how big data analytics for IT enhances IT

planning, monitoring, and troubleshooting practices and also reveals the types of IT monitoring data

that organizations leverage for those purposes. Serving as a guide to enterprises that want to apply big

data technology to their IT organizations, this report offers a roadmap not just for the optimization of

IT infrastructure and operations, but also for the improved alignment of IT and business.

Introduction

Enterprise Management Associates (EMA) analysts have closely followed the emergence and evolution of

big data collection and analytics. Enterprises use big data analytics for a wide variety of business initiatives

that are well known to the public. But EMA has particularly tracked how the use of big data has become

more relevant to IT planning and operations. For instance, the April 2014 study, “Managing Networks

in the Age of Cloud, SDN, and Big Data: Network Management Megatrends 2014,” revealed that

60% of enterprises were using big data analytics to support one or more of their network management

practices. The 2014 study also found that 40% of enterprises had experienced measureable impacts on

the behavior of their network infrastructure due to the presence of big data projects.

These early findings around big data raised several questions at EMA. First, given that many enterprises

were moving toward a hybrid, cross-domain model for infrastructure planning and operations, we

wanted to understand how the presence of big data collection and analytics affected the behavior of IT

infrastructure across all technology domains, including security, storage, compute, and the network.

And if enterprises were indeed collecting and analyzing IT infrastructure data in big data environments

to support infrastructure planning and operations, what kinds of data were they collecting for that

analysis? Finally, we wanted to understand how big data analytics for hybrid IT affected infrastructure

planning and operations practices.

This EMA research study addresses all of these questions and more.

Demographics

This study focuses on how enterprises are applying big data analytics to IT infrastructure planning and

operations. To achieve that goal, we first limited our participant base to enterprises that had at least

some big data activity within their organizations, whether they were currently exploring the technology

or had actually adopted it. Seventy-six percent of the participants in this research had adopted big data

collection and analytics, including 32% who considered big data to be an “important” part of their

business, and another 35% who considered big data to be “essential” to their business. The rest of the

participants were either researching big data or planning an implementation.

EMA also limited the participant base to enterprises that were using, or considering the use of, big

data technology to collect and/or analyze IT infrastructure monitoring data. In this study, 71% of

17%

Beginner

Intermediate

Advanced

15%

17%

16% 18%

EMA Research Report:

Big Data Impacts on IT Infrastructure and Management

participants were applying big data to IT infrastructure monitoring data. The remaining 29% either

were considering applying big data to IT infrastructure monitoring data or had plans to within the next

12 months.

To gain an understanding of how big data affects both IT infrastructure and IT planning and operations

practices, this study necessarily focused on IT personnel, with 69% of participants identifying themselves

as working in information technology, information services, or networking. The pool of respondents

featured a broad split between executives (43%) and staff (57%), providing a contrast in perspectives

between IT executives and the people who report to them

The participant base showed a broad mix of company size. Twenty-seven percent of respondents worked

at small companies (1–999 employees); 35% worked at midsized firms (1000–4999); and 38% worked

at large enterprises (5000 and up). Two vertical industries also had statistically significant representation

in the survey sample: Manufacturers (excluding makers of computer hardware) comprised 15% of

the research pool, and finance, banking, and insurance firms comprised 16% of the pool. This study

explores how these two industries vary from the rest of market in the way that big data impacts their

infrastructure and management practices.

Big Data Maturity Perspectives

As organizations gain more experience with technology and expand their use of it, the impacts that

the technology has on the organization will obviously change. In this study, participants identified the

number of big data projects—or applications—that their enterprises had in production. From these

responses, EMA was able to deduce the level of experience participants had with big data and apply

those deductions to the analysis of the survey results. For the purpose of this study, EMA classified

participants who reported having one or two big data projects in production as “beginners” (29% of

respondents). “Intermediates” were identified as enterprises with three to five projects in production

(38%). And those firms with six or more big data projects in production were identified as “advanced”

(33%). This report will reveal that the advanced firms, in particular, tend to be more innovative with

big data analytics for IT, both in terms of the infrastructure monitoring data they used and in terms

of the use cases to which they applied the technology. However, advanced firms also experience more

disruptions to their IT infrastructure and IT operations practices from big data, most likely because

they are simply collecting and analyzing more data.

2 13%

3 12%

4 11%

6 to 7

8 to 9 8%

10 or more 8%

0% 2% 4% 6% 8% 10% 12% 14%

Figure 1: How many big data projects are in production within your overall organization?

30% 40% 50% 60% 70% 80%

15%

11%

10% 20%

51%

43%

47%

51%

39%

58%

73%

61%

66%

69%

32%

30%

EMA Research Report:

Big Data Impacts on IT Infrastructure and Management

Setting the Scene: Big Data in the General Enterprise

Before digging into the impacts of big data on IT infrastructure and IT planning and operations,

EMA wanted to understand how big data was being used in the broader enterprise. Specifically, we

examined which organizational groups within an enterprise were likely to be the primary sponsors of

big data projects and which groups were the primary users of big data analytics outputs. Given that

the participants of this research were all IT professionals—and that the enterprises represented were all

at least moving toward sending IT infrastructure monitoring data into big data environments—this

sample is understandably biased toward IT project sponsors and users.

EMA first asked participants to identify the chief business goals that were driving the use of big data on

a company-wide basis. Participants could select multiple answers. Only two business goals were selected

by a majority of respondents. “IT infrastructure planning and operations” was the most prominent

goal cited (65%), followed by “security analytics/cyber security/intrusion detection” (57%). Other

prominent goals included “path analysis/customer churn/customer relationship management” (46%),

“marketing/sales campaign analysis” (45%), and “point of sale/customer care” (44%).

Advanced big data users are driven by a wider variety of business goals, as seen in Figure 1. The chart

reveals that as enterprises gain more experience with big data technologies, big data projects serve a

broader range of business goals. For instance, 51% of “advanced” organizations identify staff scheduling

and logistical asset planning g as a driver for using big data, while only 15% of “beginner” organizations

do. A business goal like “geographic optimization and relationship analysis” also goes from being an

obscure use case to one addressed by a majority of advanced users. While advanced users identify

IT-centric business goals like “IT infrastructure planning and operations” as most important to their

big data projects, nearly every other type of goal is also important to a majority of them.

Marketing/sales campaign analysis

Fraud detection

Grouping and relationship analysis: Geographic optimization

Path analysis, Customer churn, customer relationship mgmt

Staff Scheduling, Logistical asset planning

Billing: Rating

Point of sale: Customer care

Security analytics: cyber security: intrusion detection

IT infrastructure planning and ops

Other

Beginner Intermediate Advanced

Figure 2: Which types of business goals are driving the use of big data approaches on a company-wide basis?

39%

42%

53%

28%

36%

59%

24%

25%

32%

55%

33%

47%

57%

Finance 28%

Information Tech/Data Center 76%

Sales 24%

Marketing 28%

Customer Service/Cust Care 33%

Human Resources 15%

Supply Chain 19%

Manufacturing 17%

Product Research and Dev 24%

Technology Research and Dev 45%

Regulatory and Compliance 24%

Corporate Executive (CEO, CIO) 25%

Other 1%

0% 10% 20% 30% 40% 50% 60% 70% 80%

EMA Research Report:

Big Data Impacts on IT Infrastructure and Management

The Primary Sponsors of Big Data Projects

The business goals of a big data project are typically driven by the organizational sponsors of that

project. Thus, we asked participants to identify the organizational sponsors of big data projects inside

their companies. Again, the bias toward IT professionals is apparent here, with 76% of respondents

identifying “IT and the data center organization” as a primary sponsor. The technology research and

development organization proved to a prominent secondary sponsor of projects (45%). No other group

within the enterprise, however, was very likely to be a sponsor of big data, although “customer service

and customer care” was identified as a primary sponsor in one-third of companies.

Figure 3: Which organizational groups are primary sponsors of big data projects?

There were, however, some predictable variations by vertical industry. For instance, 38% of

manufacturers reported that the supply chain group is a primary sponsor of big data projects compared

to only 19% of the overall pool of participants. Supply chain optimization is critical to manufacturers

as it can reduce costs and enhance business continuity. Clearly, manufacturers are looking to advanced

analytics to identify ways to improve this crucial part of their business. Meanwhile, 40% of financial

firms identified the regulatory and compliance group as a primary sponsor of big data while only

24% of the respondents overall did. Regulatory compliance is a moving target for the financial and

insurance industry as regulatory bodies are constantly updating their requirements. Also, visibility

into business activity is a major challenge for compliance organizations. Big data can provide the

unexpected insights that can enhance the visibility of compliance controls and help an organization

adapt to new requirements.

EMA Research Report:

Big Data Impacts on IT Infrastructure and Management

The Primary Users of Big Data Analytics Outputs

The specter of IT bias emerges again with the question of which groups within the enterprise are

primary users of big data. Database administrators were the clear leaders among user groups (63%).

The only other group using big data in a majority of enterprises was IT planning and operations (51%).

Line of Business executives

Database administrators: Data analysts (e.g., IT analysts)

Marketing analysts: Finance analysts (e.g., business analysts)

Data scientists (e.g., statistical analysts: data mining specialist:

predictive modelers)

Application developers (e.g., programmers)

External users (e.g., partners: customers: service providers)

Report writers and dashboard builders (e.g., business

intelligence analysts)

IT planning, ops

IT Security

Other 0%

0% 10% 20%

Figure 4: Which groups are primary users of big data analytics outputs?

However, there are some significant minorities of non-IT users consuming data outputs. For instance,

line-of-business executives were primary users of big data in 39% of enterprises, and marketing analysts

and finance analysts were primary users in 39% of enterprises.

Through the lens of these contextual questions, it is apparent that the majority of survey participants

came from enterprises where IT organizations set the agenda for big data projects. The business goal

drivers, the organizational sponsors, and the users of big data outputs are all heavily weighted toward

the IT organization. At the same time, this data shows that big data projects serve a broad set of use

cases inside and outside of IT, regardless of whether IT is leading the way.

39%

63%

39%

47%

43%

28%

29%

51%

44%

30% 40% 50% 60% 70%