|
|
|
|
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Executive Summary
Monitoring network performance is an essential practice in today’s IT operations environments, as a
means of assuring the network’s critical role in supporting the served organization is being adequately
fulfilled – not just in terms of availability, but also in terms of efficiency and effectiveness. As IP
networks themselves become increasingly stable and reliable, operational focus has turned to devel-
oping a better, more discrete understanding of how the services and applications crossing the network
are performing, and the level of quality which IT end users, customers, and partners are experiencing in
accessing and utilizing them. This Enterprise Management Associates (EMA) Radar™ Report reviews
18 (eighteen) providers of Application-Aware Network Performance Management (ANPM) solutions
and compares their ability to deliver basic and advanced application awareness across a range network
and application technologies.
Introduction and Methodology
In the development of this Radar Report, EMA engaged eighteen top providers of ANPM solutions
in a detailed analysis of the scope and capabilities of their offerings. The solution providers represent a
rich cross-section of the IT management tools landscape, ranging from small to very large, from pure
software to appliance-only, and from point products to extensive multi-component/multi-function
suites. ANPM solution providers covered in the report are: Apparent Networks, ASG, CA Technologies,
Compuware, Dorado Software, ExtraHop Networks, Visual Network Systems, InfoVista, Lancope,
ManageEngine, NetScout Systems, Network Instruments, OPNET Technologies, Plixer International,
Quest Software, SevOne, SolarWinds, and WildPackets.
An extensive questionnaire was developed and presented to solution providers for their input, covering
details regarding architecture, integration, functionality, deployment, administration, cost, and vendor
strength. EMA supplemented responses with dialog, product demonstrations, and reviews to ensure
that each solution was represented fully, honestly, and fairly. EMA also interviewed over twenty
end-user customers of the solutions being reviewed – in some cases more than one per solution
provider – in order to validate vendor claims. The degree to which customers were readily provided
and available for dialog was one of the many indicators used for validating ANPM solutions.
Finally, and importantly, EMA leveraged ongoing industry dialogs and extensive existing knowledge of
the ANPM solution space to evaluate, consider, and validate each vendor’s strengths and limitations in
a manner that is focused on providing balanced, consistent insights across all vendors and solutions.
EMA has produced a report specially targeted at presenting and explaining Radar Reports in general:
How to Use the EMA Radar Report, EMA, April 2010. The goal is to use a combined approach for
quantitatively and qualitatively evaluating providers of solutions in a particular IT management
functional area and presenting their relative differences in a clear, graphical format. Also included is
a detailed discussion of individual criteria and how each participating solution provider rated versus
those criteria.
1
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Figure 1: The EMA Radar is optimized to show how vendor solutions cluster in terms of two primary axes: Vendor Strength (architecture,
integration, functionality) and Cost Efficiency (ease of administration, deployment, support & services, costs advantage)
Quoting from How to Use the EMA Radar Report, “No analysis of this type can tell you which vendor
is best for you. The data collected for an EMA Radar Report can certainly be used to make that deter-
mination, but it must be applied to the specifics of your current environment, level of maturity, and
goals and priorities. Since the authors of any given Radar Report do not have your unique specifics,
the Radar Report can only be a starting place and a guideline. It can inform you of the market and
short-cut your process to developing a short list.”
Functionality
Architecture &
Deployment &
Integration
Administration
Vendor Strength
Cost Advantage
Functionality
Architecture &
Integration
Vendor Strength
Cost Advantage
Deployment &
Administration
Strong Product – Higher Price
Specialized Product – Low Cost
Figure 2: Radars for each vendor solution are included in the full report and show a five-
axis contrast between the average profile and the vendor in question.
2
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Application Awareness in Network Performance
Management
Network management is considered by many to be a well-developed, mature discipline, with roots
extending back to the first computing networks. Network performance management is a relatively
newer phenomenon, though it too has been around for a considerable period of time. The latest comer
to the game is application awareness built into and on top of network performance management. As
IT operations teams make the transition from tactical firefighting to strategic, proactive assurance, one
of the most important unifying and enabling angles has been widespread awareness of the need to
understand IT user experience. And what IT users experience is not the network – they experience the
applications and services that the network delivers. Consequently, building awareness of applications
and services and how they are traversing the network has arguably become one of the most important
focal points for expanding network operations practices today.
Starting with the advent of RMON over 20 years ago, early network management technologists recog-
nized that application and service information is available from the network perspective, if you know
where and how to look for it. And while RMON itself is no longer a primary nor broadly used
foundation, the idea of putting the health and operation of the network in context with the payloads
being delivered has become an indispensable aspect of responsible management practices.
Today’s application-aware network performance management (ANPM) solutions are many and varied,
some of them are delivered by independent software vendors, some of the technology is contributed
by network equipment vendors, and some of the solutions are part of large software vendors’ offerings
of multipart integrated management suites. The techniques being used to deliver application awareness
are similarly varied, though essentially clustered around four key mechanisms:
1. Packet inspection – This technique is perhaps the most comprehensive, and delivers appli-
cation visibility by looking into packet headers as well as deeper packet contents in order to
recognize and monitor application and service use by user, allow detailed application trans-
action analysis, support detailed and definitive troubleshooting, and enable reconstructive/
forensic study. Packet inspection can deliver visibility up and down the stack, across network
and application layers and can be used to calculate response times and latencies – the heart of
end-user experience measurement.
2. Flow records – These are transaction records issued by network infrastructure elements, and
provide information regarding who is using the network, what applications and services are
being used, and how well those applications or services have been delivered. The most typical
industry example of flow records is NetFlow from Cisco, but there are many other essen-
tially similar variants, such as JFlow, NetStream, cflow, IPFIX (an industry standard) and the
statistically sampled sFlow. Flow records provide extensive traffic intelligence, but cannot be
used alone for detailed application analysis, determination of response times, or recognition
of errors.
3. Passive and Synthetic agents – Passive agents are software elements installed on either end
client systems or servers which observe and report traffic statistics including response times.
Synthetic agents generate test traffic in a variety of patterns to assess both availability and
performance of specific applications or services and characterize various aspects of a simulated
transaction or user experience. IP SLA (a Cisco device feature) is commonly used as a type of
synthetic agent, which can take a range of test measurements between Cisco infrastructure
devices and end-test targets.
3
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
4. Log file analysis – Valuable application-oriented activity records can be found in syslog and
similar data files which capture activities and events from various viewpoints within the
network-connected infrastructure. Log files are not typically considered a primary source of
ANPM metrics; however, they can provide an importantly complementary set of data points
for monitoring and troubleshooting when used in conjunction with one or more of the other
three types.
ANPM solutions utilize one or more of these data source types by collecting application-aware perfor-
mance data and delivering the following major categories of functionality:
1. Application discovery, recognition, and monitoring – this is the primary area that differen-
tiates ANPM from simpler NPM solutions – the ability to discern individual applications and
services from broader measures of traffic volume and utilization.
2. Troubleshooting and analysis – most ANPM solutions are purchased and deployed for the
purposes of delivering troubleshooting and deep, often “expert” analysis capabilities to accel-
erate incident response times and restoration of services.
3. Capacity planning – detailed insights into how the network is being used and what (and who)
is driving traffic growth are available from ANPM solutions and are fast becoming essential
information for best practices in network engineering and planning
4. Collaborative reports and dashboards – the ANPM field of vision is where the business
meets technology and represents a means for understanding how well the applications and
services upon which the organization depends are performing. But none of that value can
be fully realized without effective methods for sharing insights and intelligence, both across
domain silos within IT as well as with served constituencies such as end users, customers, and
partners.
In order to be included in this EMA Radar Report, ANPM solution providers needed to offer all four
of the major functionality categories listed above and must have direct support within their own (inter-
nally-developed) products for at least one of the three primary ANPM data source types – packets,
flow records, or agents.
Criteria
In all EMA Radar Reports, EMA evaluates solutions based on five key areas: Deployment and
Administration, Cost Advantage, Architecture and Integration, Functionality and Vendor Strength. The last
category, perhaps the only one that’s not self-explanatory, is focused on the market and industry
presence, vision, and financial stability of the vendor. In each of the evaluation areas, EMA created
a “superset” of capabilities spanning the known solutions in the marketplace, added questions about
new and emerging areas (e.g., virtualization and cloud), and balanced the result with standard compar-
ators used across all EMA Radar Report projects. The evaluation model used for this ANPM Radar
Report is presented as Figure 3. Following are details on the evaluation areas and the specific scope
and rating priorities used within each.
4
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
Deploying and administering an ANPM solution is the point of embarkation for the ANPM journey.
In this category, we assessed several important areas:
• Ease of Deployment – This includes a number of measures meant to indicate how easy or difficult
it is to put a particular ANPM solution into the production environment and begin to draw value
from it for operational monitoring purposes. As such, this section addressed three areas:
◦ Implementation Cost – Specific questions assessed time to receive initial reports, time to achieve
complete functionality, and the percentage of the solution cost which is typically required for
professional deployment services. Also included in this part of the assessment were questions
regarding product deployments models – software vs. hardware vs. virtual appliances (or other)
for ANPM central servers as well as for distributed ANPM instrumentation/collectors, plus
how complete, or “out of the box” the solution is as delivered by the ANPM supplier. Highest
ratings were given for rapid deployment and low or zero need for professional services, as well
as for those offering flexibility and multiple options in product deployment models.
5
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
for assessing ANPM solutions, is described below.
Deployment & Administration
Market Credibility
Figure 3. Assessment model for Application-Aware Network Performance Management
Cost Efficiency
The first set of measures conducted within the EMA Radar Report framework and one of the two
major axes of the Radar distribution diagram is Cost Efficiency, which consists of two major sections
– Deployment & Administration and Cost Advantage. Each of these, and how they have been applied
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
SUB-CATEGORIES
Ease of Deployment
Support & Services
Ease of Administration
Licensing Model
Price
Maintenance Costs
Architecture
Integration & Interoperability
Features
Ease of Use
Partnerships/Channel
Vision and Strategy
Financial Strength
PROFILE SCORES
Deployment & Admin
EMA RADAR SCORES
Deployment Cost Efficiency
Cost Advantage
Architecture & Integration
Product Strength
Functionality
Vendor Strength
Vendor Strength
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
◦ Staff Training – In this section, the breadth of training options as well as the length of time
expected for administrators to reach basic and advanced proficiency were determined, as well
as whether or not the ANPM technology provider offered formal certification programs. Top
marks were given to those with broad training options, short learning curves, and existing
certification opportunities.
◦ Disruption Minimization – Many ANPM solutions can be deployed without any disruption to
the managed environment, but others will require downtime for software installation, network
tapping, or other deployment steps. Some require scheduled network downtime, or a maintenance
window in production operations. This measure assesses the impact each ANPM solution has on
the monitored network during deployment, with preference given to those with lesser impact.
• Support and Services – An important part of any management solution is the facilities made
available by the technology supplier to support initial rollout as well as ongoing production use.
In this section, we evaluated several specific areas of interest for supporting and servicing ANPM
solutions:
◦ Customer Support – This area investigated the variety of customer support offerings, guaranteed
response times for highest support levels, methods for reporting product issues, and diagnostic
information gathered at the point of failure. Also of interest was the presence of organized
user community groups, which can act as a powerful supplemental resource to technology
users. Highest ratings in this category were given to those with broad support offerings, fast
responsiveness, and a well-developed and well-organized user community.
◦ Professional Services – Within the range of ANPM offerings included in this study, some require
significant professional services to fully deploy, while others require virtually none. In this
category, we gave the highest rankings to solutions that could be deployed with minimal efforts
or cost.
◦ Code Fixes – Every system goes through patches, minor upgrades, and major upgrades as
functionality is added and problems are fixed. Generally, these are all good changes, but there is a
balance to be struck here – too much change creates chaos and stability risks in the management
tools. In this section, we asked about the frequency of minor and major incremental software
updates, and gave the highest rankings to solutions that were updated regularly, but not overly
frequently.
• Ease of Administration – Once an ANPM solution has been deployed, focus turns towards
ongoing configuration and administration, to ensure that the system remains fully functional and
that the maximum value can be realized. In this section, we investigated several categories that
helped to illuminate each ANPM solution’s administration facilities:
◦ Ease of Admin – Here, the intent was to assess how much time was required by operations staff
to keep the ANPM solution up and running, with preferential ranking granted to those requiring
the lightest touch.
◦ Update Process – When it comes time to apply a patch or upgrade to an ANPM solution, two
questions are predominant. What will be the impact to my continuity of monitoring? And how
can large numbers of distributed instrumentation devices be updated efficiently? Highest scores
in this section were given to solutions that had a means to transparently apply updates without
interrupting coverage and to those that included (where applicable) features for en masse updates
of remote instrumentation.
6
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
◦ Testing/Migration Facility – While not universally the case, many IT shops require pre-deployment
shake-downs of new or upgraded technologies (including management tools) before rolling
them out into the production environment. Our interest here was to determine whether or not
ANPM solution providers gave discounts for test labs and short-term migration project use of
their products. Highest scores were given to those who charged lesser licensing fees (or none at
all) for such uses.
◦ Automation of Management – Increasingly, managed environments are becoming more complex, as
are the tools that are used to manage them. EMA is a strong advocate for automation whenever
and wherever possible within management tools, technologies, and practices as a means to extend
human operator resources, keep up with highly dynamic IT infrastructures, and reduce error
introduction rates. As it applies to ANPM, we asked if solutions included features and capabilities
such as self-configuration, automatic adjustments to changing managed environment conditions,
autopopulation/autodiscovery, and wizards and templates for configuring data source devices
(such as a GUI front end as an alternative to command-line configuration to setup up NetFlow
or IP SLA). The more automated features, the higher the ANPM solution was rated.
Cost Advantage
All management tools carry costs of one type or another. The most obvious and commonly recognized
are the licensing costs associated with the tools themselves. But importantly, there are other aspects of
total cost of ownership (TCO) for ANPM tools that are also relevant, particularly the cost of mainte-
nance (support and upgrades) for the technology. And it must be noted that solutions that carry a high
licensing cost may still provide a compelling return on investment by favorably assuring operations.
For this portion of our analysis, EMA focused on the typical licensing costs for an initial deployment,
what types of licensing models are offered, maintenance fees for the highest levels of support services,
and whether or not ANPM technologies providers offered creative delivery mechanisms such as SaaS
(software as a service) or via MSPs (managed service providers). These latter two approaches have
shown the greatest traction within smaller shops, but even large organizations are finding them advan-
tageous as a means to support regional facilities and/or supplement core network operations staff
during off hours.
Highest marks in this section were given to those providers offering lower-cost entry points, lower
maintenance fees, and SaaS or MSP delivery models. The only preferences assigned to licensing model
(i.e., processor-based, appliance-based, agent-based, usage-based) was to those ANPM providers who
offered multiple options over a single (and hence less flexible) approach.
Product Strength
The second major axis of evaluation within the EMA Radar Report framework is that of Product
Strength. This category is comprised of two focus areas – Architecture & Integration and Functionality.
Details on how these areas have been addressed specifically for ANPM are provided below.
Architecture & Integration
The first of the two major product strength categories is Architecture & Integration portion, and is
meant to gauge the underlying enabling technology base upon which the bulk of ANPM function-
ality is delivered. Following are the areas of analysis used in this research report for evaluating the
alternatives and methods for architecting an ANPM solution:
7
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
• Design – As mentioned earlier, there are several basic approaches to monitoring application
performance from the network perspective. One of the key measures in this study was
ascertaining each solution’s scope and inclusion of the various types of ANPM data sources.
Beyond monitoring, management solutions can also provide similar levels of visibility while also
including active control functions. An ANPM solution may be broad or narrow in this regard, and
may be designed to be more or less real-time. It may also go beyond application-specific metrics
to collect other collateral and supportive data. Top scores in the category were given to those
solutions that included true real-time capabilities, those solutions that included an ability to take
or invoke closed-loop corrective actions, for breadth of ANPM data sources supported (packet
inspection, NetFlow/xFlow, agents, IP SLA and log files), and for integral support of additional
supplemental/complementary data sources.
• Scalability – The basic need for any ANPM solution to support collection and storage of
large volumes of performance metrics goes without saying; however, solution scope in terms
of throughput capacity as well as distributed coverage are important points for consideration.
In terms of scalability, top scores were given to those systems capable of both high volume
processing of ANPM data as well as architectural support for very large scale, distributed
monitoring deployments.
• Breadth of Environments and Applications Supported – While some managed environments
are well standardized and are thus relatively “simple,” most have a mix of networking technologies
in play. The same can typically be said regarding the number and type of applications that are
present and are expected to be visible via an ANPM solution. In general, the broader the better,
so that barriers to coverage and visibility are minimized. It is also important to recognize that
there exists a mainstream of network and application technologies that must, at minimum, be
supported. Weightings in these categories were tilted towards support for mainstream network
and application types but also for diversity and breadth, along with the ability to accommodate
custom/non-published application types.
• Integration and Interoperability – While some ANPM solutions will come tightly integrated into
a multifunction, multi-capacity suite of management tools, most will not live in a homogeneous
environment. Consequently, it is very important that ANPM solutions be able to integrate and
interoperate with products and technologies from other vendors that fulfill other complementary
functions. Of particular interest within this research were integrations between ANPM solutions
and event/fault management systems (most commonly those that are on the big screens in a
Network Operation Center), service management systems (most commonly help desk applications
but also service operations or BSM dashboards), and CMDB/CMS solutions (present either as
part of an ITSM/ITIL initiative or in conjunction with a higher level BSM solution). Also of
interest was whether or not open APIs are available for integration with any other IT or non-IT
applications and functions. Highest ratings in this category were granted to those solutions that
had proven (certified, fielded, supported) integrations within each of the three complementary
functional areas as well as ample API options for custom integration.
8
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Functionality
Functional completeness and scope is the second major angle of analysis applied in this research in
determining relative product strength. Following are the primary Functionality criteria that were used
as part of this research and analysis:
• Application Discovery/Recognition – If a network performance management solution is to
be application aware, one of the most important aspects of that solution is the way in which it
identifies those applications. Further, to the more general assessment of application types supported
(as discussed above within the architecture and integration section), this inquiry focused more
specifically on the mechanisms for identifying applications and for how new, unknown application
flows are presented to a system operator. Important for rating well in this category was a range of
choices and options, so that the many different, subtle and unique identifiers of various types of
applications can be accommodated.
• Metrics and Measurement – Performance management systems generally gather a wide range
and large volume of performance metrics. Presented in this category were those considered most
important for characterizing application activity from the network perspective, including volume,
response times, errors, and quality by application/user/server. Also included was a special
question about support for VoIP quality measurements, since there are specific and discrete
metrics that are applied to voice traffic, such as MOS and R-Factor. Since the focus of this
research is application awareness, traditional device-centric health measures were not included.
As with many other categories, scoring in this area was based on the breadth of metrics and
measurements supported.
• Capacity Planning – One of the primary uses of application-aware data is reality-based planning,
whereby capacity monitoring and changes to network capacity can become informed decisions
made in the full context of understanding how the network resources are being used. In particular,
recognizing the influence of individual or groups of applications and the contributions they make
is paramount for reducing both infrastructure cost as well as operational risk. In this measure, we
looked for support of long-range trending reports, including trend extrapolation as well as the
ability to conduct “what-if ” analysis based on current performance conditions.
• Alerting/Alarming – When things go wrong, and performance problems are recognized, it is
essential that operations personnel be notified of the situation as quickly as possible. It is also
important not to set off lights and sirens too often – today’s interconnected and interdependent
IT infrastructures generate enough event and alert chatter even before performance monitoring
alarms are added into the mix. Basic performance alerts and alarms need to recognize short-term
and long-term patterns, as well as include as much information as possible to assist subsequent
investigation and diagnosis. Additionally, a growing number of ANPM solutions are including
behavioral modeling to recognize unusual patterns of observed activity, either in the volume of
transactions/flows/sessions or the in the total traffic bit volumes. Some ANPM systems are also
able to identify special/unique performance issue scenarios, such as microbursting in multicast
traffic – a transient, sub-second phenomenon that requires true real-time, packet-based monitoring
technologies. Scoring here tilted in favor of those solutions that provide the broadest set of alerting
and alarming supports.
• Troubleshooting – Whether reacting to a performance problem reported by the help desk
or proactively investigating a growing issue that has not yet been recognized by the end-user
community, rapid and efficient troubleshooting is perhaps one of the most important objectives of
network management and operations. For ANPM solutions, combining various types/sources of
9
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
data, accelerating workflows, and intuitively presenting data best facilitate troubleshooting. Further,
troubleshooting is most effective if analysis can include current and historical data, including
forensic reconstruction, and there are special analysis features included for discrete application-
layer technologies such as Rich Internet Applications, multicast, industry-specific application or
control protocols, VoIP/video, WLAN, or custom/in-house developed applications. Highest
rankings in the category were granted for efficiency features, support for forensic/reconstructive
analysis, and the presence of special analysis capabilities.
• Security and User Management – Since an ANPM solution will have visibility into detailed
user activity as well as potentially proprietary or private information, it must provide some form
of access controls. And while most ANPM solutions are not deployed for the purpose of security
management per se, ANPM solutions are often able to recognize potential security events. Some
solutions have been designed or optimized for this parallel purpose while others have not. Scoring
in this category was prioritized towards solutions that offer strong credentialing, discrete controls
to data and functions, and those systems designed to play either a complementary or direct role in
security operations.
• Analytics/Advanced Analysis – The leading edge of management technologies apply
automated, intelligent analysis to the data collected by monitoring systems. Such capabilities can
deliver better early recognition of performance problems, support for complex infrastructures,
and/or accommodation of recent technology innovations such as virtualized computing and
cloud services. This research included an assessment of the degree to which each ANPM
provider has developed and included a range of advanced functions, from dynamic thresholding
and baseline shift/drift recognition to data mining, and route analytics. Also of interest was
specific support for correlating ANPM data in the service of monitoring intra-datacenter n-tier
architectures, shared/balanced network links, outsourced/cloud services, and mixed physical/
virtual computing environments.
• Active controls – Beyond monitoring and analysis, some ANPM solutions will deliver the ability
to take direct actions in response to existing or pending/potential performance problems. The
extent of active controls can be quite broad, ranging from intrinsic direct capabilities to scripting
to triggering actions within other management tools. This is an emerging area of functionality
for most ANPM solution providers, unless they provide ANPM functionality as an adjunct to
a core optimization value (as in the case of many WAN optimization controller vendors) or if
they have change and configuration management capabilities elsewhere in their management tools
product lines. Highest scores here were given if direct controls were available; however, none of
the participants enjoyed this capability and thus the most common responses related either to
other products offered by the vendor, integration with third-party control systems, or simple script
launching capabilities.
• Ease of Use – The final area of assessment that contributes to the overall Product Strength score
is Ease of Use, which encompasses the ability of the ANPM solution to be used directly as a
means for collaboration between the various groups with IT and with IT’s service constituencies.
Along these lines, we looked for the ability to group ANPM monitoring and reporting in various
ways, such as by business/organization construct, geography, address range, application type,
or technology type. We also looked for integrated support for business/service prioritization,
application dependency recognition/mapping, and understanding of logical and/or physical
topologies. Next, we assessed each ANPM solution’s support for a broad mix of visualization
and reporting functions, such as consoles, portlets/mash-ups, and scheduled and ad-hoc reports.
10
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Finally, we asked whether or not each ANPM solution had the means to specifically support
consoles, portals, and/or reports for a broad range of discrete roles, spanning IT (network,
systems, applications, storage, service desk, service management, etc.) as well as non-IT (executive,
financial, Line of Business) functions. As with so many of the other areas of this study, the
highest scores were granted to solutions which supported the broadest range of options and
optimizations, as well as the ability for operators to tune the system to their specific needs without
the direct involvement of the ANPM solution provider.
Vendor Strength
The third and final major axis of analysis and comparison within the EMA Radar Report framework
is that of Vendor Strength. This section is a combination of measures that are meant to gauge
not only the financial viability of an ANPM solution provider, but also the quality of their vision,
strategy, go-to-market, and market voice. As such, following are specific categories used to assess
vendor strength and how they have been applied to ANPM solutions:
• Vision – The purpose of testing vision in the ANPM sector is to understand each solution
provider’s viewpoint of who they are, what value they provide, and where they fit into the broader
ecosystem of management tools and practices. First, an understanding of the role that ANPM plays
and the value that ANPM solutions can and should deliver in terms of IT operational efficiency
and operator effectiveness was assessed. Next, each provider’s understanding and appreciation of
integrated IT service management (ITSM) and the broader move of IT towards service-oriented
operational models was gauged. Finally, specific attention was paid to how each supplier articulated
their relationship and role in supporting and empowering their customers.
• Strategy – While vision is tuned towards a broad understanding of role and purpose, strategy
is meant to assess how each provider plans to achieve their vision. Consequently, this measure
was specifically attuned to functional roadmaps and plans to evolve ANPM technology and total
solution scope/capability over time.
• Financial Strength – An important aspect of selecting an ANPM solution provider is to
understand their viability as well as their ability and commitment to ongoing development of their
offering. Key measures used in this category included organizational size and revenues (specific to
ANPM as well as overall), access to capital, profitability, and investment rate in R&D.
• Partnerships/Channel – While ANPM solution providers must develop and deliver key technology
as a basis of their approaches, an important complementary element to their effectiveness and
presence takes the form of business alliances. Strength of partnerships was evaluated by measures
such as number/breadth/depth of technology alliances and breadth of channel relationships.
More is generally (though not always) better in both cases, and credit was given based on the length
of time partnerships were active in the field.
• Market Credibility – Beyond simple presence and visibility in the marketplace, which is more of
a measure of marketing budget than anything else, the ability of an ANPM solution provider to
achieve and maintain credibility is important when assessing their overall strength. In evaluating
credibility, EMA examined a number of measures, including how focused the provider is on the
ANPM space, whether or not it is an exclusive focus or a supplemental position supporting an
adjacent core competence, how often they compete directly with other ANPM solutions providers,
and which other credible industry voices are backing them up.
11
EMA Radar™ for Application-Aware Network Performance Management Q3 2010 Summary
©2010 Enterprise Management Associates, Inc. All Rights Reserved.
|
|
|
|
