EMA Webinar Transcript:

Change Happens How to Get Ahead with Network

Automation

Webinar Date:

July 22, 2010

Featured Speakers:

Jim Frey

Matt Gowarty

Abstract:

Undoubtedly, the rate of change in today’s distributed and virtualized IT infrastructures is

accelerating. Without a means for recognizing and managing change, the inevitable

results are surprises of the type organizations would rather not encounter: unstable

performance, connectivity issues, compliance breaches, and security risks.

There is a solution, however. An essential technique for keeping up with runaway rates of

change and getting ahead of the game is to employ automation within your management

tools and practices. But what types of automation are really ready for prime time? And,

how can they best be put to use?

You’ll discover the answer to these questions and more when you join EMA Research

Director Jim Frey and Infoblox Sr. Product Marketing Manager Matt Gowarty for this

one-hour Webinar to learn:

The current state of automation techniques and technologies within network

management

How these techniques and technologies are being used in the real world

What the top priorities should be for deploying them

Introduction:

Raleigh Gould

Welcome and thank you for joining us today for Change Happens: How to Get Ahead with

Network Automation. My name is Raleigh Gould and I will be your moderator for today’s event.

Our featured speakers are Jim Frey, Research Director at Enterprise Management Associates and

Matt Gowarty, Senior Product Marketing Manager at Infoblox.

Page 1 of 18

Jim has 22 years of experience in the computing industry, developing, deploying, managing and

marketing software and hardware products, with the last 16 of those years spent in network

management, straddling both enterprise and service provider sectors. At EMA Jim is responsible

for the network management practice area. Jim’s past experience, prior to joining EMA, includes

serving as VP of Marketing for NetScout as well as VP of Strategic Marketing for Micromuse.

Matt is the Senior Product Marketing Manager of Infoblox with over 13 years of experience in

telecommunications and networking. Matt has delivered over 400 presentations across the world,

with content including network change and configuration management, VoIP, MPLS based

networking and network and application performance management. Matt has been a guest

speaker at numerous events including Network World, Interop, Cisco Live and FutureNet.

Before I hand this over to Jim I did want to mention to the audience that we will be concluding

today’s event with a Q & A session. While Jim and Matt will defer your questions until the

conclusion of today’s event you can log your questions at anytime using the Q & A functionality

located in the right hand corner of your screen. If you are in full slide view, simply look for the

floating toolbar, you will see a question mark icon, click that icon and you can log your questions

that way. Also, today’s event will be recorded so you will be receiving a follow-up email with a

link to the on-demand recording as well as a PDF of the speaker’s presentations.

And now I would like to go ahead and turn things over to our first featured speaker, Jim Frey.

Jim Frey

Thanks Raleigh and welcome everyone, thanks for joining us today. We are going to be talking

about how to deal with change and how to apply network automation to deal with change, a topic

which hopefully everyone will find some value in.

I’d like to start by just giving an overview of what my portion of the presentation will cover. We

are going to talk about what is driving IT operations professionals to engage and embrace

automation, what we see as the sort of automation landscape, as well as the state of how that is

being applied in, specifically, the network management practice area. Then we will talk a little

bit more about what we mean by the shortened term of network automation and how that applies

as a subset of network management automation. We will then wrap up with some priorities, what

things should you be looking to accomplish with network automation.

I would like to start with giving you a little bit of a view into EMA’s landscape of what we see

happening in terms of the evolutionary and maturity growth in IT management in general.

Enterprise Management Associates focuses exclusively on the tools and technologies and

practices used to manage information technologies. What we see as the way organizations are

moving and the way the disciplines are maturing over time is from reactive sort of mode of

operations eventually into a more active, proactive, and eventually to a truly dynamic mode of

operations.

Then you can see that we also track that across several different measures or categories. We look

at processes, we look at the tools and technologies, we look at the degree of automation and we

look at how focused the result is on the business or organization that is being supported.

Page 2 of 18

So with respect to automation, which is the topic for today, you will notice a couple of things.

First of all, we recognize that automation is applied across every one of these phases, to a lesser

or greater extent. So it’s a good, relevant topic and we see it as a catalyst that helps organizations

move to the right, into the higher levels of maturity. Now you will also notice, if you look at the

dynamic column on the far right, at that point automation is being applied across all of these

different categories. We believe that this is one of the only ways that you will get to a truly

dynamic and truly effective, most possibly highest level of effectiveness in IT management

practices.

I also want to share with you something of a landscape for how we keep up with and where we

look for automation, what are the centers of automation. We watch several clusters, if you will,

where automation is being applied. To us automation includes recognition of all 3 types of

interactions that are happening within IT. Whether it’s machine to machine, human to machine

or even human to human workflow. As you will see here we have mapped these two service

management issues, like ITIL and the ITIL practice groups. So ITSM automation is one

category, data center another, service management automation, specifically oriented to automated

triage and diagnostics and then more broadly by cycle application automation.

For the purposes of what we are looking at today, most of what we will be talking about would

fall into the categories of data center automation and service management automation. And for

ITIL practice areas this means the service transition and service operation areas.

The next thing we will want to talk about here is, of course all of this has to be done along with

and can provide the opportunity to deal with the financial side. Certainly there are functional

objectives or capability objectives in looking at any sort of automation but it ends up being

important when you talk about how you are dealing with the cost of administration. We all know

that the cost of ongoing maintenance and administration of information technology infrastructure

usually far outstrips the initial cost of the technology and the deployment in itself. So there is an

ongoing focus, at all times, on what is it costing us just to run the shop?

So you have some options, of course, look to those at the top here. How do you keep a lid on the

total cost of your operations? You can always consolidate your staff and their responsibilities,

you can try to reduce facilities and simplify the management environment, you can try to find a

way to negotiate your ongoing costs of support contracts, for instance and you could even look at

consolidating the number of management tools that you use. Everyone one of those has

downsides because they are counteracted by the things listed at the bottom here. You are doing

this in spite of the fact that there is a constant influx of new applications and new technologies

being deployed. You are always seeing greater growth in the amount of traffic, the amount of

activity, usually increasing dependence upon networks in particular, since we are specifically

talking about networks today, and more devices being added and regulatory forces that keep

coming along from our good friends in Washington and elsewhere. So the counteracting forces

often times prevent much progress on invoking those options up above. Automation is one of the

areas that is quite possible to start finding some way to achieve that managing the cost of

managing. You may not bring down the cost of management as a total number but you might

help to be able to mitigate its growth.

Page 3 of 18

We did a study last year looking at the cost effectiveness of making some of these types of

changes. This study was based on a case study of how some of these changes were implemented

and the results that they gave over time. You will notice that just working better and reducing the

upfront licensing costs were beneficial but by far the strongest payback, in terms of making a

change, came out of making improvements in the level of automation. Part of that is because it

affects multiple metrics that you are looking at for cost control. So in our analysis it actually was

one of the best options for having an impact on long-term cost of operations.

I want to add in a little bit of other research here as we go along. The first one is, during the

downturn that we have been through, and we seem to be coming back out on the plus side here,

everyone’s not sure if we are really out of it yet but one of the validation points on the value of

automation for dealing with cost pressures came out of some research that we did last year where

we were asking, “What’s been happening to you? What’s happening in your shops and how has

the slowdown impacted your priorities?” Sure enough, bubbling right to the top of the various

different options here was investigating automation and ways to add efficiency in their everyday

practice. Some of the others we talked about were the options for managing the cost of

management, reevaluating vendors and strategies and renegotiating. A lot of these were being

utilized but the most popular answer was looking at automation as a way to deal with these

challenges.

Similarly, another piece of research that we did, this was right at the end of the year, we looked at

one of the areas where automation is really being most heavily focused upon and that’s cloud

services and managing the cloud. EMA did a foundation research project called The Responsible

Cloud where we looked at, in this case we had almost 160 participants, who are actively engaged

in using cloud services or are in the process of rolling out cloud services or invoking cloud

services as part of their IT infrastructure. We went through a process and asked all of the aspects

that we could come up with on how are they making this manageable, how are they making sure

that it’s reliable and predictable, how are they putting it into and turning it into production quality

and production reliability? One of the many questions that we asked them during this process

was how are they taking advantage of it and how are they applying it and prioritizing the use of

automation across the various different disciplines that are being looked at as a part of cloud

services engagement? You will notice that automation in general gets very high importance

rankings across all the different disciplinaries that we have here but a couple of them really float

to the top. Storage automation and network automation.

So storage automation, in this case, it’s not entirely surprising that it ends up as really high on the

list, in part because of a question we asked which was, where are they using cloud services and

what do they see as the best place to apply cloud services? And the number one answer to that

question was actually storage and non-critical offsite storage and backup and recovery. So since

that was actually the most popular answer on how cloud was going to be used it makes sense that

storage automation is going to be the most popular and highest priority for applying automation.

The network automation reflects another finding that we had which was that after all the dust

settles most organizations are expecting the network management team and the network

operations team to be most commonly responsible for ensuring the ongoing quality of cloud

services. So there is definitely a belief that a networking team is going to play an important in

Page 4 of 18

monitoring and assurances of cloud and therefore, I think that is something that contributes to this

strong showing in network automation being a priority.

So, a few of the drivers and some of the descriptions around automation. How is this being

applied? How is automation applied in the network management sector and in network

management practice? There are quite a few places. If you look at the different sorts of

disciplines and activities that network managers engage in on a day to day basis, automation is

being applied across the board in a variety of different ways. Certainly in discovery, we have all

heard of auto discovery, that’s been around for a long, long time. Using techniques for

automatically recognizing what is out there and how it’s all connected together and using that to

keep an updated internal representation and model of how all of the infrastructure is related to one

another.

One of the root cause analysis capabilities have been automated over the years as well. Things as

simple as event reduplication, intelligent preprocessing of events to recognize when this is just

another report of the same problem, but also things like downstream suppression as a form of

automation. There are ways to help dealing with the volume of errors and reports that can come

out of a managed network and whittle it down to those things that are going to be really most

important to look at. Another aspect of publishing, I don’t list it here but, some of the expert

analysis and interpretation of patterns of issues or data would also generally fall into this, so

monitoring an alarm management. More ways that automation is being applied, things like

intelligent performance [inaudible 0:14:35.0], things like understanding service impact. In the

reporting and analysis, automating the process of sharing of data, automating the publication of

reports, even automating the process of generating reports based on recognized scenarios or a

combination of factors. Training analysis is a key performance of the care, it’s a very hot area for

applying forms of intelligence and interpretation of automated analysis. A great example there is

the growth in the number of features that are designed to provide early warning indicators, early

recognition of potential problems that look like they are going to be an issue and need to be dealt

with proactively.

And last here, and this is something we are going to spend a little more time on, is the area of

change and configuration management. This is an area that has really developed very rapidly as a

practice maturity area in the last 5 years, the last 10 years overall but especially in the last 5 years,

where there is a great opportunity to apply automation in a number of ways, including auditing

and including consistent distribution of change, including rollback and things like that.

So, what are some keys, what are we trying to accomplish with all this automation in network

management? A lot of it is going to tie back to that cost of how do we manage the cost of

managing? We are definitely looking for better efficiencies, not only to hold down the growth

and cost of administrative loads but also to be there and help to accommodate deployments. This

is not just deployments of network management tools, it’s deployments of new technologies and

new services that are coming out in the managed environment. The more automated the

management tools are the quicker you can be up and running and recognize those new

technologies and be ready to ensure their success.

It’s also helping significantly in the way that collaboration takes place, in not only sharing data

across silos within the IT group, sometimes they are not silos sometimes they are just your

Page 5 of 18

neighbors, but often times, especially in the larger organizations, this is a non-trivial challenge,

sharing data and finding a way to easily share data across groups, but also sharing that data

externally, finding ways to get good, operational intelligence and the right information out to the

right people in the service community. The other thing it is going to do is it is going to put

operators in a position where they can achieve much more improved improvements in the overall

resilience.

So automation is a great way to take the human factor out of as many things as possible. Fat

fingering on command line configuration is always going to be a factor, really good experienced

and well trained people still make errors on occasion, they will make less than a less trained

person, there’s no question about that, but it’s always a risk and the risk is always much lower if

you are dealing with an automated process to make sure that a prevetted configuration is being

rolled out effectively. It definitely comes to play in accelerating the diagnosis of problems and in

shortening restoration times.

And probably one of the more important areas that this can help with is automation in ensuring

that there is consistent adherence to policies. Now policies can be any number of things, policies

can be the configuration of quality of service levels and can be the configuration of security

access, it can also be configurations to help comply with regulatory policies. Ultimately what are

we trying to do in these areas with all of this automation technology? It’s to do a better cost

effective job of making sure that all of the network services are up and the applications are

available.

It’s worth pointing out here that when I say network services most of the time we are talking

about the applications that are delivered across the network but there are also a number of

enabling services that require the same amount of automated configuration and monitoring and

troubleshooting. These are some of the key things happen at the network layer that if they are not

working then nothing works. So these are things like IP address management, like DNS and

directory services. As we all know, if any one of those isn’t working absolutely flawlessly then

there will be significant downside in the quality of service or even the access to services by the

server community.

So let’s now focus in on network automation, we use this term because network automation to us

means taking advantage of the network to be able to have it do as much of the work as possible in

the tool that will be applied to making sure that the network is adapting to the changes required

and adapting to the changing environment. So this is the realm of change and configuration

management, primarily, certainly some of it might be triggered by the monitoring functions but

most of this is going to be in the change and configuration area.

So the features that we would ascribe to network automation are things like what are listed here:

Applying automation in the control of configurations themselves

Applying automation for change, for applying change and for recognizing change

Applying automation to occasionally go out and make sure that everything is the way that it

should be in auditing

Generating reports based on the results of that and sharing the results of these audits and

these changes

Page 6 of 18

Offering the opportunity to automate the restoration and remediation process when problems

do occur

These are all features that can be automated, some more than others, but they all can be

automated and they all fall under that MCCM realm.

The benefits are these, hopefully you will see some parallels here from the early part of the

presentation. What this does is it offers an opportunity to improve that resiliency, to reduce the

amount of errors that are introduced in the process, if manual approaches are being used. It

absolutely offers the opportunity for efficiency, a lot of shops that aren’t doing automated

auditing of their equipment, they do it on a manual spreadsheet basis and it can take weeks to go

out and really effectively complete an audit that with automation can be done in a matter of

minutes. It certainly reduces errors and that contributes to that resiliency element above.

Probably the bottom one is the one that’s really starting to make the most difference, and that’s

to ensure policy compliance. A lot of operational policies, let’s take QOS as an example, anyone

who has rolled out voice over IP broadly in their environment recognizes that they need to take

proactive steps in the way that they configure their network devices to protect the [inaudible

0:22:00.0] voice over IP, if you don’t then you will have quality problems and you will have

irate IP end-users. So you have to do some cleaning of the network and preferably you are going

to be rolling out some consistent QOS policy enforcement.

It’s pretty important that that be done on a consistent basis across all of the network elements

because anyone whose done this in a large environment knows that a lot of that tagging, that

QOS tagging, takes place at the edge of the network and it’s not looked at as you go through the

network so if all of the edge devices haven’t been configured properly then you can get the

wrong traffic in those high priority queues or the high priority traffic in the wrong queue and the

results are not good.

What are some priorities? We’ve talked about what we mean by automation and how we apply

that to networks and then what network automation means as a subset of network management

automation. EMA put together some priorities on what to look at to get out of application of

these network automation technologies. First of all you can use them as a way to truly

understand the current state of your network. Not just to discover the devices and how they are

connected but recognizing how each of them is internally configured and how each of them is

going to be, whether or not they are in compliance with the way that you expect them to be

configured, so recognizing actual and being able to understand quickly how it differs from

what’s expected. And here’s a place where keeping an eye on those critical network services

that I talked about earlier makes a lot of sense.

Next, getting changes under control. Now this is just some of the basic practice of network

change and configuration management but here is where some automation can be very, very

helpful. Number one, for making sure the planned changes that you are making to the network,

whether that is part of regular maintenance, whether it’s part of the regular refresh of security

access or whether it’s part of a big, new technology rollout, often times there are changes that

you are going to have to make to the network infrastructure, making sure that you are doing that

on a consistent, repeatable basis across a large distributed network is very, very important and

can be helped significantly with some automation.

Page 7 of 18

And then lastly here, recognizing those unplanned changes. As we all know, almost every

service impacting incident that happens in IT environments happens because something changed.

Now that something can be something that is out of your control or it can be something that is an

unexpected impact of something that was in your control. One of the most common sources of

those unexpected impacts is changes that were made to configurations of networks or servers that

had these side effects.

One of the most common challenges in troubleshooting problems, especially the more subtle

ones, is getting down to the root of where did that change happen. The quicker you can

recognize those changes, especially if they weren’t planned, the further you are going to be down

the road to restoration.

Number three, becoming part of the team, this is a little bit of motherhood and apple pie I

suppose, but ultimately it’s very important for those that are looking after the CAC to be sharing

information and coordinating activities with the other teams in the organization. So I mentioned

this earlier, a little bit about the process for rolling out changes but it really comes back to

making sure that each group that’s involved in some of these broader initiatives, especially

things like server virtualization, cloud services, we talked about some of the research results we

had there, being interactive, being collaborative is very important in achieving the best efficiency

in these rollouts.

There is an opportunity with network automation to add some real business value. These policy

compliance audits, especially when you are talking about PCI and HIPAA, they are the things

that the CXOs care about, they want to know that they are in compliance with some of these

regulatory demands and if they are not then it’s usually on their head, they are the ones that get

held responsible. So when I say [inaudible 0:26:43.0] it also means that it really delivers some

value to the organization broadly but also to the things that the highest level folks in the

organization care most about.

Finally, this area really does give me an opportunity to make a real difference in helping

organizations that are making this transition towards more service oriented management. We

talked earlier on about the ITIL practice areas that are applicable in this case and that automation

can help with, but this is an example of something that can help shorten the loop or tie a closed

loop back between changes that have an unanticipated impact, being able to recognize that that

impact happened after this change was made and being able to rollback that change to get back

to a position of stability. Those are very good powerful and useful steps towards recognizing the

fact that anything that you do in the network could ultimately impact services that are being

received by the end-user and customer and partner community.

So, how do you make sure that when you’ve done these things that you are getting the right

result? This is some research that we did back in 2008 and we wanted to find out for those shops

that were focusing and those people who had specific responsibility for network change and

configuration management, where this network automation could be applied. What were the

metrics that they were looking at? How could they tell if their efforts were being successful?

All of these are very useful metrics and worthy of consideration. If you are going to roll this out,

look at this as a set of ways to check and see whether or not progress is being made. I think that

there are some interesting ones here. Certainly, reducing the number of trouble tickets. You

Page 8 of 18

wouldn’t necessarily think that that would be related to change management except for the fact

that the unplanned or unstructured or unanticipated effects of changes being made is usually the

source of most IT service impacting incidents, which result in trouble tickets. So you do a better

job there of managing the changes and doing them right and you will reduce trouble tickets.

Closely related is successful versus unsuccessful changes, reducing MTTR, more efficient

compliance audits, we talked about that, it was 51% here a couple of years ago, we are going to

rerun this research soon and I expect to see that climb.

The last one though I think is really important and I want to just spend a short minute talking

about it, which is increasing performance. This ties back to one of the other examples I used

around making sure that QOS policies are properly applied. Making sure that the network is

ready to do its job is an ongoing challenge. Applications are being rolled out depend on and the

IT end-user community and the IT services consumer community absolutely demand the very

best in responsiveness from all of the applications and services that they use. So that increase in

performance is becoming more and more difficult to ensure because of the fact that those apps

and services are more and more complex, behind the curtain, and yet the end-consumer, end-user

community doesn’t really care or know about that complexity, they just know whether or not

when they hit the enter key they get a screen refresh or they get their next form.

So ultimately, keeping on top of and doing the best job of keeping the network as clean as

possible and recognizing the impact and the role that a network plays really will have and can

have a significant impact on improving and protecting performance overall.

So I want to wrap-up and hand this over to Matt so he can tell you a little bit about what Infoblox

is up to in this area but I want to leave you with a couple of sort of wrap-up points here. First of

all, I hope that you have taken away from this that automation really is a very good area to look

at to help you deal both with technical and cost challenges being faced by today’s network

operations professionals. The rates of change, especially around things like virtualization and

cloud, are really aggravating the need for doing this and accelerating this need. Finally, network

change and configuration management really is a great place to look for employing automation

and taking advantage of what automation represents. And lastly, it’s our view that this whole

area of network automation and doing a more dynamic and automated change and configuration

management practice is really shifting now. It’s shifting from being something that was nice to

have but with today’s level of complexity and rates of change it’s really turning into a must have,

all shops should be looking at this and figuring out not if but how they are going to apply these

sorts of approaches.

So, with that, thank you and I will hand over now to Mr. Gowarty who is going to talk to you a

little bit about Infoblox.

Matt Gowarty

Thank you very much Jim and again, thank you to everyone for joining us for today’s

presentation. Before I go into my portion of this I did want to mention this up front, you may

have heard of Infoblox, we are a network change and configuration management and you might

not put those two together, actually in May of this last year Infoblox bought Netcordia and the

NetMRI product line, so now it’s a complete product in that it does network change and

Page 9 of 18

configuration management as well as DNS/DHCP/IP address management that many of you are

familiar with with the Infoblox brand name. So I just wanted to preface that so when you start

seeing certain things that kind of make sense if you didn’t put the two companies together if you

didn’t see the press release.

While we walk through this, I was going through Jim’s presentation, he did a fantastic job of

talking about some of the key values of automation, I was thinking about all of the conferences

that I’ve been to and all the panels and all of the things that I hear and I hear automation and

that’s one of those terms that is really a very broad term and a lot of times it causes concern for

network managers and executives. When they think of automation many times they are thinking

about if everything happens by itself then they have no control along the way and that kind of

aspect of it. When I’m talking automation I’m really not talking about sort of the self-healing

network mentality, where it is just going to do everything by itself. It’s really getting away from

some of those processes and procedures that have been done over the past decades of trying to

manage the infrastructure, those tedious processes, those things that we know cause headaches

but we try to become much more proactive and really reduce some of the risks that we might

have as we go about managing our network infrastructure.

Jim talked a little bit already, but here are just some of the challenges going forward. A lot of us

have been doing this for years or decades and have been doing a lot of the same processes,

whether it’s manual processes where you are scripting to help get some basic information back,

but what we are seeing is that things are getting harder due to the fact that change is the number

one cause of network issues. About two-thirds of all performance issues are tied to change,

whether it be unintended consequences of a planned change or the oops, I didn’t know that

happened, or the undocumented change along the way. It’s not the virus attack or the power

outage or those major things that we always think about, it’s those day to day things that we

always deal with.

For a lot of us, with these past few years with the economy, our resources haven’t been scaling to

our needs. The network is becoming more critical, more important, more users and we are

adding things like new technologies, virtualization, cloud computing and we don’t have the

people to scale and doing things the same old way are getting much more challenging. We

talked about consistency in compliance, Jim put a couple out there with PCI and HIPAA or pick

your own vertical compliances that you have to deal with, if you have best practices on the

government sector, this is a pain for a lot of network managers and engineers to get this audit

done. If you are doing an audit for PCI or HIPAA or NSA on the government space, these can

be hours, days, weeks or months to go out and pull this all down and the challenge is that you are

so busy already and now you are taking your team or your staff off of what they had to be doing

to doing something else with compliance in consistency. So this just makes it harder as you go

throughout your organization, to think about how you manage the infrastructure.

Now what do these challenges bring? Well let’s think about this in terms of how you are going

to manage your network. How did you do it 5 years ago today and then two or three years from

now, you will look at things like the number of IP end points. What is your management cost for

IP addresses? All of the sudden that is going to start [inaudible 0:35:51.0] where I think about

the management of IP address points, how do you do that manually? The old days of having a

spreadsheet to try and manage IP addresses is getting harder to do, when you start getting bigger

Page 10 of 18

and bigger and have merger acquisitions, you can even throw IP

IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING

View Webinar & Listen On-Demand

Back To Library

IT & DATA MANAGEMENT RESEARCH,INDUSTRY ANALYSIS & CONSULTING

View Webinar & Listen On-Demand

Back To Library

IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING