CIOs on BSM: An Interview with BMC Software's Mark Settle
by Bill Keyworth
Mark Settle joined BMC Software in June 2008. He has served as the CIO of four Fortune 300 companies: Corporate Express, Arrow Electronics, Visa International, and Occidental Petroleum. Settle has worked in a variety of industries including consumer products, high tech distribution, financial services, and oil and gas. During the early stages of his career, he was the director of a systems integration business unit within Hughes Aircraft Company. Settle’s formal training is in the Geological Sciences. He received his Bachelor’s and Master’s degrees from MIT and a PhD from Brown University. Settle is a former Air Force officer and NASA Program Scientist.
BSM Review editor Bill Keyworth interviews Settle on the challenges and opportunities of Business Service Managment in his role as CIO at BMC Software.
As an IT Service Management (ITSM) vendor, BMC has a history of demonstrating thought leadership for business service management (BSM), and that leadership would create a unique set of demands for you as the CIO within BMC. How has that market positioning impacted your responsibility to align your IT community with the business needs of BMC?
There's something obviously very unique about my role at BMC. I have the good fortune being the CIO of an organization that consumes all of the commercial products we supply to our customers. We routinely install beta versions of major releases in our production environments before they are made generally available to BMC customers. As the proverbial ‘first customer’ for these products, we inevitably encounter issues regarding installation, inoperability, performance and scalability that we flag for remediation before a product is released. We also have a Service Management Office which is responsible for building internal operational processes around the BMC tool suite. Our Service Management Office ensures that we are implementing service management practices at an enterprise level spanning all of our technical domains. We continually refine our BSM management practices as new features and functions are incorporated in BMC products.
This is your first role as the CIO of a major software vendor, how did your previous CIO experiences prepare you for this job?
I like to remind other CIOs that every IT shop practices service management to one degree or another. Every shop has to process incidents, release changes into production, monitor system availability, etc. Variations in service management practices from one organization to another are largely a matter of sophistication and consistency, not of nature or intent. This is the fifth time I’ve served as the CIO of a public company, so I’ve been a service management practitioner for many years – even if I might not have used that terminology explicitly in the past!
Through sheer serendipity, the CIO role at BMC has turned out to be one of the most enjoyable jobs I have ever had. Our organization provides all of the standard IT capabilities required to support the business operations of a $2B company, but we are also able to contribute to the development of BMC’s products and support BMC’s sales and marketing efforts. We provide substantive feedback to the R&D organization regarding the functionality of our products and we also discuss the business benefits we have achieved through BSM to prospective BMC customers.
What activities and metrics do you use to measure your success with your customers?
There are several internal and external metrics that I track to monitor the effectiveness of our IT organization from a service delivery perspective. The three external metrics that are most readily apparent to our internal customers are the availability and reliability of our business applications, our ability to resolve end user problems on our service desk, and our ability to deliver new capabilities to production on time and on budget. Application availability and reliability are ‘table stakes’ in any private sector company. If you aren’t able to keep your systems running reliably, you likely won’t survive as a CIO for very long. Project management skills are essential to establishing credibility with the executive management of any corporation. Every company is constantly evaluating its investment opportunities and if it chooses to invest valuable capital and people resources in some type of IT initiative, IT has to live up to its commitment to generate the promised return on that investment. Failure to deliver on those commitments tends to undermine future requests for investment funding. Finally, the service desk has a huge impact on the perceptions of IT in any company. The service desk staff needs to be knowledgeable, empathetic with their customers and able to relieve end user pain quickly and reliably.
I use the term internal metrics to refer to performance metrics that are less apparent to IT’s customers. As an IT professional I loathe service disruptions that have been inadvertently triggered through operator error or a breakdown in our internal procedures. These incidents are commonly referred to as ‘self inflicted wounds’ and represent a subset of the critical incidents that occur every quarter. I also monitor the utilization of past hardware and software purchases. We tie up a lot of capital in such purchases and frequently fail to fully exploit the capabilities we have purchased in the past. Finally, I try to quantify the automation of operational practices to the maximum extent possible. Like many other IT shops, we have very limited opportunities to expand our headcount but we are constantly introducing new capabilities into production – a good example being the recent introduction of smartphones and tablet computers. To support new capabilities we have to continually find ways of automating the support practices for legacy hardware and software assets.
We have deployed BSM processes and supporting tools to monitor all of the metrics referenced above.
What are some of the most important BSM capabilities you deliver to the business base?
I take great pride in the service catalog we have constructed for BMC. All of our performance metrics and costs are channeled through our service catalog. We refer to individual business applications or systems as technical services. We group the technical services that support a specific business function into a business service. For example, our Sales Automation service consists of a collection of sales-related technical applications. P&L owners and functional managers frankly don’t care about performance metrics and spending plans for all of IT. They simply want to know ‘what have you done for me lately and how much do you cost?’ Our service catalog enables that type of conversation and frankly forestalls a lot of other more dysfunctional discussions that typically occur between IT and its business clients.
The leaders within IT who are responsible for individual services are referred to as Service Managers. Our Service Managers typically review their performance and costs with their business clients on a quarterly basis.
The service catalog also plays a significant role in making strategic decisions about where to expand or contract IT staffing levels and where to make investments in additional hardware and software. We might ask the question ‘what would be the payoff of an incremental $500K investment in Sales Automation versus Order Management in terms of improved service quality?’
So you package your capabilities in such a way that you’re able to sell or position your IT services to your business customers. Do you use the service catalog as a marketing tool?
I would say it’s more of an accountability tool. We take all of our incurred costs and we portray them in a service context. That’s actually been very illuminating because we inevitably incur costs – especially labor costs – in a somewhat different manner than originally planned in our budget. Like most other organizations we budget our annual costs within cost centers, but the way we spend our time and the way we purchase new hardware and software can deviate significantly from the cost center-based view of resource allocation at the beginning of the fiscal year. Everyone within IT participates in a time tracking system that allows us to monitor the allocation of labor costs to services and not just cost centers. So you may have a situation in which a member of the information security team is coding time to Sales Automation, or a member of the business intelligence team is coding time to Order Management. We’ve discovered that the way people actually use their time and make hardware/software investments can be very counter intuitive, if you are basing your intuition on a cost center view of the world.
If you have a variety of groups supporting an individual service, how do you manage accountability, both internally among the different arms of your organization and to your customers in the business?
Let’s take Order Management as an example. We might have our order management applications up and running for the entire month of April with no problems at all. At the same time, the network switch in our Amsterdam office could have failed for two days which would have made it impossible to enter European orders for 48 hours. In a lot of IT shops, the application group would claim that the availability of their applications was 100% because all applications were functioning normally in the data center. In a service context, however, we would actually have logged a 48 hour outage in the availability of our Order Management service because a significant user group was unable to access key order management applications during the month. To the best of our ability, we attempt to define service quality as an integrated set of IT capabilities as experienced by our end users, not in terms of individual applications and infrastructure components.
So in fact you’re holding all groups accountable for the final delivery that their business users actually see in the European environment.
Yes. It’s critically important to evaluate your performance from your users’ perspectives. Ram Charan, a widely published business consultant, counsels IT organizations to adopt an ‘outside in’ perspective in evaluating their efficiency and effectiveness instead of the ‘inside out’ viewpoint that IT managers instinctively adopt. IT organizations typically spend a lot of time evangelizing their value using metrics of their own making, not in terms of ‘common sense’ metrics that their users might propose.
I think that’s probably one of the highest aspirations of Business Service Management: uniting the tribes or constituencies within an IT organization and get them to look at what’s being delivered to their customers in a more comprehensive, all inclusive fashion. You could have a checklist of the ten things that need to work properly for the person to enter the order in Amsterdam. But from a business perspective, your users could really care less if nine out of the ten critical interconnections are operational while one is not. If one key element of the service delivery infrastructure has failed, then the entire IT organization has failed from the user’s perspective. The end user thinks ‘we spend a lot of money on you guys, we let you buy a lot of cool technology but I wasn’t able to get my job done at the end of the day’. If your staff doesn’t understand the concept of integrated service delivery, then nine out of ten people in IT are walking around with the misinformed impression that we’ve actually done our job when our customers have a completely different impression!
Would you identify this perspective as a form of “business service maturity?”
There are obviously technical ways of evaluating BSM process maturity, but I would tend to define it in terms of the discussions and decisions that take place within an IT management team. Many individuals functioning in Service Manager roles typically have backgrounds in application development and maintenance. They are typically selected to become Service Managers because of the domain knowledge they’ve developed over the years by enhancing existing applications and implementing new ones.
If a business client wants to invest a million dollars to improve the quality of the technical services supporting their function, the typical application manager will immediately think about the long list of enhancement requests that have been underfunded for the past three years. An application manager would instinctively want to spend that money to hire staff members or contractors who could address the enhancement requests that have languished for multiple quarters.
I would hope that the instinctive reaction of a Service Manager would be quite different. The Manager of the Order Management service referenced above should ask ‘what would it cost to install a new switch in Amsterdam, since that’s where I had my greatest service quality issues last quarter’. IT investment decisions are rarely made in this fashion. They frequently are brokered compromises among the warring technical tribes that make up most IT organizations with limited, fragmented links to service quality issues. It’s the ‘inside out’ perspective: ‘service will be better if we upgrade servers, replace switches, upgrade to the current version of the vendor’s software package, etc.’ without any analytical basis for demonstrating that quality from an end user perspective will really be materially better.
I think service quality is a pretty fundamental shift in thinking for most IT organizations. You have to feel accountable to the order entry employees in Amsterdam in terms of service delivery and not accountable to the performance of the technology that you helped purchase and operate. It sounds simple and fairly obvious, but it’s a huge shift for many individuals in key management and technical roles in many IT shops.
Do you have the kind of IT individuals that can adopt these behaviors and communicate successfully with your business clients, in the ‘inside out’ terms that business executives understand?
Service Managers function as the principal translators between IT and its business customers. We constantly seek ways of bringing business leaders into IT to discuss their long range strategies, near term plans and the key business metrics that they’re tracking, We try to make sure there’s an ongoing dialogue between the IT management team and the leaders of our business units and functional groups. I think that’s a common goal of all IT shops, irrespective of their commitment to business service management.
IT managers at BMC have the dual responsibility of being competent IT professionals and key contributors to BMC’s business goals. As IT professionals, we know there are standard performance metrics that every IT shop should use to evaluate its performance. We’re not afraid to share our internal ‘report card’ with our business clients. However, we try to go the ‘extra mile’ to link our metrics to the business metrics that our customers are being asked to deliver to the corporation.
What are appropriate expectations of business leaders regarding IT business contributions?
We try to establish the linkage between business metrics and IT metrics within our service catalog. It’s been my experience that many business leaders don’t have a very crisp idea of the operational metrics that really drive their success. It’s very easy in many cases for functional leaders or business executives to focus solely on financial metrics, without a clear understanding of the day-to-day and week-to-week transactional metrics that ultimately drive financial performance. Financial results are lagging indicators of successful transactional performance. The key to managing financial results is ensuring that the right transactional metrics, such as clean order entry, on time delivery, minimizing rework/recalls, first call resolution of customer complaints, etc. are happening on a routine basis within specific functional areas.
Do you find yourself in a tutorial role with some of your business executives, trying to help them understand the kind of metrics that you can provide, in order to give them an insight into the real indicators versus those lagging financial indicators?
Yes. I have found less of a need to do that here at BMC. But in many places I have previously worked I have had to assume a more tutorial role. I don’t think that’s an uncommon problem. You often naively think ‘…oh, I’ll just go over and talk to them and they’ll immediately be able to tell me the business metrics we need to enable from an IT perspective……it will be a short conversation and I’ll get a fairly crisp answer.’ But a lot of times you don’t get a very crisp answer, and it’s a lot longer conversation than originally anticipated.
I think it’s assumed that most business executives know how to use technology to their advantage. Is connecting the dots between IT metrics and business outcomes a form of value that you personally bring to the table?
Pretty much so. I think another area where IT can add value is connecting the dots from one functional area to another. For example, I’ll pick on sales and marketing. You could have a marketing organization that has developed a set of function-specific metrics that suggest they have become increasingly effective over the past two years. On the other hand, the productivity of the sales organization, measured in terms of closed deals and revenue generation, may have been relatively constant over the same period of time. Admittedly, this is an oversimplified and highly hypothetical example…..but how can that be? How can marketing declare success if its accomplishments aren’t reducing the length of sales cycles, increasing the number of successful sales transactions and helping sales agents close deals of progressively larger value?
IT runs the systems that support the end-to-end business processes required to deliver products and services to a company’s customers. IT is uniquely qualified to identify ‘disconnects’ in any company’s value chain but frequently fails to leverage the value it can generate through its end-to-end view of business process effectiveness. Service Managers should challenge themselves and one another to identify critical cross functional business metrics that represent hand-offs or interfaces among different functional groups where business value can be synergistically expanded or inadvertently lost.
You responded earlier in this interview to a question about the service catalog that it had more to do with accountability than with marketing your services. How do you market yourself? How do you get people to understand what’s possible or what you can really do for them?
What makes this job fun is that I have a whole company that is predisposed to want to learn more about BSM and to push the envelope in terms of implementing BSM practices internally. There aren’t too many other CIOs who have such a willing crew of co-conspirators in making IT more effective through the adoption of BSM practices.
That’s a unique advantage you have working in a company that sells ITSM software to your professional counterparts. You’re in a great place because you are clearly helping business users with their business objectives …in a very unique role. But back to the question about marketing…how do you communicate your value proposition to the larger BMC community? It is because of this unique environment as a software developer of ITSM solutions that you’re so well understood? It seems like in the other companies prior to BMC that you wouldn’t have had that advantage.
To be truthful, most of the other companies I’ve worked for had much more complex business models. The business model of a software vendor is pretty simple: you have to build quality software and be able to market and sell it to customers. I’ve worked in the oil and gas industry in the past, and those organizations have a much more complex business model. In oil and gas, you have to find the oil, produce it, transport it, trade it, refine it and retail the refined products. It’s much harder to innovate within the framework of that type of business model from an IT perspective.
The value proposition of the IT group at BMC is pretty simple. We reliably keep our business systems and productivity tools up and running on a routine basis; we provide the R&D organization with ‘real world’ feedback on the utility of our products from an operational perspective; and we build BSM practices around our tools that we can share with prospective customers to demonstrate the business value that can be created using BMC products. Our ability to provide direct, quantifiable feedback to R&D is very critical to our role and how we’re perceived within the company. By the way, our nickname from R&D is ‘Customer Zero’. Many members of our staff have Customer Zero t-shirts!
That nickname and t-shirt certainly conveys a desired perception of IT Operations by BMC. What are you doing to perpetuate that positive viewpoint?
It’s important for IT to constantly work on organizational transparency. Companies spend a lot of money on IT but frankly most of the employees outside of the IT function have very little appreciation about what goes on inside IT. That lack of knowledge can be toxic, because individuals – both executives and staff members – will frequently assume the worst in the absence of knowledge. So it’s very important to share information about the cost of service, operational performance, current priorities, ability to deliver projects on time and on budget, etc. Even if there are lapses in performance or cost overruns on projects or whatever – it’s better in the long run to be transparent with the business clients who are ultimately paying the bill, rather than trying to camouflage the truth with a bunch of technical jargon or double talk. It’s paradoxical but transparency actually drives higher levels of performance within IT, because once everyone knows the metrics that will be publicly posted and displayed throughout the company, the entire IT organization comes together to ensure that those metrics are positive.
The other critical role that CIOs play in perpetuating positive perceptions about IT is to bring the voice of their customers into the IT organization. IT organizations are typically very inwardly focused. Look at the schedules of most IT managers and technical leaders. They spend most of their day going to meetings with other IT managers and technical leaders. It’s very easy to become disconnected from the tactical objectives and strategic plans of your internal business clients – and that’s always a bad thing within any company. There’s no prescriptive way of promoting executive level interactions with the business leaders in a company, but CIOs have unique responsibility for fostering this type of communication because of their positional authority within a company.
In my current role at BMC, I also have the opportunity to meet and consult with my true ‘end customers’ - other IT professionals who are using BMC products. I would strongly recommend that all CIOs spend time with their company’s customers. It leads to a much keener perception about ways in which IT can create value within the company. Without that direct customer contact, the IT organization takes its cues from internal functional groups that are trying to interpret and translate customer needs and desires. Unfortunately, each of those functional groups has their own priorities and agendas. Perceptions about ways in which IT can add value may be overly influenced by those agendas and priorities, instead of the true needs of the customers. Every level of IT management – from Director and above – should be trying to break through the internal translation services being performed by functional teams within the company. IT managers need to ultimately spend time with the users of the company’s products and services. I think the longer and more frequently you do that, you’ll inevitably find ways of delivering more fundamental value to the company than simply pandering to the never ending stream of enhancement requests that are coming from internal functional groups.
To meet the needs of a highly distributed business, are you effectively delivering service through a private cloud?
Very much so. We’ve got six R&D campuses in different parts of the world …four in North America, one in Israel, and one in India. They were operating fairly autonomously three years ago. Developers at each site tended to use the hardware assets available at that specific site. We’ve virtualized the majority of those assets, pooled them globally and installed WAN accelerator technology to reduce latency across the sites.
Today, we could have a developer in San Jose using a virtual machine in the Boston area and a colleague in San Jose who could be testing software on a QA server in Houston. We operate the pool on a global basis and try to optimize utilization across the sites. So we are private cloud operators.
We have about 6000 total physical hosts distributed across the six campuses. We operate a ‘library model’ in which developers check development resources in and out of the global pool. We monitor utilization of the assets that have been ‘checked out’ and publish delinquency reports when assigned resources go unused for extended periods of time. We employ Amazon and Rackspace to access additional capacity on demand if we need to perform certain types of scalability tests or proof of concept experiments.
Who carries responsibility for governance, risk, compliance, security issues …is that within your organization?
Yes, with respect to all of our business applications. We work cooperatively with R&D to secure the code that goes into our commercial products. Security remains a growing issue for most CIOs. The threat landscape continues to grow and almost every IT organization continues to spend more on information security on a year-over-year basis. But there’s this gnawing sense that the threats are growing faster than the protective measures we are able to implement. It’s not a very satisfying state of affairs.
It’s clear that you have implemented BSM concepts and practices pretty broadly within your organization at BMC. What kind of counsel or guidance would you give other CIOs who are interested in BSM but worried that it may become a distraction if pursued within their organizations?
My perspective on BSM is probably pretty similar to that of a lot of other IT professionals. Properly implemented, BSM should go a long ways towards improving the current operational practices of any IT organization. It’s hard to debate the relevance and conceptual benefits of BSM. But as with any process improvement crusade, CIOs worry about the bureaucracy, administrative overhead and creeping theology that might accompany a major BSM implementation effort. When CIOs evaluate the amount of effort that would be involved in realigning their current practices around formal BSM principles, the resistance of their organization to such changes and their success of such efforts in the past, it’s easy to get discouraged.
In essence, all BSM is trying to do is help improve the repeatability, reliability and effectiveness of routine operational practices. When CIOs tell me that ‘we don’t do BSM in my organization’, I challenge them. If you have three staff members in your organization taking calls from your users, you are operating an incident management process whether you call it that or not. All BSM is trying to accomplish is to manage that incident process in a more repeatable, reliable, effective way. It’s hard to argue with the proposition that we should all be trying to get a little better at performing those types of tasks. That’s one way to de-mystify BSM, to de-theologize it and really think about what we’re trying to accomplish through BSM implementations.
The other key driver for BSM adoption is the tremendous amount of reactive work that goes on in IT organizations all of the time. The dirty secret in most IT shops is that a fair portion of this reactive work is devoted to problems of our own making, caused by manual error or failure to follow prescribed processes. BSM represents a means of channeling organizational energy into doing things right the first time and eliminating some of the reactive work that keeps the staff chasing their tails day in and day out. That’s another value proposition for BSM implementation that’s difficult to contest.
In many IT shops, it feels as if the quality movement of the 70’s and 80’s completely bypassed IT. When you talk about ‘doing things right the first time’ by running discovery tools to populate a CMDB, building service impact models for Tier 1 & 2 business applications, establishing a monitoring strategy based upon the single points of failure in the impact models, etc., you are met with blank stares or rolled eyes in many IT organizations. When faced with this challenge, many operations teams think to themselves: ‘that’s a lot of work! We don’t have time for that! Why don’t we wait for things to go wrong and we’ll just fix problems when they occur’. When I encounter people who are a little overwhelmed by the concept of implementing BSM practices, I ask them ‘what if you took all of the daily effort that goes into reactively fixing problems and reallocated a fraction of that effort into proactively managing your operations?’ At the risk of being overly simplistic, that’s what BSM is really all about. …why don’t we do it right the first time and then avoid all of the repetitious manual stuff that goes on every day?
I routinely monitor the amount of time we devote to production support internally. As our BSM practices have matured, we’ve been able to reduce production support efforts within our infrastructure and operations teams by roughly ten percentage points over the last six quarters. That’s equivalent to more than 15 FTEs worth of work effort that we’ve been able to repurpose for sustaining engineering and new projects. That’s really what BSM is all about – stop the reactive craziness that can absorb so much organizational energy and develop a sense of professional pride about doing things right the first time.
So if you are a CIO how do you ‘sell’ or market a BSM initiative to your own organization?
The first overarching theme is really around continuous improvement. You have existing operational practices – you can call them ITIL processes or BSM practices, whatever terminology is ‘safe’ within your organization. I’ve been in some companies where a public proclamation about implementing ITIL processes would have led some business leaders to conclude that IT was overstaffed (i.e. ‘if we can afford to have an ITIL project, we’ve got too many people working in IT!!’) Ironically, these same business leaders are probably the ones who chronically complain about IT’s effectiveness! The other theme to emphasize with your staff is the need to extract the IT organization from constantly reacting to problems that were largely avoidable. Time is too precious to be wasted on those types of activities and that’s a message that resonates with all IT managers. I’m not so Pollyannaish to think that all reactive work will be eliminated through BSM but I sincerely believe that persistent focus on continuous process improvement can go a long ways towards reducing the ‘hamster on a treadmill’ despair that pervades many operations teams.
Isn’t there a ‘chicken and an egg’ problem here? How can you get started if you are spending all of your time fire fighting?
It’s ironic, but in many cases, the operations team knows exactly what needs to be done. But they’ve become so resigned to operating in a chaotic fashion that they would tell you ‘…oh, that can’t be done around here…we know we should do things in a more structured, proactive fashion but there’s no way we can get there from where we are today’. If the team intuitively knows that change is required and that change would be beneficial, it’s ultimately a failure of IT leadership to provide the ‘air cover’ needed to begin the continuous improvement process. If calling it BSM helps provide structure and focus to making things better, then leverage it! If the BSM framework and process methodologies can’t be referenced in a politically safe fashion within your organization, then camouflage the external trappings and get on with it anyway!
Register for our monthly newsletter