Learn how this large and diverse agency of state agencies uses the best of centralized and de-centralized IT resources, largely built on a common VMware-powered SDDC strategy.
At the recent VMworld 2014 Conference in San Francisco, our moderator, Dana Gardner, Principal Analyst at Interarbor Solutions, interviewed Tony Morshed, Chief Technology Officer for the California Resources Data Center in the California Department of Water Resources, and Michael Hom, Data Center Chief in the IT Infrastructure Services Branch for the California Department of Water Resources.
Here are some excerpts:
Gardner: You’re from the largest US state. You have a large government, probably the size of what many countries have. Is there something different about the public sector in terms of accountability or response? How do you have a different set of requirements as a public organization and perhaps enterprises, too?
Morshed: We do have a difference. Our procurement is much harder. Getting people is much harder. We live within a lot of constraints that the private sector doesn’t realize. We have a hard time adjusting our work levels. Can we get more people now? No. It takes forever to get more people, if you can ever get them.
Gardner: So it’s doing more with less over and over again?
Morshed: Constantly doing more with less. Part of this virtualization is survivability. We would never be able to survive or give our business the tools they need to do their business without this. We would just be a sinking ship.
Gardner: So the whole philosophy, Michael, of SDDC and virtualization, is of doing more with less, of automating, of boiling out the manual processes and going to more real-time responsive technology driven infrastructure makes total sense for your organization?
Hom: Definitely. To go with what Tony says, we really don’t have much overhead when we need to respond to future projects. When there’s an uptick in activity, there is no way to have more resources available. So we need to build that into our infrastructure to allow for that dynamic bandwidth to happen from a personal level.
Gardner: Help us understand the size of your organization. This is a large state government department, but you’re really a department of departments.
Morshed: Our department, Water Resources, consists of 3,500 people that are part of the agency that comprises many departments. The bigger ones are Parks and Rec, Cal Fire, and Fish and Wildlife. There are about 28 agencies and conservancies and 25,000 people.
Gardner: So in order to support all these people and all these different agencies, a common infrastructure is important, but also it sounds like you need to have differentiation and customization for their specific needs. How do you accomplish both the goal of a common infrastructure for efficiency, but also still be able to meet all your requirements for all those different people?
Morshed: When we started our consolidation effort, we decided to transform ourselves more to an ISP-style setup. We know that most departments have their own IT shops, and you know there is still that trust thing. So we just built the common infrastructure. We let them share the infrastructure, but they have their own security posture. We segregate all their traffic, so each department can still feel like they’re autonomous, but yet we all share the infrastructure, in which case we all share the savings.
Mandate to consolidate
Hom: With that, we had a mandate to also consolidate. First, the State of California is really about cost savings. Each of the 25-plus organizations mostly had their own IT shops. By combining the infrastructure as a service (IaaS) in our multitenancy data center, they’re able to reap the benefits of cost savings for infrastructure, but also concentrate each department’s specific needs and applications.
Gardner: What are some of the challenges you have in terms of getting closer to a full SDDC? Are these technology, culture, process, or all of the above?
Morshed: At first it was technology, but the culture and the organizational mindset are the bigger challenge. You can find solutions and work through technology. We’re IT people, and we’re used to the technology problems. For me, the greater organizational problems — and the structure of your processes — become the harder things, because we’re not as used to dealing with those things.
Gardner: Now, Michael, part of the adoption of a complex, long-term journey on IT is to show results early and get buy-in. Are there instances where you look at that? Maybe, it’s the DR, where you can have a sense of better dependability on your resources? Any way to describe what you’ve done early on that has lead to a greater emphasis to adopt more aggressively?
Better service levels
Hom: Definitely. One of the key things with early wins is providing a better service level for provisioning, and that’s something that everybody has been struggling with. With the cloud infrastructure, we’ve been able to provision within days, if not a day. And that typically beats most of the service tools that each of the organizations have had. So that was an early win.
It’s things like that where we decrease overhead and make IT more accessible for business. That makes it a win, and starts to have the ball rolling as far as other features, such as DR, greater capacity, and things that would be tough for each individual organization to do on their own.
Gardner: Are you exploring software-defined storage, software-defined networking?
Morshed: When the software-defined stuff came out, for me, one of the big things was disaster recovery (DR). If I could stretch my data center into another facility, DR becomes a non-issue, because those workloads can shift between sites without any trouble with automation.
That was the next piece for us — automation. We realize that we’re part-way there, but to get all the way there, we need to do fuller automation. This means that we need to quit tinkering with the network and storage every time we want to do something new.
Those were big driving factors for going to SDDC. To us, it means that we’re obfuscating the hardware. The hardware’s there. It’s just running. We’re working and tweaking everything at the software layer — so we could be a lot more agile.
Separating the physical
Hom: For the SDDC we really wanted to provide a logical data center to each of our organizations. We wanted to separate the physical, which allows our folks to support more of a logical infrastructure, where they still have autonomy, but the physical layer is basically one and the same.
Today, from a functional point of view, they get what they’ve had before, but without the overhead of physical support. We’ve used the VMware vSphere and vCloud Suite to provide that software-defined queuing. Right now, we’re embarking on a software-defined networking, using VMware NSX and third-party vendors to support that.
We’re looking to use automation soon to help us decrease overhead, and pass those savings on to each of the organizations.
Gardner: We’re here of course at VMworld 2014. There’s lot of news going on. Anything in particular piquing your interest, perhaps with the OpenStack support in the EVO Hyper-Converged Infrastructure? What is now on your agenda after hearing some news to reach those goals as you describe them?
EVO looks pretty nice
Morshed: There are a couple of things. EVO looks pretty nice. I was out on the floor and looking at it yesterday and talking with the CIO and I see it as something that we might be able to use for some of our outlying offices, where we have around 100 to 150 people. We can drop something like that in, put virtual desktop infrastructure (VDI) on it, and deliver VDI services to them locally, so they don’t have to worry about that traffic going over the wide area network (WAN).
The other piece is the acquisition made recently with CloudVolumes and looking at how we can use that to leverage our VDI structure. We’re using another product right now in that space, but again with CloudVolumes it’s been a part of VMworld. It’s more interesting, because we know that the chances of all the software being upgraded and updated in at the same time in interoperability is greater if it’s a VMware product.
For us, it’s been a real struggle to make sure that all the products that we use, interact and as there’s an upgrade, everything upgrades at the same time. To me, those are the two biggest things that I’m getting out of the announcements.
Gardner: Do you have any metrics of success that you can point to from your SDDC effort so far?
Morshed: We do have some tangible benefits. We have reduced our CAPEX by somewhere around 40 percent and our OPEX around 32 percent. I don’t have the numbers, but we have deployed VDI in the Department of Water Resources and we already virtualized about 600 to 800 desktops. Not only is it helping us save costs there; it’s also used as a strategy for a remote access as a strategy to help protect our server infrastructure by using VDI for admins.
So there are those tangible things that you can reach out and measure and those intangible things, where it’s allowing us to do something easier and more flexible. That, for me, is the bigger win. The business could come up with more dollars, but to be able to be more agile and more flexible is where it really pays off.
Gardner: So we get productivity, we have better DR, which reduces your risk, and we’ve got some hard savings and economics. It’s pretty compelling. Michael, any thoughts about how those fit together, and which ones are more important to you?
Hom: Definitely, this allows us to be more flexible, and there are some things that we’re trying to do that we would never imagine without a SDDC. So they increased security, greater capacity, capabilities to our business.
Gardner: How about VMware specifically? Is there some differentiator in terms of how they produce this products that has allowed you to follow this journey? Is this more of a partnership than a procurement relationship? It sounds like the track that VMware takes in its strategy very much aligns with yours.
Morshed: It’s very much a partnership. In fact, we basically only want to work with business partners. We don’t want to work with vendors, because we don’t need someone to sell us something and walk away. VMware has been hand-in-hand with us for this whole journey.
When we look at other products for the mix, we look for the deep partners with VMware because we know virtual machines (VMs) are core. So, when we look in the storage partners and we’re looking at networking partner, we’re making sure that those partners are partners of VMware.
One of the things that we find is the inoperability once everything has been virtualized. Everything has to connect, and it’s not a single stack. So if one thing gets upgraded, we need to make sure that everything across a stack can accept that upgrade. Otherwise, we lose the ability to take the advantage of the upgrade until everybody else catches up.
Early on, we were in that position and we’re doing everything we can to remove ourselves from that position.
Hom: We consider VMware a strategic partner. A couple of things that illustrate that is that we’ve been involved with the VMware’s Excellware and Velocity program and that’s been two-fold. For the velocity side, we have marked up a fully working SDDC with SX, with virtualized automation, operations in business as a stack.
Gardner: One of the things we’ve heard here at VMworld is be bold, be brave, be a little bit aggressive. Go out there and do these things. Any thoughts for other organizations that are just dipping their toes into the water? Is it higher risk than reward to be bold and brave in getting early? Or is it perhaps something that allows you to then be a differentiator and be better in your own environment, whether it’s a public or private sector?
Set in stone
Morshed: The first thing is always question what you’ve got that’s set in stone, because most of it is not set in stone. We’ve all heard a lot of things that you can’t do. You can’t virtualize Oracle, but you can. You can’t do this, you can’t do that, you can’t get the network focused at the top of the storage. That’s all that stuff that you actually can do.
You have to really look at it, peel it back, make sure that “you can’t” is an actual thing, and then figure out how to get around it. The way I see it is that, as the world turns, things morph, and if you don’t move into this virtualization space, you’re going to be left behind. You’re going to be the guy making buggy whips. There are no buggy whips running around. There’s no use for them.
We’re all being asked to do so much more with the same resources or fewer resources. We’re all being pushed to keep up with how the demand is going out there. Technology is just jumping, and this is the only way on the infrastructure side to keep up with that.
Gardner: Michael, any other thoughts in terms of 20/20 hindsight on your experience and why being aggressive and being bold has paid off?
Hom: Virtualization is definitely up and running, at least in state organizations. It’s probably something that we might do that or we might use as a toolset, but from looking at VMware this week, virtualization is the industry standard.
If you don’t take it on, then you really won’t be able to respond to business needs. What happens is that when the official IT organization becomes obsolete, there are going to be ad-hoc IT organizations and those would become the norm. If you want to be relevant, you have to use every tool set that you can to provide the business needs.
You may also be interested in:
- Creative Solutions in Healthcare improves client services and saves money using VMware vCloud Air hybrid cloud
- University of New Mexico delivers efficient IT services by centralizing on secure, managed cloud automation
- A Gift That Keeps Giving, Software-Defined Storage Now Demonstrates Architecture-Wide Benefits
- Nimble Storage leverages big data and cloud to produce data performance optimization on the fly
- Navicure Gains IT Capacity Optimization and Performance Monitoring Using VMware vCenter Operations Manager
- Big data should eclipse cloud as priority for enterprises
- Different paths to cloud and SaaS enablement yield similar major benefits for Press Ganey and Planview
- Time to give storage virtualization a top place at IT efficiency table