For the last few months I have been thinking a lot about this issue. Let me start out by introducing our family pet Alice. She was a rescue from the local shelter that we added to our family about two years ago. As you can see at time of rescue she needed a haircut badly… and a bath. I remember when we brought her home the whole family washed her and babied here to death. She has turned out to be a great family pet providing a number of benefits including exercising our four year old with hours of chase the toy and tug of war. She provides a valuable service to my family. She also hops up on my lap at exactly 5:30 PM every night and lick/annoys me until she gets her dinner. At 7:30 PM she is restless and expects to be let outside for the sole purpose of receiving her daily chew stick. She requires two walks a day or she has an “accident” in my back yard. She is a lot of work. Not less valuable but a lot of work. The real problem with a family pet is they don’t scale… from time to time my children ask about getting a “insert some type of house pet here” to which my response is the age tested trick of well if you walked the dog… maybe… I am becoming such an old man. I have no intention of getting any more family pets because it’s just one more thing I have to manage and I don’t have time for it. There is no way I can manage seven family pets let alone 200.
What happened to Virtual Me and why are you talking about pets?
This is a valid question has someone stolen my password? Nope. I think we can learn a lot from Alice. I think she has me trained. Everyone has infrastructure like Alice. They provide valued services but require lots of care and feeding. Early in my professional life I remember that after a older admin retired we kept having issues with a server that forced reboots and outages. When I asked my retired friend about the problem his response was “that’s because you don’t reboot it every morning”. I came to learn he had been rebooting the server every morning including weekends for five years. I think the server had him trained pretty well.. Again the problem is it does not scale. Imagine having four dogs all that require bones at different times… you could not possibly manage the nightmare.
OK So how did all my servers become pets?
One word: Customization. It’s the bane of human existence. Image taking a boat and turning it into a car… it may operate and provide the basic function of providing ground transportation but maintenance will be a huge pain. Every systems administrator is a tweaker… they love to tinker and make things better. All of these little changes provide 2% benefit but cannot be supported by a larger group. Think my dog again.. if you watch my dog and at 7:30 PM she goes to the back door… you would let her out. When she does not get her bone she will again ask to go out and you will let her… rise and repeat this process about 40 more times and you start to wonder if my dog is crazy. What has happened is she has a undocumented tweak to the process. Going outside has nothing to do with getting a bone except for her they have always followed in succession (in fact they just did again). These tweaks are almost always not documented and even if they are you will have to read for an hour to understand it… it’s easy for me to write 7:30 PM let dog out give her bone… but you might ask why? Customization cannot be your friend if you expect to scale out.
I am ready to give up all customization but my customer might need some
You are correct… customization is what we do…. but I just told you to give it up. Here is where farm cattle come into play. Dairy cows on a daily farm live out their lives for a single purpose from a business perspective to provide milk. Everything is handled together… when it’s washing time they all get hosed off… when it’s milking time they all go to the barn. When a cow stops producing product then they are removed and replaced with a younger cow. We need to get to servers like cows. They provide a business function or service and if that function stops working they are replaced not diagnosed or tweaked or rebooted once a day. They are removed. Ten years ago you would have called me crazy but virtualization has enabled the rip and replace model. But in IT we are still caring for family pets… we love them and they love us… guess what we are starting to look like the crazy cat lady. We have more pets than we can handle and we scare the business units. To make matters worse our cats cost a lot of money are slow to deploy and come in some crazy colors. Business wants a dairy farm while we run cat ladies house.
What can the auto industry teach us?
The auto industry has been facing this problem for a long time. They started with hand made cars and moved through automation. They learned you cannot customize everything but you can offer options. We need to offer three models of servers and different types… some are mini-vans others sports cars. We can offer customization but we need to automate and use modules. Let me say that again : We need to automate and use modules. We have to move build times from perhaps weeks or days to hours or seconds. We need to automate the life cycle of our services. If we don’t we will become the cat lady not the future oriented technology experts.
What do you think?
Did I just force you to loose a portion of your soul? Where do you think IT is heading? Share let me know. I definitely don’t have all the answers… either way thanks for reading and letting me rant.. I will return you to your normally scheduled content.