Suppose you build a house. When it’s finished it’s shiny, new, and clean.
Then you live in it for decades. Stacks of old newspapers litter the floors. Sometimes a bunch of these get thrown out. You only do superficial cleaning, and crud accumulates in all nooks and crannies.
You also want to embrace the new. You enthusiastically follow each fashion trend in furniture, appliances, and so forth. A new chair arrives, but the old one is still in use because grandma gets back pains when she uses the new one. You buy new nice plates and cups, but the old ones are not thrown away, just in case. The new robot vacuum cleaner cleans the living room, but the old one is still in a closet for the cleaning the robot cannot do. Even your replacements of worn out stuff leave leftovers all over the place. Still more crud accumulates in all nooks and crannies. Sounds like a badly run household, doesn’t it? However, it also looks very much how many businesses operate with respect to their IT landscapes.
Today, many executives, spurred on by consultants and pundits, are feverishly trying to get on two bandwagons at the same time: the cloud and big data analytics. Both are more than just fashion, true. There is real substance to their appearance, just as the commercial Internet was not only fashion when it burst upon the scene in the ’90s.
But the fact that there is more substance than fashion does not mean that feverish behavior during the dot-com boom did not result in a dot-com bust around the start of the millennium. The boom and bust mainly came from the problem that most initiatives did not have a proper business model. In other words, they didn’t have a way to make money, and this was painted over by grand — but naive — stories about a new economy.
This time, we’re a little bit wiser. We’re very much interested in business models to save money by going to the cloud or make it by using big data analytics.
Home free, then? Not quite, because something else is currently making live for these organizations difficult, namely the well known issue of technical debt.
For instance, the cloud is fine for operators like Google, Facebook, Twitter, Netflix, Spotify, Apple, and so forth, who operate huge homogeneous landscapes. And it’s fine for startups who are building a new house, so to speak. But most businesses these days trying to get on the bandwagon have medium-sized heterogeneous landscapes reflecting their heterogeneous businesses.
The cloud juggernauts are also the first ones to profit from letting loose simple algorithms on big data — they’re also big data analytics juggernauts. Big data, by the way, is these days also known as “the new oil,” a sign of the gold rush fever that currently surrounds it. Always a sign of danger. The medium-sized heterogeneous organizations might have a lot of data (though “a lot” is not very specific, and often is not that much compared to modern day realities) and are now looking at profiting from the new oil.
Hence, most businesses that are now jumping on both bandwagons are not homogeneous juggernauts at all. And it could very well be that, if they are not careful, they are going to pay a price that resembles the dot-com bust.
Their move towards the cloud is fraught with technical bottlenecks, for instance because their applications have not been built for cloud deployment. Another reason is that there is very little attention paid to the ugly crud that has assembled itself in the nooks and crannies of the organizations. Which means that actually making the IT-landscape cloud-ready and integrating (not just tacking-on) big data analytics runs into all kinds of problems because of technical debt.
Even if you have proper life cycle management on your infrastructure (many organisations have this), there may still be a huge amount of technical debt as LCM activities are also just another change in the landscape that often result in new debris here and there, or at least they often do not diminish the debris.
Now, hospitals, schools, factories, most organizations need the basic physical hygiene of the workplace. For hospitals this is clear to many, but it’s a fact of life for every organization. But while the cleaners visit the physical landscape everyday to prevent the dangerous buildup of crud, generally no such thing happens in the virtual landscape.
Some agile projects have cleaners in their teams, but there remains little awareness at corporate level that one needs this kind of thing for the virtual landscape as well. For one, hygiene isn’t sexy. But more importantly, the crud of the virtual landscape is just invisible to most managers. While down on the IT shop floor everybody has to live with it every day, and while strategic management will feel the effects of it (when the organization can’t move quickly enough in the direction they want it to move), it is largely invisible for top management which makes it feel unreal.
If the building is dirty, top management will experience it too. If IT is dirty, they don’t directly. As a result, top management seldom acts on the crud in a sustained manner. Incidentally, the invisibility issue of the business complexity that has resulted from the massive use of IT also hurts the awareness of the need for good Enterprise Architecture.
Many organizations have a long history of grand new schemes and brand new plans and little attention to keeping the IT environment tidy. Most likely their current top managers are next in a series of managers before them that kept bringing in new fashions, couches, plates, cutlery, wall decorations, carpets, while constantly rebuilding, which resulted in a lot if invisible, but real, crud and debris. Currently, the business world is running two fevers at once, fevers that could become rather nasty for some organizations, as they work in unhygienic IT circumstances. And while many think the cloud at least will simplify IT landscapes, both developments will also result in even more complexity and — unexpectedly for many — even more hidden debris.
So, Bi-Modal IT everyone? Well, yes, unless it is just a way to shut your eyes for the crud. In the end, that ‘other’ It is just part of your landscape, cloud or not. And BI-modal has a true risk of worsening the technical debt, the crud that builds up. Remember that “no free lunch” thing?
So, here is a message for the management of organizations that rely on IT a lot (and who doesn’t these days?): don’t think the mess won’t trip you up because it is invisible to you.
What you need is a permanent IT janitorial service with its own responsibility, its own budget (including budget to get the business side to move) and you need to make sure your business units can’t ignore the issue. This IT-janitorial service will be working all the time to clean up the inevitable mess, the technical debt, that comes with all the changes that happen all the time. If you don’t do that, your new initiatives will remain to be like trying to do heart surgery in a pigsty, like trying to run a Formula 1 car on a dirt road, and like trying to do clean room operations in a mud hut. You get the picture.
Pay attention to the new bandwagons, certainly. There is real gold there. Don’t be naive about them, though. But make sure you don’t try to jump on one laden with so much technical debt that the jump will maim you.
Discuss this or other opinions with me? I’ll be giving the Enterprise Architecture keynote at the Enterprise Architecture Conference Europe 2016.
This article appeared first on the blog Architecture for Real Enterprises on InfoWorld.
Hi Gerben — I couldn’t agree more about your assessment of the problem. But I was a bit surprised when I read “Incidentally, the invisibility issue of the business complexity that has resulted from the massive use of IT also hurts the awareness of the need for good Enterprise Architecture.” Surprised, because I believe this is far more than incidental. As I read through your piece I was sure that point about need for EA was going to be your bottom line.
For me, EA needs to be a distributed function, charged with coherency of all aspects of the enterprise. Not that EA is a janitorial function, because I suggest that the janitorial function is intrinsic to every initiative that changes the enterprise. EA needs to keep an eye on the development of the overall portfolio, assuring that the new acquisitions all make sense ni the big picture, and they are all performing their janitorial function as well.
Just one guy’s opinion!
Agreed, it is more than incidental. I meant ‘incidentally’ more as conversational in the sense ‘incidental in the context of talking about technical debt’, but maybe my being a non-native English speaker is showing here.
I think the janitorial function, important as it is in the broader view on matters (which is of course an important aspect of EA) is not an architectural function. It has not to do with the design of the Business-IT landscape (for me: EA) but with the maintenance of it.
For me, the crud is not so much that, say, after a merger the landscape has for instance multiple applications for the same thing. For me, the crud is all the little stuff. Small ad hoc batch jobs that have been set up to integrate things, systems that have not changed while they should have (but workarounds that have been created instead), interfaces that have not changed, etc., etc.. Very small stuff, but it has an impact if not cleaned up, just like real crud.
Great post. There’s a book to be written about the challenges of “medium-sized heterogeneous” IT, which I believe is where most IT is actually happening (these Eurostat figures: http://ec.europa.eu/eurostat/tgm/table.do?tab=table&init=1&language=en&pcode=tin00148&plugin=1 nearly confirm this, but frustratingly use 250 employees as the top size divider).
It’s where I work, but I often feel that we don’t have *so* much to learn from the dramatic “transformational” stories we hear about either new start-ups or technology-led giants. Our problems are far more mundane but no less awkward; however “Regional packaging company carefully and successfully retires ten undocumented but critical widgets without spending very much money, migrating wholesale to the cloud, or disrupting key business processes (except on the cutover Saturday)” doesn’t make great headlines.
Middle-sized IT architects unite: you have nothing to lose but your anonymity!
LikeLiked by 1 person
I love reading your blog because it more than halves my ranting budget. I particularly liked…
“So, Bi-Modal IT everyone? Well, yes, unless it is just a way to shut your eyes for the crud. In the end, that ‘other’ It is just part of your landscape, cloud or not. And BI-modal has a true risk of worsening the technical debt, the crud that builds up. Remember that “no free lunch” thing?”
What I am seeing in effect is a stampede from ‘mode 1’ to ‘mode 2’ by project sponsors.
Now we have major platform renewal projects running on two week ‘sprints’ with almost zero architecture, and very little documentation. Apart from unsightly platform build-up, (the technical debt issue you are writing about, and on which I heartily agree), I am also seeing an emerging scalability issue.
The Agile projects are spitting out MVPs that are generating enthusiasm, but are at serious risk of failure should any serious attempt be made to add functionality and or scale them out to the kind of capacity we are likely to need.
Interesting times 😉
You’ll probably like the other ones here too.
Yep, scalability is also an important issue. People start with laying the foundation for a cyclist bridge and they expect it to carry heavy lorries in the end. Technically, agile methods like scrum can handle this (the fundamental change in architecture becomes yet another action on the backlog), but the product-scope owner is seldom also the architecture-scope owner and I generally tell people that it is important to remember that project-deliverable-scope is not the same as project-architecture-scope is not the same as enterprise-architecture-scope.