Low-hanging Cloud

(Sorry, long story. It will return in an appendix in Chess and the Art of Enterprise Architecture. It’s not related to that book’s main message, it is just an ‘illustration’ used in the book, which I thought would be nice for a post)

Often, when changes in the Business-IT landscape are required, organizations are confronted with a series of results to achieve, all with a different effort required to reach the intended result. It is common practice to look for `quick wins’ that can be achieved first. An often heard phrase for such easy picking is low-hanging fruit. From various professional perspectives, this approach is circumspect. For instance, agile development methods state that it is important to tackle the most difficult use cases for a solution first.

After all, basing your foundational design decisions on the easiest use case is a recipe for failure. Chances are, that a bridge designed to get cyclists across needs to be completely redone when the more difficult twenty ton lorries have to be serviced. A bridge designed for heavy lorries, on the other hand, will probably carry the cyclists with ease. In enterprise architecture, this is true as well. Design and develop the easy stuff (such as the business process’s ‘happy flow’) first, and you will probably end up in serious trouble when the difficult work flows — such as all the exceptions (which as a rule are always there) — need to be supported as well.

What is Cloud Computing?

What does actually change when you move stuff `in the cloud’? Interestingly enough, much stays exactly as it was before. Let me explain. When you look at a business that is supported by IT, you can use the standard `layering’ that most enterprise architects employ. This layering says that you have a business layer — consisting of processes, roles, actors, et cetera — that sits on top of an application layer, where we find the applications that are used by the business, which in turn sits on top of an infrastructure layer: the servers, operating systems, networks, and other foundational IT elements that are needed to actually run those applications. For instance, a sales manager (business) may use a contact relations management (CRM) application (application), running on his desktop (infrastructure) to store information on his clients. In larger setups, the desktop may be a data center, but by and large this is how the Business-IT relation can be modeled. So, what happens with this setup when this moves to the cloud? The answer is: basically nothing. There still is a business layer (our sales manager with his processes) using an application, which in turn depends on the infrastructure. The ownership of those layers changes, though. If the sales manager uses a cloud-based solution, e.g. he uses an application that runs `in the cloud’, there still is that infrastructure and there still is the application, the only difference is that the application does not run on his own infrastructure anymore. The only thing he still needs is a little bit of infrastructure of himself (an internet link, a browser, maybe a plugin such as Java) to access that remotely running application. And he would have needed that infrastructure too if the application had run locally on an application server in the company’s data center.

If nothing changes, architecturally, why then is cloud computing growing so fast? Something must change, or we would not be discussing this. Well, something does change, and it is not just ownership.

Economy of scale and of flexibility

The reason for the growth of cloud computing is that it brings an economy of scale and an economy of flexibility. The economy of scale is for the provider. If you take the example of the CRM application that runs `in the cloud’, and you look at the architecture of the provider of that application, he has basically a very simple landscape. After all, his infrastructure does not need to support many different incompatible systems with in- compatible architectures, he just has a single architecture to maintain, only he has lots of it. For the user of the CRM application, this has the advantage that he does not have to support the complex infrastructure required for this application (application servers, database servers, system support programs, monitoring, logging, auditing, virus protection and other security systems, the list is rather long). If you step over legal issues (such as data protection acts that are national, while the cloud is global, or regulators who try to set rules and necessarily expand them to the service providers you use — or it would be very simple to escape the pressures of regulation) the solution is ideal.

The economy of flexibility is for the client of the cloud-provider. If you provide a web site for your customers, it is very busy between 8:00 and 22:00; you may need many load-balanced servers to handle the traffic. And during peak shopping season, your web shop will be hit very often, on a monday morning in Februari, traffic will be slow. But as a web shop, you need to provide enough capacity for peak traffic. The rest of the year, that infrastructure and the web applications running on that infrastructure are doing nothing. So, what you really would like is to scale your deployment based on actual demand. Busy time? Switch a few extra servers and the applications running on them online. Quiet time? Switch them off, and not so much switch them off, don’t pay for hem while they are switched off. Virtualization has made this possible, and this is what cloud deployment offers.

These economies are driving adoption of the cloud. Everybody wants to push their IT deployment in the cloud (public, private of hybrid) because they do not want to pay for IT running idle. That will work well, until this cloud system is part of a complex landscape of connected applications. If your accounting system is in one cloud and the inventory system is in another, connecting them will be slightly problematic. This is so because the cloud-application provider’s economy of scale requires a homogenous internal landscape: not too much diversity in the landscape, or the whole setup becomes again so complex that the cost becomes prohibitive. So, providing connections to all kinds of different specific environments with different requirements is out. The standard solution rules.

A second scenario where cloud works well is where the software deployed is actually capable to automatically make use of scaled up or scaled down deployment. Just moving your existing software to the cloud doesn’t really help, it will only be more expensive. Cloud providers must make a living too, and if you push a server to the cloud and it has to be there all the time (no scaling up and down), you will pay much more than running it yourself. Maybe even two to three times as much.

Using cloud-based applications is very attractive for simple landscapes. Are you a consulting company, and your enterprise architecture is more or less restricted to an office suite, a CRM application and a basic accounting system, the cloud is all you need. Dependencies between the three are minimal and each can be easily provided by the cloud providers. This is probably even true for large consulting firms, who may cope with large but simple architectures. But what if you are an organization with a large and unavoidably (because of business demands) complex IT landscape, with a myriad of dependencies and many mission-critical custom applications? Application silos in the cloud (SaaS) will have integration issues. Pushing your infrastructure into the cloud won’t help you unless your landscape consists of cloud-aware applications. If not, moving stuff to the cloud infrastructure makes your landscape only much more expensive.

Many supply-demand boundaries

One of the underestimated consequences of the cloud is that your architectural dependencies suddenly cross multiple owners. This means that making coherent, landscape-wide, architectural decisions — in other words: enterprise architecture — has become more difficult. This was already the case with using different systems from different providers in that you had to live with various ‘canned architectures’, now suddenly you get a situation that even something as simple as life cycle management is even less under your control. An example may make that clear: if cloud-application X is updated to a new version that requires version Y of Java on your desktop, and cloud-application Z cannot work with that version of Java but the provider of cloud-application Z has no plans yet to update, you are stuck. And given the nature of cloud-applications, you cannot postpone updating application X. And this is no fantasy. Already providers like Microsoft and Apple force updates of infrastructure plugins like Flash and Java, for security reasons. What happens if that cloud-application is incompatible with that new version?

The above is a simplified example and cloud providers are already reacting, for instance by providing multiple versions of their offerings in a cloud (which somewhat negates the cloud provider’s advantage). But except in the most simple of enterprise architectures, systems are more than just support for a human that performs a human process. Cloud-based systems for risk management for asset managers, for instance, are already the norm. That is more than just using a browser to access a user interface. Such systems need to be fed with real data from the actual investments and the data and reports they produce must often be fed back into the organization’s own systems, such as data warehouses. There, ‘cloud’ does not only mean ‘lower infrastructure cost’, but it also means ‘more complex connections and dependencies’ and so forth.

Architectural Fog

So, while cloud is going to make it easier and cheaper for the most simple of enterprise architectures, it’s going to produce new requirements and extra complexity in large enterprise landscapes, extra complexity that is also more difficult to manage because of the demand-supply boundary that is part of cloud. Cloud, too, will not be a silver bullet. Cloud will undoubtedly become an important aspect of the landscape, but the fracturing it promises will probably also produce many headaches in the years to come.(see note on the right)

Back to the low-hanging fruit: there are already several cloud-based services, from infrastructure to applications. These figure prominently in brave new worlds, in which businesses expect the cloud is going to enable them to outsource much of their IT operations, or at least make it much simpler. But that seems to be based either on the cloud’s obvious advantage for simple enterprise architectures, projected on much more complex landscapes, or on ignoring the fact that cloud deployment (IaaS) requires cloud-aware applications to bring any advantage. The cloud is seen as low-hanging fruit, the available services must be profited from, without paying enough attention to the more complex issues. If you have only limited visibility, in this case because only short distance effects are taken into account, you may safely equate quickly picking the low-hanging fruit in the cloud with driving with a high speed into architectural fog.

PS. Speaking of fog and its role in representing danger, remember the Challenger disaster?

[If you want to discuss my views with me: I will be speaking at the Gartner EA Summit 2015 London UK on May 20 and the MBT-Congress in Houten NL (closing keynote) on May 21]