Open Letter to TOG #7

Dear TOG,

I hope this letter finds you well.

As shepherd of the ArchiMate standard, you are currently working on its next iteration. As I am not part of the ArchiMate Forum, I am going to send you a few open letters with suggestions for improvement of the language and what to do an not to do. I will be voicing some stern/harsh criticisms on ArchiMate in these letters, please don’t take it personal, I’m only interested in strengthening the language which I already find useful as it is. And I apologise beforehand for my language here and there.

This is the sixth of such a letter. The previous ones: sixth, fifth, fourth, third, second, and first.

Today, I’d like to talk to you about the Composition and Aggregation relations and how they play a role in ArchiMate’s derivation mechanism.

Warning: if you are new to or not very deep into ArchiMate: This is a very long and convoluted analysis. Don’t let it scare you away from a rather practical language. Think of it as the equivalent of you listening in on a discussion between car designer-engineers discussing the limits of the car, while all you need is to be able to operate the car.

What initially really struck me as very nifty when I first looked at ArchiMate, around 2007, was the derivation mechanism (if you are not TOG and new to ArchiMate and you’re not very familiar with derivation, download the Free Syntax Excerpt of Mastering ArchiMate from the book’s home page and read the explanation therein first). Derivation, in ArchiMate, promises that you can model in detail, but that it is possible to automatically derive relations between various parts of the modelled landscape based on the relations in between. As such it promises an easy way to combine both detail and oversight in a single coherent model, which is very attractive.

I should have known, however, that this doesn’t work all that well in practice. ArchiMate is a grammar more than a language and creating meaningful summaries by a simple mathematical procedure is not possible in any real world situation. Both precision and recall are affected: there are both meaningless conclusions that can be drawn that way as well as meaningful ones that cannot. The standard has leaned to prevent above all meaningless ones, as a result, many meaningful ones cannot be derived.

I’m going to start with a specific example and then look at derivations.

Here, I have modelled two Application Components, each with subcomponents and four Infrastructure Services and how they are related. Note: the Used-By relations here are already derived, but this does not matter for the argument. I could have used Application Functions instead. But I though this was easier to comprehend by people who are new to ArchiMate.

Anyway, what you see for instance above is that the Reporting part of the Trading & Reporting application uses a file share, but the Trading part does not. And, for instance, the Reporting Engine Infrastructure Service is used by both application’s Reporting subcomponent.

I’ve added Aggregations (as I normally do) to group the Infrastructure Services that are needed for an application to run. Such an Aggregation has a couple of advantages (e.g. it is a clear point to attach SLA requirements for the infrastructure people). Nesting the Aggregations and Compositions results in this:

Which makes it more pleasant to read, but some of the relations are now visually hidden. I’m now going to work with a fragment of this. In the fragment, the black Used-By is the one originally modelled.

What about the other Used-By’s drawn? I can argue that they are all true, but you cannot derive them in ArchiMate from the original black one (the blue can be derived from the red one and the orange one from the blue one, though). That has to do with the direction of Aggregation and Composition. In ArchiMate, a derivation is only allowed if the direction of all the relations in the ‘route’ are the same. And the direction of Aggregation and Composition goes from parent to child. The direction of Used-By is the same as what the arrow tells you. In current ArchiMate, the direction of the Aggregations and Compositions run opposite to the Used-By, so you can’t derive anything form the black Used-By. You can check this in the initial diagram.

Which brings us to the following questions: What does it actually mean for the children if some element has a relationship with their parent? What does it mean for the parent for some element to have a relationship with its children?

ArchiMate has taken the Composition and Aggregation relations from UML. But UML has been designed for software engineering and its idea of Composition and Aggregation has much to do with pointers, memory allocation and deallocation (see below). One can (should) wonder, though, how appropriate their wholesale adoption is in an enterprise architecture setting.

To have a look at the meaning of relations with children or their parents and how these are related (or can be derived), I am going to introduce a generic non-IT example: a driver stops his car at a rest area where he can get gas and buy food. I will be using ArchiMate relations here, but note this is not valid ArchiMate, as ArchiMate has no real place for the physical world (yet). It is possible to build such a model with IT too (within a single layer in ArchiMate), but the argument is clearer using something more familiar.

In our example, we have a car with an owner/driver. The simplified car has parts: an engine and a set of wheels, of which one wheel is the spare wheel. The rest stop has a gast station and a market, both owned by different proprietors. To make matters interesting: it is the market that owns the tyre inflating apparatus. Now, the engine needs gas and the wheels need air and the driver needs food. Nested, it looks like this:

And we can unnest this to get:

In using Composition and Aggregation I have made a couple of choices, which are of course debatable (otherwise, as Uncle Ludwig explained, they would not be choices, would they?). E.g., I consider both wheels and engine to be inseparable from the car. This is not from all possible perspective so, e.g. they can be removed or swapped. But here I do subscribe to the definition that a car without engine and wheels is not a car as you cannot use it as one. In natural language, the meaning of any word, including the word ‘car’, is not that strictly defined. But bear with me: in our setting: it is not a car if it does not have an engine or wheels. I have made the Spare Wheel something that can be removed from the car without this action rendering the car a non-car, though.

Now, the owner is going in for full service: he fills the car up with gas and air and himself with food.

If we follow ArchiMate’s derivation rules from the bottom up, everything seems fine at first. Up to Engine and Wheels of the car. The engine uses the gas pump, the wheels use the air pump (note: I’m using the ‘use’ word freely here. Again: this is not important for the argument).

But wait: Starting from Food via Owner, we can, according to ArchiMate derivation rules, deduce that the engine and the wheels are using Food. And we cannot deduce that the Car is using Gas (or Air). We cannot solve that problem by connecting the Used-By from Gas to Car instead of Engine, as this would result in the Wheels using Gas. Hmm. There is trouble in the land of ArchiMate. The simple derivation rules are not by definition producing sound results.

Now ArchiMate says: if you have a Used-By (or any structural relation with the same direction) with a parent of a Composition (or Aggregation), you have it with all its children. But that is plain silly. It is as if you say: if you model a Used-By relation between File Sharing and the Trading & Reporting application, it is therefore true that both Trading and Reporting use File Sharing. But thát, you cannot deduce in the real world. You cannot even say, either of them does, as there might be a third subcomponent that wasn’t modelled. There is, after all, no ArchiMate modelling law that says a model is valid if and only if all elements are modeled.

What you want to be able to deduce is in fact the other way around: if a child uses something, so does the Composite parent. If the Engine uses Gas, it means that the Car uses Gas because the Engine is part of the Car. For that, the direction of Composition would have to be reversed.

Hence, solution #1 is to change the direction of Composition and Aggregation for derivation. There is still a parent-child dependency, but for AchiMate’s derivation, we change the direction.

This solves the problem we have in the upper half of the view, but it creates the same problem in the lower half. Hmm…

Solution #2 is to model it slightly differently:

You might not immediately spot the difference. I have reversed the upper Compositions. This is technically not allowed in ArchiMate because you are technically not allowed to have multiple parents of a single child in a Composition setting in ArchiMate. Composition in ArchiMate takes its restricted use from software engineering, something I have argued above would be nice if it was relaxed, because the world of enterprise architecture is not exactly like the world of software engineering. And, as I mentioned above, above all in the application software engineering of the 70’s and 80’s, ‘true sharing’ was an important problem (memory leaks and crashes from dangling pointers were the most common source of application bugs in those days). Interestingly, modern software engineering handles this in a way that makes the old UML-design-safety-feature less important, but I digress.

Anyway, what about Aggregation as used in a decent ‘grouping’ element in some way or another? Suppose ArchiMate would get (as I have proposed elsewhere) a generic Group element, instead of the horribly confused Grouping ‘relation’ of today, what would that mean? Take for instance the proposal I created for Capability? Here is the version from the book:

How would derivation have to work out in that case?

Can we derive relations from the shown Realisation from Capability to Product? Using the existing derivation mechanism, only Realisation from Capability to the Aggregated children of Product. Or — if the direction of Aggregation would change — Realisation, from the Aggregate children of Capability to Product.

We might extend the derivation mechanism for this kind of grouping from the ‘generic’ element type (of which then the Group element that I have proposed elsewhere is the root and Capability and Product are Specialisations). E.g. we could say: an Application Component that is an Aggregate child of Capability may be Assigned to an Application Service that is an aggregate part of the Product that is Realised by that Capability. But it would break down immediately if we had multiple Application Components in the one and multiple Application Services in the other. After all, we may not conclude from the Realisation of Product by a Capability that all Application Components that are part of the Capability are Assigned to all Application Services that are part of the Product.

The easy cop-out for this conundrum is to use Association as the derived relation, but frankly, Association is probably true between any element of your landscape and as such means next to nothing. Quoting a well known book about ArchiMate: “Association is for wimps”.

So, the conclusion could be, that we might only create derivations from relations between the children of the different Groups. In which case the rule would be: any relation that exists between children of different Groups is also valid as a relation between the Groups themselves. So, if a Business Process that is part of Capability above has a Realisation relation to a Business Service that is part of Product, then there is a Realisation relation between that Capability and that Product. And if we have the (derived) Assignment relation between the Actor in the Capability and the Business Service in the Product, we have an Assignment between Capability and Product. All strictly one-way derivations: from relations between the children of Groups to relations between the Groups themselves. Not the other way around. Relations with Group elements would by definition have to be derived.

Tooling could help us here, by the way. With a derivation rule like this, all relations are allowed between Groups, but they must be based on what happens between their internals. E.g., suppose we would create an Access relation between an Application Function that is part of one Group and a Data Object that is part of another, our tool might give us a little exclamation mark if that relation was not supported by an underlying one in the model.

[Update 5 Feb 2016: The following section was updated after I read comments made by Richard Jarvis in the LinkedIn ArchiMate Group. The first examples originally used Realisation and not Used-By. It turns out these two behave differently with regard to what you want or do not want to be able to derive.]

There is, however another approach that might work. First, have a look at the following diagram:

 

We start with the blue relation. What  can we derive from that one? The green one, we can derive following current derivation rules and we like that. We cannot derive the orange one, and we like that too. But we cannot derive the violet ones, and these we would like to have. After all, if the engine uses gas, the car uses gas.

Now have a look at this one:

 

Again, the blue one is the original relation. With the current derivation, we can derive the green one and that is OK. We can also derive the red ones, but we do not want those. If a Function uses a Service, we know that some part of the function uses the Service, but not which part. If an Application Function uses a file share and has two sub-functions, we do not know if one of these uses the file share at all, it  might be another unmodelled subfunction. We draw too broad a conclusion if we do it the current way. In the example above. Suppose in reality Child Function 2 uses the Child Service, we also conclude that Child Function uses the Child Service with current derivation rule, but it doesn’t. Its a false conclusion.

[Update until here. The original example was with Realisation, but Realisation has a bit of a different semantics in your EA models. It kind of looks like a Composition itself. Which is not strange, after all, I’ve suggested in an earlier letter that Realisation between Functions/Processes and Services should be replaced with Compositions, because the behaviour of a Service is part of the Function. There is more going on in the background, but my guess is that a better derivation mechanism than the current one is definitely possible.]

This leads to the following derivation rule for Composition (we will talk about Aggregation below): You may move the endpoint of a relation to an element that is the child of another element to its parent. This then is independent from the direction. It resembles what we have for derivation for the dynamic relations, which is also about end points (this solution is thanks to Jean-Baptiste Sarrodie, who put me on this trail by asking me what I thought about derivations in my Capability proposal, and then in the following exchange suggested the use of something akin to what was done with derived dynamic relations: moving endpoints — honour where honour is due: it is a very smart and elegant proposal).

Basic examples:

[Update 5/Feb/2016: note the remark about Realisation above. I should have done this with a UsedBy instead. The proposed new mechanism might not work for all structural relations in ArchiMate 2, more needs to be adapted for it to work.]

Here, I can derive the blue Realisation from the black one because the starting point of the black Realisation can move along the Composition to the Parent Function.

Here, I can derive the blue Used-By from the black one because the end point of the Used-By can move along the Composition to the Parent Function. As you can see, this is independent from the direction of the Realisation and Used-By relations, which are opposite in these examples, but our smart derivation rule works perfectly for both cases.

Then, what is left is the question of what to do with the Aggregation relation.

Question: do we allow the orange Used-By as a derivation of the black one? I have been thinking about this one and my conclusion is yes. The main reason is that — though the existence of the children here is independent from the parent — we might say that that if one child of a collection has a certain relation, then the collection has that relation. Think of it as Venn-diagrams:

If we have an algorithm that uses multiplication, it can be said that it uses a commutative function (which is a function where you can safely swap arguments, as in a times b is by definition the same as b times a). We can also safely say that the Algorithm is using a function in general. But we cannot say it is using Division, which is a non-commutative function (a divided by b is not the same as b divided by a). And this is true, even if we have collections that are aggregations instead of compositions. Venn-diagrams don’t say anything about this, you can use them for both if you think about it (it has to do with the dynamics of the sets, not any particular point in time), but for a better feel:

If I give away my Electric Screwdriver to Jean-Baptiste, the Electric Screwdriver still exists, it is just not part of the ‘My Tools’ collection anymore.

The other reason is that having Composition and Aggregation behave the same in derivation is simpler. And we like that, don’t we? Enterprise architecture modelling is complex enough as it is.

There is a last effect I need to discuss. What if we have Composition and Aggregation (e.g. a Parking Lot Aggregating Cars which Composite Engines)? I would say that any series of Aggregations and Compositions can be replaced by an Aggregation and any series of Composition by a Composition. That part at least then will be backwards compatible. Always important for people who love ‘backwards’ ;-).

So, that is it. A very long story and I’m afraid convoluted analyses like these will scare away potential new users from a rather practical language, but the conclusion is that we might improve matters if we change the derivation rules concerning Composition and Aggregation to: You may move the endpoint of a relation to or from an element to the Aggregate or Composite parent of that element. You may replace a series of Compositions and Aggregations by an Aggregation. You may replace a series of Compositions with a Composition.

Yours truly,

Gerben Wierda
Former Lead Architect of APG Asset Management and of the Dutch Judiciary
Team Coordinator Architecture & Design at the All Pension Group (APG)
Author of Mastering ArchiMate and Chess and the Art of Enterprise Architecture