The ArchiMate 3.0 Relations Table…

…has serious, and many, errors. According to my personal analysis, 152 valid relations are missing, and potentially 2941 relations that are in that table shouldn’t be, if the text of the standard is to be believed or if we use some common sense. Here is why I believe that, and how I did calculate the table myself (code included).

The ArchiMate standard’s metamodel consists of:

1. A set of element types (e.g. Business Object, Node, Assessment, Capability) in a number of domains (Core, Motivation, Strategy, Implementation)
2. A set of relation types (e.g. Access, Serving, Realization, Assignment)
3. A core set of relations defined from element to element (e.g. Application Function — Accesses — Data Object, Technology Interface — Realizes — Requirement)
4. A set of derived relations (more about these below)
5. Relations to/from relations (as of ArchiMate 3)

We ignore 5 for now.

3 and 4 together fill the the table in appendix B of the standard. Formally, this table defines all the allowed relations between the elements in ArchiMate in an actual model. These relations are all are equally valid. Note: that there is a ‘core set of relations’ and a ‘core domain’ within ArchiMate may confuse people. These are fully separate concepts.

How the table of allowed relations is calculated is not open to the public, and that way can also not completely be inferred from the text of the standard. What is clear, though, is that there are errors in the table. This post is about an attempt to create a calculation, and an comparison of this alternative to what is actually in the official table. Running ahead of things, 152 relations (including core ones) turned out to be missing from ArchiMate’s official table and 2941 relations in the official table are suspect and  many clearly illegal.

The resulting Excel Spreadsheet and Python program I share under BSD license (so totally free), for all the world to see my feeble first steps at programming in Python :-). And VBA.

[UPDATED Feb 9/10: The Core Set Data in the spreadsheet and the Python program have been updated. Some relations were marked as illegal which shouldn’t and it turned out it was a combination of missing data and an algorithm error. It is important to remember that this is a work in progress and while it does fix several (serious) problems with the official existing table in the standard, it is by no means guaranteed to be perfect, if at all possible given the state of the standard.

NOTE: the distribution now contains a file Archi-relationships.xml which can be used as a stand-in for the existing relationships.xml to get at least many of the missing core set relations, such as Technology Function Accesses Artifact, back].

First things first: I’d like to thank Michael Herman for providing me with an Excel-format of Archi’s (ArchiMate 3.0) relationship table (that saved me a time-consuming job), and for providing me with a piece of VBA that got me started on how to program a VBA macro. I’d like to thank Jean-Baptiste Sarrodie for working with me on the initial attempt in Prolog, which we could not get to work at scale, after which I restarted in Python. The Prolog work laid groundwork for what followed, though.

Derived Relations

In ArchiMate, if you model something in an actual model of the enterprise using those relations in the table, you are allowed to create ‘derived relations’ in your model. E.g. if System Software is Assigned-To Technology Function and Technology Function Realizes Technology Service, then you may deduce that a derived relation exists from System Software to Technology Service. These derivation rules are (mostly) explained in Section 5.6 of the standard. There are two types of derivations-in-a-model that play a role in this story.

The first type of derivation is a derivation of ‘shortcuts’ using dependency and structural relations. A simple example is shown here:

Here, a Node Realizes a Technology Service which Serves an Application Function. The rule for deriving structural & dependency relations says that if you can create a chain of these relations in the same direction, you may deduce that between both ends of the chain, the weakest relation of the chain holds. In this example, the weakest relation is Serving, which means that (derived, in red) the Node Serves the Application Function.

Secondly, there is a type of derivation that handles derivations of dynamic relations in actual models. An example is shown here:

Here the rule says that you may move the start- or endpoint of a dynamic relation to the start of a structural relation (in the case of Trigger, only the Assignment) relation. Hence from the Flow from Process A to Process B we can derive a Flow from Process A to the performer of Process B, via the (structural) Assignment relation between the Business Role and Process B. Something likewise can be deduced for the Trigger. As Process B Triggers Process A we can deduce that the performer of Process B Triggers Process A.

The Table in the 3.0 Standard

ArchiMate states that not all derivations that follow from the rules are allowed in an actual model. If the rules lead to a derivation, this derivation is only allowed if the result of the derivation appears in the table of all relations in the appendix.

In addition to the model-type derivation rules above, another derivation is used that can not be used in actual models, but that holds in the metamodel only: derivation by inheritance. The metamodel contains elements that are Specialisations of other elements in the metamodel. E.g. Contract is a Specialisation of Business Object. Note that the standard is confusing, in that it contains two quite different Specialisation relations, one valid in models and one only valid in the metamodel and it calls  them the same: Specialisation. Anyway, taking our example from the metamodel:

One relation from the core set of relations is Business Process Accesses Business Object. Because Contract is a type of Business Object, we may, in the metamodel, conclude that (derived, in red) Business Process may also Access Contract. This is the red relation, above. Section 15.2 explains this for possible extensions of the language, but it is in fact also used in the existing metamodel: Collaborations are meta-model Specialisations, Contract is a meta-model Specialisation and Node has several Specialisations as children, such as Equipment, Device, System Software, etc.

The states in Section 5.6 that the table is populated by using the same derivation rules that are defined for deriving relations in a model on the metamodel. A Note is added that some special handling of Artifact is in place in the metamodel derivations that populate the table, to prevent unwanted derivations, but it isn’t defined exactly. The use of inherited relations in metamodel derivation is mentioned in Section 15.2. But even with that added, the content of the table cannot be entirely deduced from the text and diagrams in the standard in combination with what is said about derivation rules.

Now, our trust in the ArchiMate designers is of course great, but that doesn’t mean the table is perfect.

For instance, the 3.0 table contains Business Object Accesses Grouping, which means, a Business Object can Access a Grouping. Now, the standard says about deriving relations when Grouping is involved, that you’re simply not allowed to derive anything via Grouping and it also says “A grouping element may have any relationship with any element (provided that the element is a possible target element for the relationship).”. Ignoring the fact that it says target where it could possibly be target or source (as most relations have a direction), I can understand Grouping Accesses Business Object, because since a Business Object can be Accessed (passive, as object) so a Grouping must also be able to be Accessed too. But I can’t understand Business Object Accesses Grouping, because a Business Object cannot Access anything, thus, according to the provision in the text of the standard, also not a Grouping.

But it is part of the table nonetheless.

Another example of an interesting derivation in the table is Business Process — Realizes — Application Function. Since ArchiMate 3, the layers of the Core (‘the enterprise itself’) can mingle and the application layer can depend on the business layer and not only the other way around. But you can only go down in layers through the Serving relation, so any derivation should not result in a Realization relation, as Serving is weaker.

The 3.0 table contains several of such errors. Even some ‘foundational’  relations from the core set of relations are missing. E.g. Artifact Realizes Data Object is not there and neither is Technology Function Accesses Artifact. These are not just omissions, they are essential omissions. It’s like the Eiffel Tower missing some essential rivets at the base.

A New Calculation of the Table

It bugs me that “what is and what isn’t in the relationship table” has never been properly documented. Because the ‘lies-to-children‘ answer “the table defines the allowed relations, trust us” is woefully inadequate, and more or less is equivalent to the answer ‘Because!”. We’re mostly past that maturity level in our lives, I would expect. So, I’ve long thought of trying to create my own derivation implementation using what is stated in the standard and adding those few ‘extra’ rules to prevent chaos. Just as a thought experiment and as a contribution to the standard, to make it better. And, in the end I’ve done such a thing (‘better’ is of course debatable), consisting of a spreadsheet (with data and a VBA macro) and a Python program to calculate all the derivations. It was, I must admit, a lot more difficult than expected, mainly because — here and there — the standard is unclear, ambiguous, or incomplete.

First, I’ve created a spreadsheet with all the relations from the ‘core set’ as far as I could gather from the text and diagrams of the standard. I put in a few fixes and indicated several potential others. The sheet is called “MA Core Set”. The cells that are (or could be) adapted are marked in yellow (you can play with this yourself). Some remarks:

• According to the text, motivation elements cannot compose themselves. That seems to be an omission in the text to me (and the table does contain them), so I’ve added them to the spreadsheet anyway.
• According to the text, Material cannot Realize a Business Object. That seems to be an omission to me. I’ve not added it to the spreadsheet, so my result stays as close as possible to the original. But I think it should be there.
• I think Nodes, Business Roles and Application Components should be able to influence a Business Role. I’ve marked these cells in the spreadsheet but (again) left them out of the calculation input.
• Trigger and flow to/from services are missing in the diagrams in the standard. But with the new derivation rules of ArchiMate 3, this means that services cannot flow/trigger anymore because these cannot be derived anymore. As they are in the table, I’ve added them to the spreadsheet as core set relations instead.
• I’ve marked the cells (but did not change them) that would allow events to trigger up-layer. Triggering down-layer is often already allowed because of the new ‘upward’ Realizations between layers and the new derivation rule for dynamic relations that travel ‘backwards’ along those Realizations.
• I think business roles (not just actors) should be able to influence stakeholders and business roles should be able to influence themselves. I’ve marked these cells as well, but I’ve not added these relations to my core set.
• Junctions, Grouping, Location, and Plateau (core set) relations are not fully administrated in the spreadsheet, they are calculated by the Python program. There is a slight difference if you put some of it in, though, as the program generally treats input relations as always OK, even if they are forbidden by the rules implemented in the program.

The spreadsheet also contains a table of all the relations in the official table (sheet “ArchiMate 3.0 Full Set”. This data has been taken from Archi 4.0.

The VBA macro in the spreadsheet takes such a table and produces columns on the left of the sheet. It also selects the data that you can then copy/paste into a file for the Python program. Note: watch out for end-of-line trouble.

So, taking the “MA Core Set” sheet with the core set of relations, running the VBA macro in that sheet, and using the result in the left-most column as input for the Python program (via copy/paste), then comparing the output with the result of that same VBA macro run in the sheet with the Archi relations (“ArchiMate 3.0 Full Set”), gives us a difference between what is in the table in Appendix B of the standard and what we calculate from our core set and our derivation rules. Note, the format I use is the format of Prolog clauses, because the first (failed) attempt at calculation was done in Prolog. The output can be switched to a CSV format (see below). But as we want to compare with another input set, I generally still use the Prolog format for writing down relation tuples.

A full command run looks like this:

• Create the core set from the direct relations sheet, then copy it into a file, say CoreSet.prolog (note, I name it .prolog because that is the format). Then do the same for the full set that can be created from the Archi sheet (say Archi-all.prolog)
• Then run:

./Derivation.py CoreSet.prolog --outfile Derivation.out --logfile Derivation.log --compare Archi-all.prolog

• That takes CoreSet.prolog and creates Derivation.out and Derivation.log. The file Derivation.out is the file containing the calculated new set. The file Derivation.log contains progress information that can be trawled to find out why certain relations were allowed or forbidden. To make life easy, the program has a –compare flag. You can give it as argument a file with a full set, like the one extracted from the “ArchiMate 3.0 Full Set” sheet. With that flag, it will compare the calculated result against that file (e.g. Archi-all.prolog), and produce two more files: one called Archi-all.prolog.missingincompare and one called Archi-all.prolog.missinginresult. The first file contains all relations that were in Archi-all.prolog, but not in our result (e.g. Derivation.out). These should all be wanted relations that are missing in the original table. The second file contains relations that were in the compare table but not in our calculation. Those relations should be ‘wrong’ or ‘unwanted’.

The result is rather surprising. Thousands of differences in the total set of relations. For one, the standard has 11,761 of them, and I have about 9,561. And those include 155 that the standard has not. The 155 relations that are not in the standard I consider potentially missing in the standard’s table, and the 2364 relations not in my result I consider potentially illegal relations (mostly derivations) in the standard’s table. That number 2364 sounds like a lot, but many are repeats of one single problem or another.

The question of course is: whose result is best? Let’s first explain a bit more of the rules I followed when calculating.

The Rules

Most of the rules I use are of course taken from the text of the standard where it explains these rules. Before I document my adaptations and choices, first have a look at An AchiMate 3 Map (Layers? What Layers! — 1) where you can find a ‘map’ of ArchiMate into:

• Strategy (top layer)
• The Enterprise Itself (middle layer, itself again layered), aka ‘Core ArchiMate’
• Implementation (bottom) layer)
• Motivation (column)

I use the domains mentioned here for defining (the limitations of) cross-domain derivation rules.

Then, we say:

• Metamodel-specialisations inherit the relationsof their parent (implied by the standard in Section 15.2)
• Within a domain (layer, column) we allow derivations according to the rules in Section 5.6
• Between all layers and the Motivation column, we only allow Realization and Influence as derivation result. Other relations must be given explicitly in the core set of relations (input) to be allowed. Again, this is as described in 5.6 (though that description is not very precise, e.g. it does not explicitly cover relations between implementation or strategy on the one hand and motivation on the other — it only mentions the core).

And we also limit as follows (this is not per the standard, but these are my ‘hidden rules’):

• No Serving to passive elements (makes only convoluted sense in the case of Artifact)
• No Access from passive elements (makes only convoluted sense in the case of Artifact). Note: read Access between behaviour and passive technically runs from behaviour to passive, so remains allowed.
• No Assignment is allowed if an Assignment in the opposite direction already exists. Note: This rule is not followed for collections and junctions.
• For the collection elements Grouping and Plateau, follow B.1 as much as possible. See the Python code for additional details.
• We do not allow derivations via a collection type where source or ultimate target are not collection types. This way we prevent the ‘permissibility’ of the collection types to spread too far.
• I limit derivations via Work Package. Business Role to Work Package is a relation from the core set. So, it is not derived and that implies it can be used in derivations itself.
• We do not allow derivations that include the networking elements. The networking elements can aggregate other technology and that leads to lots of relations from them that are rather useless/meaningless.
• We do not allow model-specialization of the metamodel_specialized children of Node to Node itself.
• We do not allow model-specialisation of hardware technology to software and vice-versa (e.g. Factory cannot be a Specialization of System Software)
• Product is defined in the business layer chapter but it is shown as a Collection element (though it (and Plateau) are missing when collection elements are introduced earlier). So, does it count as a business Layer element or not? This has consequences for the derivation rules.
• For Junctions (connector elements):
  • As prescribed in the standard: only allow relations to/from a Junction that fit with an existing ‘junctionable relation” from/to a source/target
  • Between Junctions: we drop all other relations but ‘junctionable relations’, which means we also exclude self-composition, self-aggregation, and self-model-specialisation for the non-element types that the connector-types are.
• An element cannot be derived to be assigned to itself, unless it is about deployment:
  • System Software to System Software
  • Facilty to Facility
  • Device to Device
    Note: I think ArchiMate should allow deployment of an Application Component on System Software.
• If the first step is Product Aggregates X, then there may be no further derivation using what the Artifact Realizes as active element. This rule is necessary now that Product also Aggregates passive elements from the lower core sub-layers. It fits Section 5.6 in that Product is not in any domain (but a collection).
• A derivation that starts with a network element and Aggregation may not extend beyond the technology sub-layer
• I consider Location as a generic stand-in of sorts for elements that can be Aggregated by it. Hence, I allow Location Aggregates Business Actor Assigned-To (core set) Stakeholder to become Location Assigned-To Stakeholder, even if they are in different ‘domains’ technically and should thus according to section 5.6 be limited to Influence and Realization. (Actually, the collection types aren’t really a domain as shown in the map above.) The table in the standard has many of these relations. I suspect these are calculation/core set leftovers from ArchiMate 2, when Location was part of the core). I like them, so as not to prune them, I added this exception for Location.

For all the details, see the Python code and its documentation. It is very complex to make sure the whole business doesn’t explode.

Some details from the Results

Now about the results, and comparing it to what is actually in the table of the standard.

The 152 ‘missing’ relations in the official table:

• A lot of Access relations, most to Artifact. Some are even relations from the core set, such as Technology Function Accesses Artifact. These are clearly (serious) errors in the table of the standard.
• Some missing Aggregations, mostly from Location, that should be allowed
• Some Assignments, some that clearly should be there, such as Business Actor Assigned To Business Collaboration. Also something seemingly weird like Business Role Assigned-To (instead of Composes) Business Interface is a legal derivation. It comes from Business Role Composes Business Collaboration Aggregates Business Actor Assigned To Business Role Composes Business Interface.
• Influence relations from Implementation to Motivation are missing
• Quite a few Realizations are missing, apparently because Artifact Realizes Data Object from the core set is missing. Also core relations such as Technology Function Realizes Application Function are missing. These are mentioned in 12.2 (and in 12.1 for application to business), but they are not in the diagrams (and hence probably forgotten). Here the standard is actually incoherent (and it is that in more places). I’ve followed the text, not the diagram. All the yellow cells in the spreadsheet are about (potential) issues in the set of core relations. Read the comments in the spreadsheet on those cells.
• The (core set) Trigger from Implementation Event to Implementation Event is missing.

Some examples from the 2941 ‘wrong’ relations in the official table:

• A lot of illegal relations with Junctions and Composite elements. For instance, according to the standard, you’re allowed to have a certain relation between an element X and a Grouping, if there is such a relation with element X and any other element. E.g. because Technology Function can Access an Artifact, a Grouping can be Accessed by a Technology Function (because Artifacts can be grouped) and it can Access an Artifact (because ‘Artifact-accessing’ elements like Technology Function can be grouped) . (In the case of Junctions, only the ‘junctionable’ relation types are allowed in this derivation). The official table is full of relations that break these rules. E.g. Grouping Accesses Business Role. There is no way to get with an Access to a Business Role, so Grouping too should not be able to do that.
• I block some relations while the standard doesn’t. E.g. Equipment Assigned-To Business Actor can be derived if it is not blocked in some way. It is a silly relation, I think. You can derive it like this: Equipment Composes Facility (somewhat silly already) Assigned-To Business Actor. It’s a good example of ArchiMate 3 having received logical new core relations that play havoc when being used in derivation without limitations, especially in the case of a new ‘overload’: Assignment being used for yet another meaning (next to fulfils, performs, deployed-on) .
• My Location (as per standard) only Aggregates stuff, it doesn’t Compose stuff. The table in the standard has results that include the composition. The same for Product.
• The table contains derived relations such as Artifact Flows to Application Function or Application Interaction Flows to Artifact. These are blocked by my rules.
• In the table, a Business Function can Flow to an Application or Technology Event, but there is no derivation route that makes that possible.
• In the table, Grouping can Specialize anything and anything can Specialize Grouping. That seems to be an unwanted too permissive consequence of Section B.1.
• Contract Realizes Capability. A Contract can Realize a Resource (core set) and Resource can be Assigned-To Capability. Hence, Contract Realizes Capability. But allowing this breaks the rule in 5.6 about Core to Strategy (and others). There are many other strange Realizations. Work Package Realizes Meaning. Etc.
• I’ve noticed Trigger derivations via Realization, while the text allows this only via Assignment.

With the over 2300 relations not-derived or blocked by my code (for mostly, I think, understandable reasons) I could go on longer. You can play with it and take your pick. Note: I do not state that 100% of the relations I block or do not derive should not be in the table, but my feeling is, for a large majority this might be a good choice. Some of these are a matter of more arbitrary choices and taste. The difference: my choices are public :-). And given that I release my code under BSD license, it’s easy for people or organisations to adopt it and adapt it. TOG is free to use my code to create a publicly understandable relationship table.

Conclusion

The derivation mechanism in ArchiMate has always been a bit of a naive dream at ‘student paper’ level, and in reality it is not that very important in real models, neither for producing ‘automatic viewpoints’ (which tool can do this?) or for analysis. It’s far more effective in ArchiMate to choose good modelling patterns and then analyse a model regardless of the derivation rules. They are not that important in daily practice.

What  remains is the role of derivations in filling that all-important table, the table that for some ‘defines ArchiMate’ even and that you need to follow if you want to pass your certification.

With regard to that role: ArchiMate 3’s relationship table’s construction is not fully explained and it contains thousands of relations that are questionable, many of them actually provably wrong from ArchiMate’s own definitions. Even some relations from the core set itself are missing. Frankly, the table is a mess, and in my opinion not trustworthy as an arbiter of what is allowed.

Note that — while I am critical of that mess — this story should not be interpreted as being negative on ArchiMate’s practicality. Ultimately ArchiMate remains a very practical instrument. It shares with humans “being practical regardless of not being extremely logical”. Given that enterprises and their behaviour are very human constructs, this conclusion should not surprise us (for a language/grammar that is supposed to be able to describe the reality of enterprises). And that (grammatical) rules and logic are a poor foundation for meaning, we alreade knew from Uncle Ludwig, so the fact that a metamodel full of circularities and simple derivation rules do not simply give only useful results should not surprise us either.

I can’t add the files directly here as WordPress doesn’t allow me to add a zip file, so currently the only way to get them is via the free ‘tool support bundle’ of Mastering ArchiMate, which can be found via the book’s home page. Note, for fixes and changes (within the philosophy of the existing approach), please let me know so I can incorporate it in my release. See the about page for contacting me.

SP: Don’t forget to read Agile & Architecture (on combining architecture governance with agile development methodologies) on the Enterprise Chess site (SP = shameless plug :-))