How well does your top management understand IT?

Made from the oil of the Chinese water snake, which is rich in the omega-3 acids that help reduce inflammation, snake oil in its original form was effective, especially when used to treat arthritis and bursitis.

Organisations are hierarchical beasts. While — the ‘how much’ depending on the culture you’re from — everyone gets a say, decisions are made by those responsible often in name of those that are accountable. And the latter may overrule, even as they will — as a rule — not often do so. Hence, in the end, when arguments about decisions get escalated, or when decisions are strategic in nature, they end up being made ‘at the top’ and then there is an interesting question: how much does ‘the top’ understand what they are deciding about, and does that really matter? Quite certainly, they will not have all the knowledge themselves, they can’t know everything about everything.

Summary: I am going to propose a way to 'measure' the 'understanding' the top of an organisation has — how capable it is of making informed strategic decisions — on a subject, e.g. legal, finance, or what is my particular interest: IT.

This is for instance why large corporations often have someone with the title “Chief Legal Counsel”. This is someone (with a small staff) who makes sure in-depth know how regarding the legal field is available for the Board of Directors. They rely on it for legal opinions. It is their ‘trusted advisor’ on legal matters. It is also why they might have a Chief Technology Officer, or a Chief Information Officer in or attached to the Board of Directors.

IT Quackery at the Strategy Level

Board members are management and leadership ‘specialists’, but they generally do come from an original non-management background. After all, the organisation is not ‘management all the way down’ and they had to start somewhere. Often in finance, or operations, or sales, or marketing, or in policy creation. Look at the board of a large hospital and you will find management with a thorough background in the medical sciences. Can you imagine the board of a large hospital making sound decisions when there is not a single person with an actual medical background in the top 3 levels? I cannot, but in IT this is more common than you might expect. Can you imagine a board of a hospital never discussing medical issues? I cannot, but in IT again this is common. Can you imagine a board of a large organisation never discussing a legal tangle (based on what the Chief Legal Counsel has prepared)? I cannot, but in IT again this is normal.

Extremely rarely, we might find a board member with a true IT background. If they actually have worked in complex IT it will generally be long ago, before their rise in management began. Current Board of Director members, if they do have actual experience, often have experience from the 90’s maybe, and given the rapid changes in the information revolution, that experience is probably somewhat out of date. After that, it was mostly a management career. And this is of course good: management too is a skill and finding a combination of both management skills and IT-understanding is rare (though Jeff Bezos seems to have had it both and that has greatly contributed to his success). Not all top managers are as extreme as Jeff Bezos. Luckily, I think, given the way his profit-drive also undermines a decent society (Amazon workers in Bessemer and other locations said they were forced to work 10-hour shifts with just two 15-minute bathroom breaks), as Adam Smith already in the 18th century warned us about. But I digress, as usual.

Anyway, the situation becomes risky when the feeling for the subject matter at hand becomes so weak that the difference between good and poor advice cannot even be determined by top management. Then, either the high level managers need to blindly trust their advisors or the risk increases that poor decisions — especially without a decent insight in what is actually possible — will be taken. So it then becomes very important to select your advisors properly.

Edsger W. Dijkstra, one of the founders of IT, said this about it in 1982:

The wide-spread existence of pulp science has to be stressed because, containing as few scientists as they do, many bureaucracies […] are no longer able to identify pulp science as such. And this has more serious consequences than the fact that some people are willing to pay [a lot] for a two-day course that others instantaneously identify as bullshit.

      Nowhere in computing the quack density is as high as in the broad area of design methodology; needless to say, all the miracle ointments are at the very least “proven” or “tested”. With that abundance of wonderful remedies the conscientious scientist, who refuses to offer a cure-all, may have a hard time convincing his administration or the funding agency that his efforts make sense. Whether he likes it or not, the language of the quack is almost forced upon him when that has become the only language that the authorities still understand. Slowly the distinction between the scientist and the salesman erodes; but once wide-spread acceptance has been allowed to become a goal, it becomes a quality criterion, and the world of science has become infested with the standards of the bestseller society. And those discourage any deviation from the status quo so strongly that for many the road to innovation and improvement is effectively blocked.

      Under these circumstances the scientist has to remember that there is no virtue in pandering to the tastes of The Great Unwashed and he has to learn to take it as a compliment when he is being “accused” of being not enough of a salesman.

Uncle Edsger’s own rigorous, rational, uncompromising (quoting Donald Knuth) approach to IT helped IT from a world of pure ‘alchemy’ into a semblance of ‘chemistry’, and he is rightly honoured for that, even if his own strict mathematical ideas of solving the problem may never have been practical — they just did not scale to the volume of machine logic employed in society today (or even then).

Made from the oil of the Chinese water snake, which is rich in the omega-3 acids that help reduce inflammation, snake oil in its original form was effective, especially when used to treat arthritis and bursitis.

But Uncle Edsger’s warning against ‘quackery’ in IT has stood the test of time well, in the same way that Frederick Brook’s 1986 warning “There are No Silver Bullets” (which even has its own Wikipedia page) has stood the test of time. And that may sound great, but the fact that these warnings are as relevant today as they were almost four decennia ago is bad news. It means that the situation hasn’t improved much. There is still a lot of quackery and alchemy in IT, and the more high level it gets — like in Enterprise Architecture or in IT Strategy — the more of such quackery and alchemy we encounter, often sold as ‘best practice’ to unwitting management. ‘Best practice, by the way,’ is high on my list of misused terms in IT, if only because many things labeled ‘best practice’ are in reality often not even practice, let alone ‘best’.

An abundance of quackery can only exist in a vacuum of true knowledge. Legal Counsel of large corporations is generally of a very high quality. A legal ‘quack’ has little chance of establishing themselves as the ‘trusted advisor’ of higher management, there is too much actual knowledge around. In Enterprise IT, Enterprise Architecture, IT Strategy, etc. this is different. Alchemy and hype are rife and they are listened to and taken seriously by people who do not have the background or the inclination to be able to discriminate between quackery and real medicine. As Uncle Edsger already observed, the most successful strategic IT consultants seem to be those with good sales skills, not necessarily good IT skills. This is illustrated by the fact that much ‘medicine’ has been sold for decennia without ever having a real proven ‘therapeutic’ effect for the organisations that took it to heart. Most organisations wrestle with the same kind of problems around IT as they did thirty years ago. And they still often are sold and believe the same sort of quackery and hype. Note: sometimes behind the quackery lies real value: snake oil actually was a decent medicine for the time as it turns out, until fake snake oil tarnished the reputation of the stuff. There is a lesson here: does your ‘medicine’ dealer understand what they are selling you? Real confidence tricksters are rare in IT, I think, but naive consultants and solution sellers there are plenty. Uncle Edsger was not wrong in his assessment. And Hanlon’s Razor — “never attribute to malice that which is adequately explained by stupidity” — holds.

Measuring IT (and other subject matter) knowledge at the strategy level

Back to the top management of large organisations. Human society is becoming more digital by the year. The IT-revolution is having profound effects on society. E.g. Boards are being told they are or are becoming ‘digital organisations‘. But then it stops. There is a vague idea that being a ‘digital organisation’ has to do with having ‘apps’ and using those to interact with customers. But that is only a very small aspect of it and taken in isolation, or done based on unrealistic notions of what IT can do may do you more bad than good. Boards generally have no idea. Because in or close to those boards we do not find a lot of actual IT-background that can help them weed out the nonsense.

There is a lot of IT know how in modern organisations. But decent knowledge about IT just seems to be unable to reach the top. And the result is that decisions are made without much realism in terms of what is or is not possible in IT.

I have been thinking of a way to measure this knowledge. And I’ve come up with a formula, firmly tongue-in-cheek, I must add. And of course, from here in the article we run the chance of saying goodbye to those that assume that any show of actual mathematical symbols means that there is for them no useful content. I invite those to read on, because in words, the formula I came up with is much less daunting. Here is the formula:

A formula to measure the understanding (K) of the top of an organisation

What this formula does is the following: We want to calculate how much knowledge in a certain subject is in play at the highest, strategic level of the organisation. It thus starts from the common orientation that there are levels (L) in the organisation and Level 0 is the CEO. The next Level is everybody who directly advises the CEO. These are the other board members, but also for instance that Chief Legal Counsel we used as an example before or an IT Board that directly advises the CEO. And so we go on in levels. The main formula for calculating the know how of the organisation at any level n is:

Formula for knowledge at level n

This consists of three parts:

This part just sums all the know how of the employees at that particular level. We’ll get to what constitutes as know how below. At level 0 there is just one employee, the CEO. If that CEO has a background in legal and we are measuring the strategic legal know how of the organisation, the CEO directly adds to it. But if the CEO has a background in finance and has no legal know how whatsoever, this term is 0.

This part is the know how of the level below the current level (in organisations, generally the top is level 0 and we can extend downward, so a ‘level 2’ manager is in effect a manager two levels down). The knowledge of the level below is added, but it is divided by a ‘passthrough’ value. Because indirect knowledge is less effective than direct knowledge as it has to pass through another layer. We use the 3 for rp, this is a personal estimate.

This part is a factor that diminishes the effect because of the ‘distance to the top’. This is the applicability of the knowledge. The further we get away from the top, the less directly applicable (to the issues the top wrestles with) the knowledge is assumed to be. At Level 2, the value is 3 to the power of rl. We use the value 2 for rl, this is a personal estimate. At Level 0 this factor is 1, so no effect. At level 2, the value then becomes 9 — from (1+2)2.

Where ‘c’ is mentioned it stands for a cutoff, the level where we end taking effects into account. So, K(0,2) stands for a measurement of the knowledge at level 0, based on taking level 0 to 2 into account. Anyway, you start at the bottom of your layer set and work yourself up to level 0.

It is easiest to give an example. Our example organisation’s ‘top’ consist of 1 CEO, 4 other Board of Director members, 6 ‘Level-2’ unit managers and an ‘advisory board’ for our subject that consists of 6 members. At level 3 we find another 100 employees (and potentially thousands below that). The outcome of the formula is the number highlighted in red and bold. The percentage is a transformation of the end result of the formula (not explained here, it is complex enough as it is).

LevelNr of EmpAvg. KTotal KK(L,3)
Other BoD members, advisors to the CEO140.251.000.29
Direct reports and advisors of BoD (e.g. BU managers, CIO, CTO)2120.253.000.45
Direct reports and helpers of level 231000.5050.003.13
Example organisation 1, Subject like IT with a lot of know how lower in the organisation

This first example is an organisation with a CEO who has no background in the subject, where one of the board members has a strong background (again: we will define background below). At level 2 we have 3 out of 12 with a strong background (or 6 with a medium background) and below that we have 100 employees where there is a strong background, e.g. think the people working in an IT department when the subject is IT.

To illustrate what happens when the lowest level we take into account has no knowledge at all, look at this one:

LevelNr of EmpAvg. KTotal KK(L,3)
Other BoD members, advisors to the CEO140.251.000.28
Direct reports and advisors of BoD (e.g. BU managers, CIO, CTO)2120.253.000.34
Direct reports and helpers of level 231000.011.000.06
Example organisation 2, Subject like legal with very little know how lower in the organisation

You can see that changing the knowledge level at the lowest level (that we still take into account) has little effect at the top. What knowledge there is, is not as applicable to the strategy issues (rl) and it has to travel through many layers (rp) to get to the strategy level.

Now let’s see what happens when this organisation adds a special advisor to the CEO with a staff of 3. Level 1 gets one fully qualified member added to it and level 2 gets 3 fully qualified members.

LevelNr of EmpAvg. KTotal KK(L,3)
Other BoD members, advisors to the CEO150.402.000.58
Direct reports and advisors of BoD (e.g. BU managers, CIO, CTO)2150.609.001.00
Direct reports and helpers of level 231000.011.000.06
Example organisation 3, added advisor like a ‘chief legal counsel’ (with staff) to the CEO

This has a huge effect. The organisation becomes three times as savvy in the subject from a strategic perspective. This kind of shows why we do find these Chief Legal Counsels with their staff in many large organisations.

Let’s take an extreme as our next example. We show an organisation with a lot of IT where the CEO has no knowledge, the direct level below doesn’t have that background either. E.g. the IT advisory board consists of managers and they have operations, finance, and HR backgrounds — but not real IT. We’ll define ‘background’ below.

LevelNr of EmpAvg. KTotal KK(L,3)
Other BoD members, advisors to the CEO140.000.000.01
Direct reports and advisors of BoD (e.g. BU managers, CIO, CTO)2120.000.000.12
Direct reports and helpers of level 231000.5050.003.13
Example organisation 4, no background for a subject on the first three levels

This organisation will probably spend an outsized amount on external consultants because they will be very uncertain about the subject. And they have no way to gauge the quality of those consultants, be they quacks or real doctors. The quality of strategic IT-heavy decisions becomes a matter of luck. And when bad decisions encounter hard realities in layers below, they may fail, and day-to-day, the people understanding IT at lower levels have to scramble to make unrealistic decisions work. Much of what the board says may be ignored. All in all, not a healthy situation.

As a final extreme, let’s take the opposite: an organisation that consists solely of IT-savvy people:

LevelNr of EmpAvg. KTotal KK(L,3)
Other BoD members, advisors to the CEO141.004.001.13
Direct reports and advisors of BoD (e.g. BU managers, CIO, CTO)2121.0012.001.56
Direct reports and helpers of level 231001.00100.006.25
Example organisation 5: IT-savvy people only

This, I think, is unhealthy too. This organisation is so solely focused on IT that it will fail because in other areas it will resemble the previous organisation. An organisation skewed like this will probably fail miserably.

What is ‘a background in‘?

We can see that it stands or falls with the assumptions being reasonable and definitions being useful. E.g. in the example above, rl is 2 and rp is 3. These numbers come from my personal estimation, in other words: I made them up. But then again, so were Porter’s five forces. So this, of course, is ‘quackery’ of my own. It is as much quackery as the advise of any lightweight consultant, it would only become real medicine after a rigorous scientific study. I’m not calling this ‘best practice’. Still, it is better than nothing I think.

One key element above all needs a bit of thought: what constitutes ‘a background in X’? Because even without these numbers, the fact that you need IT know how in the top will become more and more accepted. But what constitutes for instance ‘IT know how’ in this context?

Well… to have a background in legal, you need to hold legal opinions, right? To have a background in medicine, you need to have worked with actual patients and their actual health issues and have a well-founded medical opinion, correct? OK, then, to have a background in Information Technology, you need to have worked in the technology. You need to have learned from the inevitable mistakes and other failures. To have a background in IT you must have experience in the (often difficult) decisions and choices that are made when IT is created, operated and provided, and have a well-founded technology opinion that is not just parroting superficial ‘quackery’. And to make matters worse: the higher education setting in IT does not provide you that experience, just like the mostly theoretical nature of the basis-medical higher education doesn’t provide you with ‘doctoring skills’. Those, you get ‘in the real practice’. In IT, higher education is necessarily based on ‘toy problems’ done by a few people, and the result is thus ‘toy knowledge’ for small scale ‘worlds’.

So, for me, ‘a background in X’ means ‘actually having produced X’ in a realistic setting. So, if your ‘background in IT’ consists of having been HR-manager in an IT organisation, you don’t have a background in IT, you have a background in HR. If you have done psychological research into the thought processes of IT designers*), you have background in psychology, not a background in IT. If you have managed IT, you have a background in management, not a background in IT . If you have done. If you have done strategic consultancy in IT, you have a background in strategic consultancy — a worthwhile field of its own with methods like scenario planning and such — but you don’t have a background in IT. If you have a background in international relations and you’ve done one summer school course in IT**) , you have a very, very, itsy, teeny, tiny background in IT. You get the picture.

You can of course have a background in IT if you have worked in the psychology, HR, management or strategy of it. Being a manager or a strategy consultant doesn’t mean you are incapable of understanding IT, but it doesn’t come from your HR, management, psychology, etc. work. Again: if you haven’t been a substantial part of the thick of it, of the difficult discussions that led to actual IT choices in a complex reality, you have no background in IT as far as I’m concerned.

With that definition in place, the method above gives a way to express the amount of how IT-savvy the top or your organisation is. Identify the first 3—4 levels (0 = CEO, 1 and 2, maybe 3), make a list of all people at those levels and talk to them or look at their CVs. How strong are their IT credentials? Would they be able to truly understand a technological design discussion in a complex environment, given the time? Are they capable of recognising what Uncle Edsger calls ‘bullshit’ in IT? And then calculate K(0,2) or K(0,3).

A ‘hard measurement’ like this might help in convincing organisations like that of Example 3 that they have a problem in the form of an ‘unknown unknown‘.

*) resembles an actual example of someone stating to have a strong background in software engineering…
**) resembles an actual example of someone stating to have a background in IT…


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: