After I joined Sitewards I began to discover that I am able to leverage a skillset and interest I have for better outcomes for our clients — specifically, the knowledge and passion I have around delivering and managing production software.
As I have matured at the organisation I find myself in a position of key influence when making decisions governing (software delivery) infrastructure, and with administrative access to various systems.
Part of the responsibility I inherit as a result of this privilege is to be a judge as to the capabilities and ideas of other people who also interact with the platform. Things such as:
- Who can have access to what resource
- What firewall rules should be added, and granted access to what resources
- Approval of general design patterns that will implemented against this infrastructure
Needless to say, I do not approve all requests. Indeed, I place the burden of proof on others to “prove” that the requirement to change the infrastructure is sufficiently justified. The goal for this process is for it to be democratised among several developers using infrastructure as code, and our standard code review — but for now, I essentially make arbitrary decisions.
This post documents some of the lessons that I have learned about how it’s a requirement to not only pass on a decision about the infrastructure, but also its justification and how to challenge the underlying logic of the decision.
From Wikipedia transparency is defined as follows:
Transparency, as used in science, engineering, business, the humanities and in other social contexts, is operating in such a way that it is easy for others to see what actions are performed. It has been defined simply as “the perceived quality of intentionally shared information from a sender”. Transparency implies openness, communication, and accountability.
An apt summary that can be easily framed against the making of decisions. When arbitrating some decision, transparency is about providing the justification behind the decision and a mechanism for appeal against the decision (or at least, future similar decisions). Generally this comes as a burden of proof that must be met prior to me shifting the decision that I have made.
There is a couple of semi-formal mechanisms for discussing infrastructure related changes:
- The infrastructure channel in Slack, and
- Directly setting a meeting with the administrators.
However, I regularly find myself making decisions outside these semi-formal loops, in a sort of “back channel”. It’s not intention, but rather a simple deference to the pragmatics associated with making these decisions. It is, however, something that I am deliberately trying to avoid, for reasons that will become apparent shortly. However, that being said it’ worth evaluating (non-exhaustively) how back-channelling occurs:
A requirement to make decisions quickly
Due to poor planning on either my part or someone else’s, it’s occasionally a requirement to have changes implemented quickly. Where change management has not been democratised, this means directly querying myself or another administrator who will make a decision arbitrarily, communicating only the result through centralised channels.
Trusting only a few with the responsibility to influence the decision
We are an organisation of quite a few developers, all of whom have different levels of experience. It is my judgement at this time that there are those with whom having a conversation about these decisions would be unproductive. While I do not actively exclude them, I do find myself seeking the council of specific colleagues, rather than attempting to involve all in decisions.
Selecting users who will likely endorse my judgements
As much as I try to be objective in my assessment of decisions, there are certainly colleagues with whom I find having conversations easier. Rather than looking specifically for an audience who will agree with my assessments, I will instead avoid others who will outright objects to my opinions without a burden of proof I decide.
The problem here being that I am both the judge and the arguer, and the biases I have will invariably corrupt the decision making process.
The problem is, the aforementioned back-channelling (or indeed, any non-justifiable decisions) tend to negatively affect my peers in a few particularly destructive ways.
Given the fundamentally creative work that is required by myself and my colleagues, as well as the arbitrary and somewhat opaque results ensuring that colleagues are engaged is fundamental to delivering good outcomes for our clients.
When given an arbitrary decision that a college has hand no hand in crafting and accordingly has no understanding of, it is simply a ruling that makes that colleague’s life unnecessarily difficult. By failing to communicate the underlying premise of a decision I succeed in communicating that I do not consider a colleague’s understanding important, and that their opinion is worth nothing to me.
Additionally, by providing no mechanism for appeal of further discussion I fail to give that colleague the opportunity for growth such that they can partake in the decision making process in future.
Dual sided stupidity
Decisions do not get made absent context. When making a decision I must consider several factors including time, maintenance overhead, skill and budget. Occasionally one of those factors (budget, for example) will have an outsized impact on the decision, leading to (apparent) sub-standard results.
When passing down a decision absent any additional context about how and why a decision was made, it can seem extremely stupid to those who must go and implement the decision. At best, those people will then simply refuse to implement the decision and demand more insight as to how this decision was made — short-cutting any time saved by back-channelling in the first place. At worst, they will simply implement that (and other) stupid decisions.
The team will end up in a situation in which those who are implementing decisions refuse to own the results and consider the decision makers, and those who make the decisions consider those who implement the decisions stupid.
Only a limited number of people are administrators, by very nature. Those people, myself included, are going to have a limited perspective of the whole problem. This, combined with back-channel decision making, means that I will make a fundamentally stupid decision, or appear to make a stupid decision.
Passing on this decision absent context and the right to appeal means that there is now a sub-standard outcome being implemented for our clients, and there is no mechanism to evaluate and improve this outcome now and in future.
Additionally, it establishes a disconnect between myself and my team, and I will not get feedback as to how to improve these decisions in future — my colleague simply accepting that I am both too stupid and in a position of power, and thus not “worth saving”.
Quick or Correct: Pick 1
There always exists pressure to quickly make decisions in a business context. This pressure is what drives back-channelling, which in turn creates the negative outcomes aforementioned. However, there are certain feedback mechanisms that we can put in place to limit the amount of damage caused by poor decision making circumstances.
Make decisions in a structured, transparent way
Perhaps the simplest solution: don’t back-channel. While the overhead of making decisions in a structured way can be large, it’s often necessarily so, and its cheaper to do things correctly in the first place rather than attempt to shortcut by querying people directly.
This skips the entire tree of problems defined as a result of skipping some users from the decision making process. In my case, I have a couple of simple rules:
- Attempt to make as many decisions in possible in the public slack channels, where all users can see decisions develop
- Where meetings are required, document the outcome of the meeting, as well as as much of context as required.
Routinely documenting the context as well as the decision
While practically speaking a decision is the required part of the communication that ostensibly influences the behaviour of my colleagues, routinely passing on the context allows them to further understand this decision and interpret it in a way that maximises both their personal happiness and the client outcomes.
This documentation must be part of all decisions, and the right to appeal universal. Practically speaking, the documentation I write often comes in the form of these posts, explaining the general pressures that operate around a decision. Other times it comes in Jira, and always in git commits.
Publishing an early draft of a decision
When making decisions, it’s often good simply to publish an early draft of the intended decision. It’s useful to generate the necessary heated conversation that will occur should that decision be controversial, and gives a chance to amend the decision or the context provided to adequately explain the proposed decision.
Additionally, simply having the conversations includes people, and will lead to a much greater ownership within the team.
Allowing a post decision analysis of the decision
Sometimes called “Arm-chair decision making” it’s important to welcome the occasionally uncomfortable post-event debate of whether a decision was correct. This allows colleagues to express their frustration at the outcomes of a decision, and we are able to work together to build a better framework for making these decisions together in future. Additionally, welcoming the post-decision analysis helps colleagues who have been sidelined by a decision feel that they will not be sidelined in future.
Models of successful decision making
A model of decision making that replicates the above issues extremely well is that of code review. Code review is a process of submitting a patch to a given project as well as the notes that justify the change and asking a colleague to approve the change.
Code review usually goes through several iterations before being merged into the main code base, of which much of the feedback is simply ‘I do not understand this’ or ‘can you explain that’. It additionally places the burden of proof on those who want to make the decision to prove it such that another colleague will approve the decision.
This approach is not limited to specific changes in code, but is also adopted regularly as part of standards drafting for various computer protocols. Titled an “RFC” (or request for comments) this consists of writing a draft documenting a decision, as well as the justification and risks thereof and submitting this decision to editors for review. Once suitable, several people will vote on the proposal, where it will either reach approval or not. This approach is popular to help guide the development of the computer language “PHP”
We are all forced to make decisions quickly, all of which will influence our team members in future. It’s critically important to provide the context around these decisions if not to include people in decisions as early as possible.
Expressing decisions in terms of code provides hope for a far more democratic decision making process, but may be impractical for non-technical implementations. However, having proposal/approve processes in other systems such as Jira or other decision recording engines modelled after the PHP RFC process may be one way to ensure that decisions will be more readily understood and accepted by a team.
- Antonius Koch, who reviewed this post and gave feedback on it’s structure and grammer