Quick Technical Decisions: Trade-off, Pay-off, Decision, Communication

May 9, 2023

A skill that a lot of engineers develop as they move towards a Senior level is the ability to make quick technical decisions verbally or in short form (say, a Slack post). Mastering this skill allows for triage: where do I need to pause and spend time working through or spiking the idea with peers, where can I just make the decision and keep moving under my own steam, and what should I set to one side for another time? Making a decision slowly is easy (or easier, depends on the decision obviously). You have space to develop your ideas and gather feedback from others. You can use a structured decision making process like a TDD to support you. But what about quick on the fly decisions where you don’t want to spend time working through the problem? One technique I’ve worked on a bit for this use case is a short mental checklist called “Trade-Off, Pay-Off, Decision, Communication”:

  • Trade-off: What are the downsides of the choices I’m making? How can I judge the value or the impact of these things that I am trading off? Do I have any datapoints to help estimate the size of the trade-off?
  • Pay-off: What are the upsides of the choice that I’m making? What’s the value or impact? How can I be sure that I’m right about that value or impact? Do I have a datapoint or evidence to back up what I’m saying?
  • Decision: How can I balance the tradeoffs and payoffs to get to a clear decision? How can I do a quick sanity check of my answer?
  • Communication: Who do I need to go and tell about the decision I’ve made? How should I communicate that decision?

This is glaringly obvious but hopefully it feels intuitive, easy to remember and simple to apply. The idea is to add just enough additional rigor to our thinking to avoid the obvious mistakes, a bit like Fermi estimation.

Here’s a worked example to develop the idea further:

I discovered a use case for AWS AppSync but I only thought of it after I’d already built the solution to my problem. I think it would fit how my squad work as we want to build product experiments quickly. Many of these experiments will be short lived with a few winners which will be developed further. What should I do next with this idea?

  • Trade-off: There’s a learning curve for others that haven’t used the technology before. This would be different to the rest of our architecture (i.e. not using a service template which is our default way of building on the backend), does this cause potential issues for long term maintainability? I have some uncertainty about AppSync - can I eject easily from AppSync at the point I want finer grained control?
  • Pay-off: It would take me an hour to set things up with AppSync, versus ~half day for scaffolding the lambda, api gateway etc. So I could make around a 3 hour saving in time up front. It fits the workload of the squad where we have few winners but lots of experiments which don’t get past an the A/B testing stage.
  • Decision: It’s worth it, especially for more throwaway use cases where speed of iteration is what matters, though we should use our level of confidence about the feature to choose carefully. As we all get up the learning curve with the technology our build time for scaffolding new services could shrink considerably and we could consider adjusting our standards to favour using AppSync.
  • Communication: I’ll talk to my squad first, perhaps bringing it up at the end of Tech Refinement. I’ll also run it by one of the Senior engineers outside of my squad asynchronously for feedback. If all goes well, I’ll put together a worked example and demo how our workflow could change in an engineering all hands. I’ll keep an eye out for upcoming work that might suit AppSync. I’ll stub a Notion page quickly (marking it as a draft) to show this is something I’m considering so others can read rather than listen to my reasoning.

Let’s dive a little deeper into why we chose each of these dimensions for our checklist.

Trade-off

We start by straw-manning our argument. We all have a strong confirmation bias for our ideas; we actively seek information that confirms our world view. Shifting perspective and spending a short time only thinking of reasons not do something counters this bias. We’re OK with being imprecise here as we’re going fast, so think of around 3 reasons and stop there. Getting past the first reason you think of is important though: forcing ourselves to dig that little bit deeper helps to combat availability bias. For each answer, test for assumptions: what evidence or data do you have?

Pay-off

Now flip to the other side, iron-manning (or iron-ladying if you will) your argument. Do this in the same way as the prior step, be confident and assertive in your decision - why is this a GREAT idea? Again, get around three answers and stop there. Test for assumptions and think of data points to back up your thinking.

Decision

Weigh the two perspectives and come to a clear decision. If you’re still uncertain then it’s time to put this to one side or slow down. Be careful here, great decision makers are typically humble and have a good feel for where the limits of their knowledge are. Good decision makers also update their decisions when they get new information, so if you’re in a meeting make a note to come back and sanity check your answer. If you’re crafting a Slack post or you’re in some other asynchronous channel, it’s time for a swift Google or to open up ChatGPT. Be careful to give any new information you discover at this step equal weight to your prior thinking (avoid anchoring). Don’t get stuck tunnelling to the centre of the earth either, we’re trying to move quickly here (if you struggle with this, set a timer to 5 minutes).

Communication

This reduces to who and how. Small groups (think around 5 people) make better decisions, so a key to good decision making is sharing those decisions and giving them space to be challenged (if you’re sceptical, a bit more detail on this can be found here). This doesn’t have to be in person, and personally I’d always default to at least sharing things in writing using tools like Slack and Notion. These tools give you a broader reach pretty much instantly at lower cost; even getting a short stub for your decision into a wiki like Notion aids maintainability and discoverability in the longer term.

How this can go wrong

Probably the chief failure mode is around the trade-offs. When I’ve watched people making technical decisions, it’s rare to see someone both advocating for a technical decision and failing to list all of the myriad reasons why their idea is great. Where I know I’ve gone wrong in the past is taking a step back and giving careful thought to how to challenge that decision. Equally, others may not challenge you (and you may be assuming that they will). Social dynamics are complicated afterall. This is why we do this step first in the process. Frontloading the critical stuff also gives us a quick exit if the ideas we’re coming up with are garbage - you’ve wasted less time and mental energy exploring a deadend. If you can verbalise and share this process with others (I had this idea but I decided that we shouldn’t go ahead because of x,y,z) it’ll also have positive cultural impacts over time (great leaders tend to admit and share mistakes as it helps others learn and gives permission to others to take risks).

Secondly, communication. Some folks will get their heads down and just plow through a bunch of decisions without talking to others first but this, in my experience at least, is rarer. A more common way to fail when making a technical decision is that the decision is never written down or it’s shared with only a narrow group of people (or both). Think carefully about breadth of visibility and which channel you’re using to communicate. You’ve now thought about the problem in a structured way, so hopefully that gives you confidence to share that decision with others. Equally, be respectful of interruptions - I’d suggest thinking through how the information can be there when folks look for it, rather than forcing a jolt out of context.

How to practice this technique

Right, you’ve got this far and you want to work on this skill. Here’s a few tips to get started:

  1. Get used to long form formats of technical decision making first (by which I mean TDDs/RFCs and ADRs). If you’re not using these formats where you work, then grab a template and get started. A common way to do this is to over-engineer your decision making (write up a long form of a technical decision for something that doesn’t need that level of introspection). We’re learning, so some waste as we practice is fine - there’s productivity and quality payoff at the end as you get comfortable with the skill.
  2. Favour asynchronous channels: you’ll have slightly more time to work through the loop and refine your thinking, even if it’s a faster paradigm like Slack, Teams etc. Once you’re a dab hand at driving decisions asynchronously, take courage from that process and start to try to lead group decision making verbally in smaller group settings. This can help for those low on confidence, or folks who are worried that their voice is not heard.
  3. Tell people that this is a skill that you are working on. Your tech lead, manager or a trusted peer might be people to talk to this about. You’re trying to improve, and this isn’t something to hide from others. In return they’ll be able to give advice, feedback and encouragement.
  4. Shadow the decisions of others: as you hear peers working through technical decisions, step back and use the steps - be analytical, break down the reasoning and try and get to where the decision is coming from. Listen only first, and then start to verbally challenge the decision and prompt using the checklist.