Thomas Betts: Hello, and welcome to another episode of The InfoQ Podcast. I’m Thomas Betts, lead editor for architecture and design at InfoQ and a software architect at Blackbaud. Today, I’m joined by Andrew Harmel-Law for discussion about a conversational approach to software architecture. Andrew is a technology principal at ThoughtWorks. His focus is on helping software teams and the organizations they work within to deliver valuable outcomes in the most efficient way. Andrew is also a trainer for domain-driven design on the O’Reilly training platform. Andrew, welcome to The InfoQ Podcast.
Andrew Harmel-Law: Thanks for having me, Thomas.
Architecture conversations [01:00]
Thomas Betts: You recently wrote an article. It’s posted on martinfowler.com. We’ll have a link on the show notes, entitled Scaling the Practice of Architecture, Conversationally. It’s been shared around InfoQ and within my company. I can say personally, that it’s led to some great discussions about how to be an effective architect and how to structure an architecture organization for success when you need to scale out with many, many development teams.
We’re probably all familiar with the idea of the Ivory Tower architects sending down “thou shalt” directives, and this kind of goes against that idea. Can you describe to our listeners your ideas about having conversations about architectural designs and decisions rather than a monologue coming from the architect?
Andrew Harmel-Law: The first thing to point out is the conversations part is completely taken from Ruth Malan. Ruth Malan talks a lot about this, and I came across Ruth through the DDD community. She’s got some stuff… I’m not sure if it’s still on the internet because I think she took some stuff down because she let me know that I was linking links that were dead links, but she had this whole alternative take on architecture. One of the key points behind it was that architecture is a conversation. Therefore, there’s the outputs and the outcomes, which are decisions and designs and all these kind of things, but one of the key things is the kind of the conversation and the curation of that conversation and the point in time in which that conversation takes place and all of these other things. That really struck a chord with me, and specifically that struck a chord with me because as you mentioned, the nature of architecture is changing because the nature of the way we deliver software is changing, the nature of how we structure our teams is changing, because of the way that cloud-based platforms let us do things.
All of these different things and all of these phrases like autonomy and microservices and distribution and with COVID distributed teams and all of this kind of stuff is happening, things are getting split up into smaller pieces and the practice of architecture has always been important, but in my opinion, as you split up into more smaller pieces, getting those divisions as good as possible at the time you do it is even more important. More small pieces, more decisions about splits and stuff and the decisions as to what you do in those pieces. I spent my life avoiding being an Ivory Tower architect, but as the person responsible for the architecture, however it gets delivered, I was feeling more and more stressed and under pressure to figure out how to scale myself to be in all the right places, to be in all these conversations, to give people the advice. DDD helped a lot in around about figuring out different domains and map domains to teams and aggregates to microservices and all these kind of things, but it was still hard to be in all the right places all the right times.
The thing that coalesced it for me is there’s a quote from Alberto Brandolini, which is something like, “The thing that goes into production is the developer’s assumption about the domain.” That’s the same thing, right? The architecture that goes into production is not your beautiful diagram or the thing that you thought you’d explained to them is what they took from that. That started keeping me awake at night, right? I was like, “Oh, my God. How do I know that this stuff… Is anything I do relevant?” Then, I started looking around for something and then I came across all this stuff that became this article. Mainly, it was me scratching my own itch and helped myself sleep at night. Then, over time I put bits together to make it work. That’s where we got to.
Anyone can make an architectural decision [03:58]
Thomas Betts: The way you wrote about is that you get to the idea of the advice process, which I love. It’s nice and simple. It has one rule that anyone can make an architectural decision, but there’s a qualifier. Can you explain those to our listeners?
Andrew Harmel-Law: Again, this isn’t mine, it’s just bits I’ve glued together, so the advice process comes from trust organizations. There’s a book by Fred Laloux where he talks about it, and he did loads of examples, but did lots of case studies. He noticed that in trust organizations, decision making is decentralized or completely spread around. People are empowered and trusted to make decisions, but then lots of the companies that do this, and one of the examples is, I think it’s a Pakistani nuclear company. They run nuclear reactors and they use the advice process to make decisions. Obviously, just anyone can make decisions is a bit risky, especially if you run nuclear reactors. They have the process. Like you said, the rule is anybody can make any decision that they need to, but the caveats are they first need to seek advice from the people who will be impacted and the people who have expertise.
That then leads on to the fact you need to identify who’s impacted and you need to identify who has expertise. That alone makes whoever’s taking the decision or needs the decision to be made, they have to think about what is the impact, who do I need to speak to, who has expertise, et cetera, et cetera. They need to start engaging with the problem and that when used for software architecture turns out to be a nice kind of ground level framework for just getting people to do at least a bare minimum of things that you typically do as an architect without really thinking about it.
The architect as an advice facilitator [05:25]
Thomas Betts: You talked about having all these little teams separated out. One of the benefits of that is you get very focused and good at only needing to understand your little view of the world, but as part of a microservices architecture, you’re not standing alone. You’re part of a big system. You’re part of this big architecture. Those people impacted, you might not be aware of. Is that where the architect who jumps around and sees the bigger picture is involved and can help find and identify those people that are impacted and get the people experience to help make the decisions?
Andrew Harmel-Law: 100%. This is the conversations piece because how, in my experience, it turns out is the architect is still playing the role, but you’re more facilitating and you’re curating these conversations, right? To begin with, you’re helping people realize that they probably need to speak to this person, this person, this person, and then this person, and you direct them off and ask them to go and have the conversations. You can also seed that with questions and stuff. Instead of deciding things, you can ask the questions and make people engage with those things. Therefore, maybe the decision making is spread out, but the architectural thinking still stays, not to a great extent, but it doesn’t disappear from the person who’s a traditional architect. But maybe they’re curating. Like I said, to tackle the Alberto Brandolini problem, which is like, “How do you know…” It’s not just in your head and you’re telling people. You’re engaged in the architectural and the decision-making process with everybody who’s impacted. Therefore, the chances of there being fewer mistakes and fewer misconceptions of what the decision is because it’s coming from a different place, right? The people who are leading the decision process are more… Their skin is in the game and they can engage more fully and completely with it.
Use ADRs to document why a decision was made [06:53]
Thomas Betts: A lot of the developers I work with and that I enjoy working with that have the same mindset is, “Don’t just tell me what to do, but I want to understand a little bit of why we’re doing this.” Sometimes those why decisions are the architectural decisions. Why are we using this framework? Why are we using this platform? Why are we using these patterns? If you engage the people in the conversation and some of that why sinks in a little bit more rather than, “I was just told to do this.” Is that what you’re seeing too?
Andrew Harmel-Law: 100%. This is to bring in one of the pieces that go with it. It’s why ADRs are really nice. The reason I’ve started bringing ADRs is because I’d used ADRs before, but ADRs give awesome structure to things and I’ve experimented with various standard templates for ADRs. But the nice templates for ADRs are… It’s the point in times that it makes it clear when this happened. It makes you think about the context of what’s the context of this decision that’s happening. What are the options you considered and what are the consequences of that decision? Again, that’s a standard kind of thinking framework, which I use as an architect. If you give that template to someone, then they can think about the same things, right? What are the consequences? What is the context? What are the options that we can consider?
Lots of the time, bringing them back to the advice process. You’ll get people who begin to do this and they’ll pre-form an opinion. They’ll go off and ask the people who they know will give them the opinion that they want to hear or the advice they want to hear. What we end up doing, and again, it’s conversation curating. You’re like, “Right. But who should you speak to? Who will disagree with you?” Because the advice process, you don’t need to agree with them. You’re not looking for consensus, but you are looking to hear their viewpoint and hear their opinions because it’s advice, right? They’re not telling you what to do. You get to decide. But if you seek out people with different opinions and people with broader perspectives or longer historical perspectives, then it becomes more of a learning process as well as you get good decision making, but also the person making the decision is being exposed to in the context of their problem that they need a solution for. They’re being exposed to all this stuff, which when we take decisions outside of a team or maybe with a team, but it’s still the decision lies with us as the architect, the engagement is lower, but now the engagement is high, right? Because they’re like, “Oh, I didn’t realize.”
Sometimes they change their minds and sometimes they don’t, but it’s based on the context that they find themselves in. Then, you get better ownership and you get… The delivery of the thing is far more likely to happen and it’s faster and it’s not going to be done begrudgingly. There’s loads of knock on side effects, but it comes from the trusting, right? The reason you build the trust is because people you trust care more about stuff, right?
Thomas Betts: Right. You’re not going to do this, simply be because it gives you better developers or happier developers, but happier developers and people who are more engaged are going to write better code, are going to think more about it. They’re going to come back and ask and is like, “I might not need to do a full ADR on this, but I can at least go and talk to those people again and say, ‘Hey, I just want to get your opinion.'” Little point to go back on, acronym police. ADR, architecture-
Andrew Harmel-Law: Architectural decision record. Yeah. The classic Michael Nygard stuff, they’ve taken off in like there’s a million different varieties, but yeah, the ones I go back to, there’s just the standard Michael Nygard stuff. Again, I didn’t invent them. Completely someone else’s thing, but they’re just nice and they’re small. They’re not an onerous thing. You’re not saying, “Please update this gigantic software architecture document or blah, blah, blah.” It’s just a thinking tool, which by, as a side effect, gives you this beautiful kind of historical view of your architectural decisions, but that comes from the people who are writing the software. That’s awesome.
Because they’re written down, then the other bit that we’ve added to the advice process or I’ve added to advice processes is advice process says seek advice. In this, what we do is we say, “Right. Your responsibility is to write down that advice.” You don’t need to agree with it. You don’t even need to follow it, but you do need to write it down.
When you look back at an ADR six months later and you’re like, “Right. This decision was bad.” Did we know that it was bad? Did we just ignore it? I’ve come back to that again. Again, it’s happened four or five times, which as a consultant is nice because then the client can see, “Did we think about it? We didn’t think about it.” There’s a nice kind of trail of the context and the conversation as opposed to just, “We picked this. We picked Log4J as opposed to something else.” That’s nice, so it feels like it’s a consumable architecture as opposed to just like a list of facts.
The right size of an ADR [10:47]
Thomas Betts: It’s not just boxes and arrows. You don’t know why you drew those boxes this way or those arrows that way. Where do you think ADRs fall on the heavy to lightweight scale of how much work goes into them? Is this something that it takes a half hour, I just dash it off? Is it done over a period of days or weeks while I think about and muddle on a decision? Does it vary based on how big that decision is?
Andrew Harmel-Law: It totally depends. The idea is they’re smaller, but they can live for quite a long time, so it’s nice. With the rule of thumb I’ve found out and I think this is in the article. Typically, if someone’s running a spike, it’ll probably turn in to an ADR. Unless the spike goes, this was a disaster. We’re not doing that. Or if someone’s picking a new tech and they’ll probably write the context and they’ll probably brain dump the kind of options that they’re going to look at. Even then, so we then expose that as draft. The current client, we’ve got it set up so that they’re done as JIRA tickets, which some people will probably recoil in terror, but it’s nice because we can then listen to it and people can see it. We’ve got it on dashboard. The fact that that’s been created as draft, people get pinged, they can see the title.
We spend a lot of time or I spend a lot of time helping people curate the titles, because it shouldn’t be like something woolly and it should state the decision. Right? It shouldn’t be like, “If you don’t want to read the whole ADR, I can just read the title.” It tells us what the decision is. That will trigger people. Lots of the people who are affected or who have an opinion, they’ll come out the woodwork really fast, but also other things I learn and I learned this over my career and I’m still learning it, but people who are new to decision taking will take what they think is a decision. It’s actually seven decisions all crossed into one. When they write it, if it’s too hard to write or it’s too big, before they’ve even formed the opinion of what it is in their heads, when they do that, they’re like, “Oh, this is too big.”
Then, again, if a draft is published or even if it’s like in progress, me and my fellow architects, my client and previous clients can go, “Okay. Cool. You’re trying to decide three things at once.” Actually, one of those is an NFR or CFR as we call them in ThoughtWorks. Right. There’s an implicit CFR, which we aren’t aware of, so we need to pull that out. There’s also maybe some design principles which are implicit or whatever. The sooner people can get that thinking out loud down, then you can start responding to it. Then, the conversations are helping themselves curate themselves. The answer is it depends. But, I’m a consultant, so that’s kind of part of the territory, right?
Thomas Betts: The answer is always it depends. It’s an architecture discussion.
Andrew Harmel-Law: Exactly, right? Like I say, the key thing is if you think of everything, because secretly again, it’s a learning tool. Everything’s a learning tool. Getting people to share where they’re at and then all of this stuff, you’re like, “Oh, this is awesome opportunity to teach and to share knowledge and to get people to grok this stuff.”
Making cross-functional decisions [13:09]
Thomas Betts: Since you mentioned the CFR is cross-functional requirements or non-functional requirements, that seems like one of things you wouldn’t want to put down at… One individual team is deciding that. That comes back, but you have a way of identifying.
Andrew Harmel-Law: Yeah. The things that typically come back are tech strategy. That does sit with some architectural level or from higher up enterprise architecture or CTO in our case, but what’s nice is it comes down and then it’s like we try and make that embedded into the ADR. If an ADR is looking like it’s going to go contrary, why? Maybe that’s right. Maybe the strategy is right, but for this instance, it’s wrong. Ditto for principles. We source the principles with the team. But again, that’s a facilitated conversation. Then, we use the ADR process to see if are the principal’s standing up or our principal’s getting a bit out of date and do we need to re-up them and CFRs.
Again, on previous projects, there have been a list of CFRs, which we’ve turned into testable CFRs and put them into backlogs JIRA or whatever. Then, we had sections in the ADR, which is like applicable CFRs and applicable principles. On this current client, they don’t have CFRs, but I thought that’s crazy. But actually there’s a level of maturity, which means they kind of are doing them without needing them. But sometimes implicit behind an ADR will be a CFR. Then, we, as the architects pull it out and go, “Right. Okay. This is the CFR. Let’s agree to that.” For those kind of things, we do need agreement like log format because we need an agreed log format, right? We can’t have 17 different flavors. Let’s agree that, but then within that, different teams can decide to implement that log format in whatever ways they want. We’re separating out the bits that architecture do own and are across the piece from the individual implementation of those things.
A spike will likely create an ADR [14:42]
Thomas Betts: I think you mentioned when people don’t have CFRs or those documents of, “Here’s how we do stuff,” you always have a software architecture. It’s just whether it was intentional. I like the ADRs, that paper trail saying, “These are the decisions we made that got us here. It wasn’t just arbitrary and we just happened upon it.” Sometimes that gets lost. I like the idea of the ADR being as lightweight as possible, so that I can make it part of the process. Like you said, if you do a spike like, “Let’s figure out what we should do. Here’s the outcome.” I think a lot of people struggle with what’s the outcome of the spike? Is it functional code? This is a good example of that.
Andrew Harmel-Law: Yeah, exactly. This is the thing. If it comes out the back of a spike, the ADR writes itself. It’s a nice spike with nice acceptance criteria like the ThoughtWorks BAs are all over this. You’re not doing a spike for fun. You’re doing a spike to learn something. What do we want to learn? If you learn something and it’s worthwhile, you’re going to do something with it, that goes straight into an ADR. It’s really easy. Then, people see the value of spikes because sometimes they don’t and they do. They’re like, “I’m not really sure why I’m doing this,” but if that’s where you’re going, then that’s really good.
Using the advice process for non-architectural decisions [15:44]
Thomas Betts: This seems very well-suited for distributed teams. That’s the thing. It spreads it out. One of the air quotes features of autonomous teams is that they are allowed to make their own implementation decisions. Do I want to use go or node or .net or Java to write my service? Can you do the same approach for those type of decisions that might not be architectural necessarily and just design decisions or implementations?
Andrew Harmel-Law: Exactly. It’s interesting. Having done this three or four times now, there’s always a way in. To begin with, it’s like you roll this out. Step one, people are like, “Oh, I haven’t just been given autonomy because that sounds like crazy talk. No one would give me that.” Initially, there’s a lull. Then, people realize that we’re actually serious. Then, they’re like, “Okay. Cool.” Sometimes one client, what the thing was, there was this pent up frustration that people wanted to just choose in certain circumstances because they had to write an ADR. They had to justify it, but certain circumstances. They wanted like a kind of bounded buyer, right? They wanted for their circumstances to change something like one team on a previous client wanted to use functions like lambdas. We wrote the ADR and there was a lot of to and fro and then people were like, “Okay. But startup time is bad.” All this kind of stuff.
All of the stuff came out and we had a really nice educated discussion about lambdas and the decision was actually this isn’t the standard architecture. It’s not where we’re going, but for this case, it’s cheaper. It makes more sense. We don’t care about slow startup time and we’ll learn a bunch of stuff, so that was good. Another team actually wanted to move backwards and go back to a slightly slower, safer tech as well because they were like, “We’re being forced into this architecture, which doesn’t really fit for our circumstances.” They did that. You’ll see that the other people that typically really get and really like it is lots of the work we do is moving teams from build with a separate support team to build and run, so that then places this kind of DevOps mindset, desire a kind of need onto people.
Then, you’ll see that what used to be the support team or the infrastructure team retreating back into more like a delivery platform type thing. They will use ADRs to share and get input into, “We’re going to start changing your load bouncers. Does anyone have any feedback, thoughts on this? We’ve done a bunch of research from what we know that you were going to move from ELB to network-level load balancers or whatever. These are the pros and cons.” Then, there’s a forum for that kind of stuff to come out. Right? Then, you’ll see lots of kind of infra level stuff and you’ll just see standard design decisions, right? Again, typically the ones you’ll see first and I’ve seen all the times I’ve done this, the first ones you’ll see will be my team needs to do an interface with another team, so let’s sit down and figure it out.
They’ll kind of be, “How are we going to do it? What’s the nature of it? Et cetera, et cetera.” Then, maybe you’ll see some internal stuff, but the first ones will be… Like you say, these ones where the teams are trying to clarify and draw a line on their interactions between each other. Again, that’s nice, right? Because sometimes there’s an architecture like, “God, I wish teams would worry about this more.” This gives them a forum to have those conversations and talk about that stuff. You see all of those different things. You don’t have to curate them, but you do sometimes need to go with that.
I need to make a meme actually. What I’ll do is I’ll hang out on Slack and go, “That’s an ADR,” because people will be talking about stuff and you’ll be like, “This is an ADR.” It’s caught on my current client, so the CTO will be like, “That’s an ADR.” Then, a few days later, you’ll see an ADR pop up because someone will write it down.
ADRs document point-in-time decisions [18:46]
Thomas Betts: One of the points you mentioned is these are all time specific that the ADR is documented. It has a timestamp. This was made on January 15th, 2022. It’s got people associated with it to say who was involved in the decision? Who’d you talk to? That seems really useful, because the example you gave of, “We decided to use lambdas.” Seems very much of at the time, these were the situation, but it’s a year later, someone brings it up and if they just said, “Well, we don’t do lambdas because we don’t trust functions,” or I don’t know why, but if it’s documented, you can go back and evaluate and say, “We want to evaluate. Again, should we use them now? Because these new enhancements might have solved some of the problems that we had identified in the past.”
Andrew Harmel-Law: Exactly. This is the thing I emphasize. Again, it’s not me. This comes from Michael Nygard’s original post about it. They’re a point in time, and like you say, doing with the advice process and you’ve got all of this context and the surrounding advice and what people thought at the time and stuff changes. Every time I do anything with any cloud-based platform, sometimes we’ll put a story on a backlog and by the time it gets to the top of the backlog, the reasons why we thought this was good have changed because the cost of it is completely upended or it’s added features that we didn’t even know existed at the start, whatever. That’s good to make sure it’s tied in time.
We’ve got statuses, so I mentioned draft. The statuses we have I see are: Draft, proposed. Proposed is kind of just like we’re looking for advice. It’s definitely probably not going to… We’re not still writing it. Then, accepted means it’s like we’ve made the decision. Adopted means we’ve actually put it into code and it’s running in prod, but then we have superseded or retired. Sometimes we’ll make a decision in various pieces of work I’ve done. We’ve put a timer on it. We’re like, “Right. We know this is moving fast, so let’s put a clock on it and say we’ll come back to it in three months. We’ll come back into six months.”
Other times, we made a decision which has been the right decision for tons of reasons. Typically, the big ones. There was one around about active directory and deciding whether or not to share an active directory across a bunch of stuff. At the time, that was right decision for cost reasons, security reasons, risk reasons, et cetera, et cetera. Then, at some point, the right decision was taken to use this shared thing. Later on, someone decided to do something to the shared thing, which for that point was, again, a valid decision to do the right thing. But it invalidated this previous decision, which is a thing. Right? In the old days, you’re like, “Why did we make this previous decision?” We could go straight back and we could see with diagrams, with commentary, who do we speak to, what’s the history of this thing? We could see that. That was what would normally be like in a witch hunt for why did we make this bad decision? It wasn’t. It was the right decision at this time, followed by a right decision at the later time and things changed.
That, right up to CTO level, was valuable because it’s like, “Right. Okay. We were doing the right thing based on what we knew. We’re now changing, so things have to change. We have to re-plan and rework,” but it short circuited all of the kind of why and naval gazing and all this kind of stuff and just got back to, “Okay. Cool. Right. We’re moving forward. We’re doing things. We have to re-plan. We have to re-engineer and retool stuff.” It’s just nice. That’s the thing. If you have all this beautiful stuff written into the ADR, it’s like, “I remember exactly what we were thinking,” or there was a blind spot. There was a thing we didn’t know about because we hadn’t googled properly.
Thomas Betts: One of those pieces of advice that I heard later in my career is assuming someone’s built a system based on the best knowledge they had at the time, that they made the best decisions with the given information and the fact that the ADR can actually capture that. You can understand like they weren’t an idiot. They did the best they could. It doesn’t make sense anymore and saying, “This is the time to adapt and revise that.”
Supporting elements in the advice process [22:06]
Thomas Betts: I want to make sure we cover these today in the podcast, four supporting elements. We’ve talked a lot about decision records. The other three, I’m going to list and we can come back and talk to them. A time and place for conversations, which is the architecture advisory forum. A light to illuminate a unified goal, which is team-sourced architectural principles. And then a landscape and current climate sensing tool, which is your own tech radar. Let’s start with the purpose of the advisory forum.
The architecture advisory forum [22:32]
Andrew Harmel-Law: The advisory forum is like… That’s the most fun bit, and I look forward to. It’s always a bit scary because it’s called advisory forum on purpose, right? Everybody has a architectural review board and lots of people… I think this is one of the near enemies, so I think lots of people read this and fall into. I think they’re like, “Oh, we already have one of these.” You do, but it’s not exactly the thing. The key thing is this is where the advice process happens. Right? People aren’t coming to look for approval. They aren’t coming to look for consensus. They’re looking for advice. Current client, I was warned by one of the architects. They were like, “We set one of these up before and we could never come to a conclusion.” I said, “I think it’s going to be different because we’re not looking for conclusions. We’re just looking for a discussion,” and freeing that up. The discussions were awesome.
The client that gave me this advice, there were tons of smart, smart, smart people. Right? But then because they were all smart and they all had like 80% of the view, they couldn’t ever come to a conclusion, but we sidestep that problem and combined with things like point in time decisions, small decisions, so it’s easy to come back and remake them and all this kind of stuff or right size. Right? Some decisions are big, but making them as small as they can be, it totally works. Again, I think it’s in the article. We’ve got like a standard kind of standing agenda. What’s typical is so we’ve got… My current client, we’re trying to make it as async as possible. We do as much preparation for as possible, as much tidy up afterwards as possible, but it still gets like 40 plus people for half an hour a week, which for a client that’s not desperate to have meetings, that’s good.
I’ve been told that people are coming because there’s a value to them being there, right? There’s a value to the conversation and not just the people who are contributing, but the people who are listening. Again, it’s the whole architecture thing, hearing people think out loud and share history, et cetera, et cetera is super valuable. That’s that one.
Architecture principles [24:11]
Andrew Harmel-Law: The next one is the principles. Again, everybody’s probably got principles. The nice thing about this is it makes sure that the principles are executable and they’re giving value because there’s always principles, which everyone agrees, but they’re not really helping me decide of A versus B versus C. This makes them executable because you’re giving the principles, putting them in the hands of the dev teams who are doing the work.
If it’s an irrelevant principle, they’ll tell you. If it’s an out of date principle, they’ll tell you. I think it’s in the article. Again, there’s a thing which we’ve evolved, which is it’s not invalid just to do something which is contrary to a principle, but you need to write down on the ADR why it is better to do this the other way. It makes you think, right? So why are we doing this thing? That’s that bit.
Your own tech radar [24:47]
Andrew Harmel-Law: Then, the last bit, the tech radar, which is A, is just tech radars are awesome for again getting people to see where they are, see what everyone else is doing, cope with the fact that, like you said, you want teams to a certain extent. Well, as much extent as make sense, you want teams to be able to pick the tools for their job, right? It’s in the Accelerate, right? Nicole Forsgren said, “One of the ways to make yourself into a elite performing DORA type organization is teams pick the tools that are right for them.” But then people are like, “There’s so many tools. I don’t even know what’s going on. How many different logging things and frameworks have we got?” Right? Do a tech radar. Let’s find out. It’s super easy to do. The instructions on the website’s free, and it’s fun.
You’re having conversations about what tech are we using? What platforms are we using? What practices are we using? What frameworks are we using? The ThoughtWorks tech radar is aimed at… The standard rings are aimed at us and looking at the industry. I’m not sure this is in the article. What we change it to is a historical progression through the company. The one in the center, which is actively looking to retire or something like that. The next one out is we’re using it. We’re not actually actively retiring it, but the one beyond that is probably the current pick and the one on the outside is things we’re actively looking at. It’s not someone did a Google search, but it’s like a team is proof of concepting this or are playing with this. Therefore, you can say, “Oh, interesting. That other team who was doing this thing, let’s go and speak to them.” Again, it helps the advice process. It helps the advice seeking, et cetera, et cetera.
Thomas Betts: I like the idea of that creating guardrails. Don’t give you the blank slate. Here’s some things to choose from, or if you have a new one that you’re thinking about, “Here’s a way to evaluate. Does it come into the possible stack?”
Andrew Harmel-Law: Again, the rules are like if you want to bring it on the radar, then you got to write an ADR. Right? Then, you’re putting people back into the thought process which someone would do. This is nice. Evaluating tools, whenever I’ve evaluated tools as an architect, I always go to the team because I’m so far away from doing these things these days, that what I will think is a minor problem, they’ll be like, “Man, that sucks,” because it gets in my way every 15 minutes. Again, you’re giving them that power, but you’re like, “Right.” You need to think about the cost. You need to think about how many people with these skills are available. You need to think about the licensing, all of this kind of stuff, so you can, but pay attention to all this stuff. Right?
The architecture forum is for advice, not for making decisions [26:59]
Thomas Betts: Then, you can circle back to the architecture forum. I like that idea of don’t make decisions in that meeting. Don’t come to a conclusion. It’s about the advice about the conversation. Architecture’s all about it depends understanding the trade offs and identifying those. When I’ve been in meetings that have been that conversational aspect, you’ll see somebody sit back and listen and listen and listen and listen. Then, their camera comes on. They make one little comment that no one else had been thinking of and everyone’s like, “That’s a really good idea.” Have you seen that?
Andrew Harmel-Law: Yeah. Yeah, totally. There’s a bit near the bottoms like how to fail. Things you end up policing, people will still think that they’re… Even if you keep telling them, they’ll still be looking for others to approve and you’ll hear people in their language, they’ll be like, “Do we all agree?” I’m like, “We don’t need to agree. Stop trying to agree. Why are we trying to agree?” Or, “Is everybody happy?” If you’re not happy, that’s probably good. Right? Because then, like you say, you’ll get someone who’ll come in with this lightning bolt of, “Yeah. But what about…” And they’re like, “Oh, yeah. God, we forgot about the fact that we can’t upgrade this thing because of X or whatever.” Right?
The invitees are also important. Everywhere I’ve done this product people come because product, it’s really early warning, A, for them to know about cool stuff that’s coming out of tech. B, it’s good for them to head stuff off at the pass before devs go off and do something without the context that that product might give them.
But C, it also helps them remind or provide that perspective. For example, if there’s a cost element to something, they can be like, “Right. Remember our scaling strategy is to go here.” Therefore, if there’s a per user cost of licensing something, that blows my option A out of the water. That’s really cool and so they see stuff and they know what’s going on.
It’s also awesome for security. Security, everyone’s trying to get security higher up the food chain and get them earlier on and shift security left. This is almost like before a story’s even been written, right? Security can be there and go, “That’s GPL version one. That’s just never going to fly for these reasons.”
Then, because it’s the conversation and everyone’s learning, people are learning about security and they’re like, “Oh, yeah. Maybe they’re not jerks. Maybe they actually are worried about stuff that we need to worry about.” They’ve just explained as opposed to saying no after I’ve put all this work in, they’ve said, “You really need to think about at the very start.” Like the richness of the conversations is amazing.
Then, like you say, the last bit, when it beds in, I spend most of my time thinking like, “Who’s not talking? Why are they not? Is there anything that they can do?” Then, behind the scenes curating stuff, because it’ll start off. It’ll be the typical voices. Then, you’ll hopefully see some people who are like the leads of most of the teams. But then, you’ll start seeing people who have strong opinions in those teams.
This is why advice process is good to write. There’s no right or wrong answers. There’s no yes or no that the power dynamic changes. There’s the loudness of voices, which you need to watch out for. But if you want a high performing organization comprised of high performing teams, you want the safety to have opinions. You want the safety to disagree and this is a safe space for that. Right? As long as you look after it, as long as it doesn’t become the three big architects battling it out in front of each other.
The role of the architect in the advice process [29:43]
Thomas Betts: I think one of the things we talk about at InfoQ on a somewhat regular basis is the role of the architect, and whether you’re just the person who has all this technical knowledge and you can make the decisions, a lot of times, you need to be able to communicate and you need to be able to communicate at different levels. I think it’s the architect elevator by Gregor Hohpe. I know we’ve mentioned that before on the podcast, the idea of, “I need to talk to the developers doing the work all the way up to the product managers, to the CTO,” whoever it is, being able to change your story. Are you saying that now the architect needs this role to be almost a facilitator to help guide discussion and help the advice and conversations happen?
Andrew Harmel-Law: I personally think that a great deal, yes. One key thing and hopefully it’s clear in the article, I’m not saying this is the way to do all architecture. This is how I do architecture every time I get an opportunity. You could then say, “I’m a consultant. The whole point is I’m not around. There’s a vested interest for me and my clients in me making myself redundant and building the capability of architecture as much as possible for my clients.” But it works really well because there is… At some point, this doesn’t go right to CTO. Although current client, the CTO, whenever she can, comes to the meetings and ditto for previous clients. Well, actually the elevator piece is taking that kind of stuff and playing it back and making it clear at different levels of the organization is so important. I think for a great extent, this gives you the context and the detail and the confidence and the support of the people who are doing it.
When you’re higher up the architect elevator, you’re frequently asked to be very categorical about things. If you’re like, “I’m not sure how much support I have for the teams who are actually going to deliver this.” Right? But you’ll be able to reflect that because you have seen that or just the general feeling of the general acceptance of ideas and all this kind of stuff. It works. It frees time up for that, and it works quite well.
There is one law or a soft underbelly. If people higher up the elevator want some beautiful unified overall diagram of all of this stuff, this will not produce that. Either you have to produce that or in other places, point out the fact that you’ve gone for this autonomous teams distributed architecture. Therefore, you’ve surrendered this beautiful elegant… We all use Kafka and we all do this kind of thing.
There is a unified architectural blueprint, because if you really want to get the benefit of all of this, then maybe your aspiration should be in that direction, but that’s a flaw. If you need all this stuff for regulatory, whatever, then you’d have to put the timing to do that because this isn’t going to emerge naturally for the team, but all of the pieces to put in place to do that would be available to use.
Thomas Betts: The distributed architecture diagrams or documentation or ADRs, it’s a feature, not a bug of this approach.
Andrew Harmel-Law: Yeah, exactly.
The advice process is a paved road for decision making [32:11]
Thomas Betts: One of the things we see for microservices to be successful is people need to have the paved road that you need to be this tall to use microservices. There’s all this stuff that needs to go around it. It sounds like the architecture forum is part of that paved road approach for architecture. We’re going to set you up on success. You said that you’re a consultant, you come in, but your job is to make yourself redundant. Let me give you the platform. This becomes the platform. Then, everyone just writes ADRs because that’s the way we do things.
Andrew Harmel-Law: Exactly. It’s interesting. If you read, I think it’s in Teams Topologies, they talk about platform teams and they talk about minimum viable platform. For the Teams Apologies guys, minimum viable platform is… I think the minimum is a Wiki with people to go and speak to because they’ve done this already. I was like, “That’s an awesome idea.” I’m not even sure. I think I refer to it in the article again. Right? This, like you say, is a minimum viable kind of set of approaches. Always, we end up layering stuff on and adding things into AAF, agenda adds pieces, because people like the mindset is in the right place. This, again, is like the learning is happening out in the open and the decisions… You’ve got your principles, you’ve got your radar, you’ve got all of the standard stuff you think it’s going to be, you’ve got your existing architecture, but then people are constantly building on top of that and building the in stuff.
For example, like I say, if the infra people are there, they’ll also get for the real paved road, right? They’ll be hearing people doing stuff again and again or the architects will be like, “Oh, they’re doing it again and again.” Then, that’s a good opportunity to say, “Right.” As an architect, I have suggested various things because we’re like, “We seem to be doing a lot of SQS integration. Do we think it’s time for us to start putting Kafka amongst all of this kind of stuff?” Right? Then, you introduce like a broader thing or API manager, right? API manager is the classic. We should put in API management, but you can introduce that as a topic and then see what people think about it and go, “Okay. Cool.” You are the person who would be taking the decision because no one single team because of their limited scope, are going to say, “We need an API manager,” but you can still use that process and use that kind of forum to get a feel for that kind of stuff and you can then advocate for the fact that the combined pain of this team, plus this team, plus this team, plus this team, plus the fact that at some point, this stuff’s going to start tripping us up.
Which is where things like the four key metrics are nice because you can then get teams thinking… They’ll be thinking very specifically, but four key metrics makes them think more broadly and holistically. Those things combined can help you drive and have far greater adoption and understanding, and buy-in into these core, bigger piece things because you’ve got all this stuff, right? It’s like minimum viable architectures, like it’s happening and it’s getting into prod. Right? For me, it’s like… Otherwise, it’s all irrelevant, getting into prod, then the skills are building up. Then, the understanding of, and desire for architecture comes up and then you’ll start seeing people see common cause arising, people supporting each other, advice will be going back and forth and all this kind of stuff. Right? It drives out all of this stuff, all of this kind of collaboration sharing. It’s like one client didn’t want to call it an AF. Basically, they call it the architecture guild. It’s an AF, right? But the nice bit, the guild bit, the discussion piece, all that kind of stuff.
Thomas Betts: The connotation, I know of guilds is guilds are like, “Here’s our little fiefdom. We decide all the rules and you have to be part of it.”
Andrew Harmel-Law: It’s like an architecture guild, but everybody’s invited. They’re like, “Oh, actually I’m doing the architecture. I didn’t think I’d be doing this.”
Thomas Betts: Choose the naming that fits your environment.
Andrew Harmel-Law: But the big thing is it’s not where people go for approval. That’s the thing I fight. There’s a few things I’m like, right? With this, we’re not doing this. That’s the key thing. Every single client where I’ve done it, I’m like, “Right. This is going to happen. This is the key fact. If you’re not comfortable with this, I still suggest strongly that we do it and I would love to find out how we can experiment with it.” Because some people are like, “You’re going to need to take me on a journey for this stuff.” Other people are like, “Woo-hoo. This is spot on. This is exactly where our culture needs to be.” Keeping that, but it goes back to the advice process. Everything else layers around about it. If you get that, that’s how you kill it. If it stops being advice, and it starts being approval.
The one thing is two clients ago when we did it, I’m still in touch with the people who still do it after I left, it’s two years old and it’s still going and it’s still thriving. They’ve changed it as it belongs to them. But one thing I don’t know… Again, full disclosure, I haven’t done this for five years and seen what happens. There’s nothing that makes me think it would rotten degrade, especially the fact we revisit the ADRs and we come back to stuff, et cetera, cetera, and it evolves, but I can’t tell you what it looks like for us in five years in the future because I haven’t been there yet.
Thomas Betts: Things will always evolve, right? In five more years, we’re not going to be doing microservices, there will be the next thing after that. We’ll definitely be looking for changes.
Well, unfortunately, that’s about all the time we have for today’s podcast. I really enjoy this conversation. We’ll, of course, have a link to Andrew’s article in the show notes and all the other things we talked about. If this conversation is giving you some ideas or you have more questions, please write a comment on this episode by going to infoq.com/podcast. I want to thank our guest. Once again, Andrew Harmel-Law.
Andrew Harmel-Law: Thanks very much. It was an awesome conversation. Just to Thomas’s point, I’d really encourage feedback. I’d love to hear what people think. Good ideas, bad ideas, kick holes in it. I’d love to see what people think.
Thomas Betts: We hope you’ll join us again soon for another episode of The InfoQ Podcast.
QCon brings together the world’s most innovative senior software engineers across multiple domains to share their real-world implementation of emerging trends and practices.
Find practical inspiration (not product pitches) from software leaders deep in the trenches creating software, scaling architectures and fine-tuning their technical leadership
to help you make the right decisions. Save your spot now!
From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.