Orthodox and Reformed Scrum, a Retrospective

I have recently been doing a lot of Scrum process at work. Enough to make me feel entitled to critique it1. I figure that so long as we’re being assessed on our implementation of the Scrum process, I should be able to assess Scrum on its implementation of a software process. Note that I have already discussed the term “user story”. So I’m going to try something different today, and actually discuss salient conceptual points. Eventually.

As a dual Australian/American, I am very well qualified2 to critique the name of the software process “Scrum”. A scrum is a formation in Rugby (one of several Australian football-style games) to get the ball back in play after a minor infraction which does not warrant assigning possession to the defending team. A huddle is a formation in Grid-Iron (American Football), usually performed by the offensive team, where they confer rapidly on-field about their strategy for the upcoming play. I hope those who chose the “Scrum” process name were actually meaning a huddle, especially since the use of Scrums in Rugby has been reduced due to negative spectator feedback3. They look similar enough, especially given the (incorrect) stereotype that software developers don’t like or understand meat-sports.

Let’s lay some foundations here for a serious soliloquy. My canonical source for scrum practice has been http://scrummethodology.com, for agile principles has been http://agilemanifesto.org, and for strategic thought has been http://suntzusaid.com4. My perspective on “process” has long been a pragmatic one, for realism is the essence of process to me. Imagine you have a “Platonic Engineer”, who is the idealized essence of everything needed to create software. They would require no process, for they already know what to do. They have an innate plan, they can demo or estimate at a moment’s notice, and their idealized nature provides the ultimate in quality assurance. But every platonic ideal shares a common imperfection, namely that it does not exist in reality. Process is how you herd an imperfect group along a reliably imperfect course, for an unreliably imperfect course could dip below your standards of satisfaction. Sadly, reality makes even that statement an oversimplification.

The imperfection of humanity is not constant, nor does process come from some source external to humanity. You could have process written by people who aren’t good at understanding processes, and assign that process to people who are really good at software development. I’ll call this “artificial drag”, as this hypothetical case is one where you’d reliably get a better result without that specific process than with it. You’d think the answer is simple, get your best people to develop a process which lifts up the rest to an acceptable standard. The final, and most depressing thought, is that the ‘best people’ tend to want to do the real work instead of building abstract structures that are a disposable scaffold around real work.5

Moving along, there is no real final thought. It’s all about where you draw the finish line(s). This is a key concept in Scrum, with short ‘sprints’ instead of a long product lifecycle. Modern schooling produces workers who fit this pattern much better, for schools break work down into short, individual items of assessment. These can invariably be completed the night before it’s due and this pattern of effort will fail if applied to a long product lifecycle. So by breaking down the product lifecycle into sprints you manage to shape the work in a way more amenable to the workers.

Despite the flippant joke about cramming sessions, I actually do feel that these short sprints are easier to work with. Most productivity literature I have read urges you to break down tasks into short-term achievable work items. Not that you need to constrain your goals, but that in order to actually pick an item off your to-do list you need to have it well-defined. Otherwise you’re never sure if now is ‘the time’ to do it, and tasks seem like nebulous ogres instead of sharp-edged lollies. Short sprints force people to do this, if they are not in the habit of doing it normally. A little constricting to be always lining up to an arbitrary deadline, but better than if you weren’t breaking down tasks into manageable chunks before.

This clearly lines up with the goal of process: bring up the standards for the low performers and accept some unnecessary overhead on the high performers. I would consider the short sprint cycle a success for the scrum process. But then how do you plan for the future? You can always just drop the waterfall model on top, and have multiple sprints per release (with ‘feature’ and ‘hardening’ sprints). But that makes a mockery of agile, which explicitly values responding to change over following a plan.6

I favor the theory that a plan is nothing, but planning is everything. Strategic leadership needs to plan, but they need not commit to a specific 12 month deliverable. They should adapt to make use of the deliverable they have after 12 months, which is actually fairly easy if you’re constantly adapting your plans and can see the trajectory for estimation purposes. It’s even easier if you’re constantly planning, and not worrying about the multitude of plans you devise but reject. By constantly adjusting your plan on top of a minimal process, you reduce overhead7 and allow the chance of using best-case exceptional output from your exceptional performers (and the minimal process sets a minimal standard you can live with in the worst-case). However this style can seem ad-hoc to the development teams, and can lead to a lack of product vision if not managed well. This year was the first time I saw engineers actually asking for a meeting about the product vision! It’s a sign that I have failed them all, but the scrum process provides little help.

In fact, for the scrum process all product-side knowledge is concentrated in the “Product Owner”. This seems questionable to me. Are we talking about a workshop of corporate drones who work ‘to spec’, or about passionate engineers eager to create a lasting and utile monument? Only in the former case should the product owner be telling them what to make. In the latter case the employees, even if merely salaried minions, are making something for themselves as well as the company. It is a challenge to strike the balance between passionate creators and actual market needs, but great products are forged upon that balance. I’ll concede this could be our implementation, where product owners spend the core focus on forward planning and external stakeholders. But I would like to ask the creators of scrum8, should the product owner work closely enough with the team that they inspire them to internalize the product vision?

I would assume so, as one of the goals in Scrum is “Self-organizing teams”. We can take it for granted then that the team wasn’t self-organizing to start, or else they wouldn’t need a scrum process layered on. The reasonable interpretation here seems to be that scrum is about individual development, kind of in line with the agile manifesto valuing individuals over processes. It aims to help teach team members the skills they need to become a self organizing team, and no longer need a specific “scrum” process imposed on them. As any company slashing their R&D budget can tell you, that kind of long-term focus is hard to swing by your typical corporate executive.

I think this aspect of scrum needs further thought. A purely facilitating scrum master is part of the formal plan, helping a self-organizing team to work through their differences and achieve something wonderful together! Anyone with that kind of ‘facilitating’ power would be better spent brokering peace in the middle east. The self-organizing team ideal, while it is ideal, is quite far from reality. Even with a dedicated resource to drive towards that ideal, that resource is hardly ideal either. In contrast there are existing models of leadership, particularly technical leadership, which have experience at being effective in the real world.9

In our implementation of scrum at work, technical leadership is essential. The idealized scrum team is cross-functional. For leadership in software development, ask the software developer. There’s only one, so his mastery of the realm is indisputable. When you cram an R&D department at a large company into scrum, which has twenty developers and five QA Engineers, the best you can get is a QA member per team and everyone else has to peck away at a social order. Naturally where I work we put the QA engineers in a separate team so as to have mono-functional teams, where everyone has the same skillset. With five developers in one room you’ll get a minimum of five different ways to write the application. Now there is a clear need for more than just a facilitator, for how can a non-technical scrum master choose which technical approach to go forward with? Helping them seek consensus among themselves is a much slower route than choosing between opinions, as there’s no doubt in the team that any of the approaches could work.

To make things harder for a facilitator, multi-national companies mean multi-national teams. I like the diversity of cultures, being the only speaker of ‘strine (en_AU) in my team, and it has many advantages that could be discussed in a different topic. On this topic let’s just say it’s hard to get the team around one virtual, let alone physical, table when the time-zone difference spans eight hours. Fifteen when I’m working from ‘home’. It is a difficult communication problem, and my preferred solution rules out non-technical facilitators even more. I prefer focusing on the code, and communicating a lot through code reviews, as that gets the most out of a limited communication frequency. When you write out your thoughts in such clear and excruciating detail that even a computer can understand it, another software developer has a very easy time. As code reviews tend not to be piecemeal, you can often make a wide variety of comments (many of them trivial) all at once. Further efficiency can be gained by splitting the task into logical modules and assigning them by time-zone, but you’ll still need someone unifying the disparate modules. A non-technical facilitator cannot do that, and getting the module owners to work as equals means a lot of going back and forth (assuming they are capable of agreement). If you have them, strong technical leaders are a superior solution than scrum masters in this situation.

Of course, the best tool is always the one on hand. If you have skilled non-technical facilitators, don’t waste them (but they work best in the ‘idealized’ scrum environment: everyone in the same room, no duplications of team roles). When you get down to it, a similar theme rings true for process. Having something, anything, on hand as a process raises the minimum standard for the result. Risk-averse companies probably reach out for scrum because it’s well defined, on-hand, and very open to being twisted into whatever form you really need. I don’t even object to it as a starting point, so long as no-one is zealous about maintaining strict adherence to ‘scrum doctrine’.10

Now that we’ve covered the theoretical basis, I can safely move on to a completely separate topic. How I’ve seen scrum used where I’ve worked. The scrum process is practically a platonic ideal itself, for it does not exist as theorized when used in practice. I already talked about some conceptually issues I’ve run afoul of, but now it’s just story time11.

When I worked at Trolltech, we basically did cowboy coding. The lead developers for each team applied their ‘brand’ of leadership and herded us along pretty well. It worked pretty well, and we got a lot done, but with an organization of engineers all the way up we didn’t actually make any money. We were kind of lucky when Nokia bought us, especially since our noble Trolltech management made the supreme sacrifice – interfacing with Nokia executives so that we were isolated from their influence.

Still, it was around that time that we were supposed to “do agile”. I can’t remember when exactly, or if we were actually doing scrum (we weren’t told to do XP, I’d remember the hysterical fits of laughter if we were). The important thing is that it was left entirely up to the team to implement, and so my team chugged along as we were always happy to.

Other teams did what they wanted too, and I recall looking across the hall and seeing another team playing with their colored sticky notes on a wall with columns and task numbers. It looked cute, and they said they liked it, so maybe individual taste is a factor. They had several skilled developers12 and I would have thought so much process would only slow them down. My interpretation is that they slowed themselves down because they were having fun, but an alternate interpretation is that this level of process truly did help them with their work. That’s an isolated example though, a rare team which I’ve seen who seemed to like process and who might have been helped by it.

In my next multinational, I got to scrum master two teams (while still developing software… special circumstances). It was eye-opening to experience two very different teams, dynamically trying different things with each13 to try and improve efficiency. One thing I tried was anonymous surveys in the retrospective, to help encourage honest and open feedback. We all had a good laugh about that one – not really effective in a group of five people.14 A more successful experiment was the “Outstanding Deliverables” meeting. Every sprint, just a few days before the deadline, we ask everyone with unfinished tasks what’s left to do. In theory this would allow us one last chance at identifying and resolving blockers, and also that team members who are done with their sprint tasks can help the others in crossing the finish line. In practice, this is where people ask for code reviews or check with the product owner whether they’ve done “enough” to count as done. For teams new to scrum this proved useful because otherwise they leave these questions until the last minute, at which point it ends up being completed a day after the sprint ends.

But while both teams were unfamiliar with scrum, one team was unfamiliar with more substantive details. Like the technologies they were supposed to be using – they had been pulled from disparate other teams and had never done this sort of thing before.15 This team seemed genuinely to benefit from scrum. We estimated as a team, broke down stories into sub-tasks, and used the sprint-level planning to keep the developers on track. I’m not saying they enjoyed it; like classical software development this was like herding cats and I benefited from every metaphorical scrap of tuna. However I would say that it provided much needed process and we’re certainly better off with scrum than cowboy coding in this team.

My other team was a different story. A story about a young boy, who always wanted to be a carpenter. He’d pick fruit all day to earn his keep, and whittle at sticks all night making beautiful carved wood ornaments to hang around his home. Then one day a big multinational comes in and hires him as a carved wood ornaments consultant.16 Getting back to the team, it was filled with talented software engineers passionate about what they were created (plus myself). We practically told management what our requirements would be, made up wonderful stories17, and the team needed virtually no facilitation – other than conforming to the process. And using the same tools as everyone else. And reporting accurately to management. This was a team where it felt like the process was adding drag to our velocity, although in true agile spirit we cared about the process less than the people. So it didn’t turn out that bad.

Every team is different, and I’m working with one team right now who just loves their process drag. They spend half a day each sprint filling out lovely spreadsheets and plan their work hour by hour. I hope that even they can see that the process is slowing them down! They most likely take this approach because speed is not their primary goal. BlackBerry has a reputation as a global leader in secure software solutions, naturally they have a vested interest in maintaining that. So their minimum quality standards (which the process intends to ensure) will be a lot higher than, say, a particle system. Maybe it’s a glacial pace, but when’s the last time you saw a glacier compromised by the NSA?18

When reliability is more important than productivity, I’ll concede process is king. In programming camps during high school, we sometimes took the ‘no-compile’ challenge on practice questions. You’d have an informatics question, where you submit your code to the website and it will compile and run it on some data to assess your result. When you’re feeling bold, you just write your code and submit it to the judge-bot without doing even compiling it on your end. There’s a reason we only did that for practice questions, as even Australia’s brightest young programmers made the occasional mistake.19 So people relatively close to the platonic ideal of a programmer still found benefit from some process (like compile, test, and only then submit) when the reliability of the result matters. Thus we used an agile process in the actual competitions, where you’d get a penalty for incorrect submissions (or sometimes no immediate feedback – they just judge your last submitted code).

Which takes us back to my competitive programming days, and back to an underlying philosophical point. For in competitive programming the exact rules and requirements are very clear. You learn to code to spec, and waste no time going beyond it. You can remember your single letter variable names for an hour or two? Go for it, and save that typing! You need to solve an O(2n) problem, but n ≤ 8? You know the specs of the judging computer, just brute-force it! You can solve the problem in O(n log n), where n ≤ 1000? Just do it in python, you can afford ginormous constant factors!

This makes the needs of the process very clear. Your standards are to write correct code, with the specified performance characteristics. You therefore evaluate the algorithmic complexity of your design, implement it, test it, and submit. It sounds obvious, but that’s because the clarity of requirements makes the process straightforward. Competitive programming is a contrived scenario where they can put the exact specifications on a single sheet of A4 paper, including examples, and they will never20 change their mind.

There is definitely less clarity of requirements in the real world, when you move outside of competitions and school.21 But I ascribe that to the vast diversity of problems encountered, for not everything can be solved with a stdin -> stdout shell script. It’s harder work, and it’s not done for you, but with a clear enough product vision I feel the ‘chaos’ of how to develop a software product is greatly stripped away. Which means that you need exceptional product vision as well as exceptional software talent to create an exceptional process. Fortunately most people will settle for a decent process (and unfortunately, that would probably be an improvement22).

All in all, I feel that Scrum (and to a lesser extent, Agile) are traps. If you can implement them right, then you don’t really need the structure it provides. If you can’t implement it right… you need to be honest with yourself that it’s not scrum, that it may not be agile, and that you need some degree of process. Waterfall is known to be a poor solution for modern software development, but too often Agile vs Waterfall is presented as a dichotomy. I feel a realistic middle ground could be achieved – but don’t make me have to invent one myself! I write about process as a hobby23, not to champion a cause.

In closing, I feel I was too harsh on the word choice for the “Scrum” process. I ask myself which is the more appropriate metaphor, a process to resume the game after a minor rules violation (and which is a little boring for the spectators), or a quick face-to-face communication between the team to plan the critical next 30 seconds. In theory the Scrum process matches one of those fairly well; and in practice it’s the other.

Leave a Reply

Your email address will not be published. Required fields are marked *