Sunday, October 2, 2016

We Adopted Agile. Where Did We Go Wrong?

Or, how methodologies can lead to misery

Lean. Adkins. Extreme Programming. Paleo. TDD. Agile. South Beach. Scrum. They're all lovely tools for either dieting or organizing the process of software development. Poorly used, they’re a fast path to misery and disillusionment.

Like diets, development methodologies require discipline, effort, learning, and social support to adopt. If on Monday you abruptly adopt a strict diet, you'll feel depleted by Thursday. Unless you possess great willpower and motivation, you’re at risk to slip back to your old eating. A diet builds on many habits buried deeply into our routines and physiology. Changes require more than simple knowledge of the new diet and it’s benefits  Why do we expect an easier time switching software development processes?

Software development methodologies are designed to address specific deficiencies in outcomes. They assume a certain level of proficiency, and a certain ability to change from your current methodologies. If you don’t meet those table stakes, if you have different deficiencies, or if you have additional deficiencies, the methodology won’t be enough.

If you can’t stick to a diet, it can’t help you. Even if you diligently stick to the diet, your lifestyle isn’t healthy unless you also exercise. Failure is also possible if you exercise and eat according to the diet, but do a poor job managing stress. Successfully achieving your goals requires a holistic approach.

The consequences of a poorly implemented software development methodology change read like fine print in a pharmaceutical ad: loss of productivity, attrition, missed goals, and painful meetings.

The Agile Team Myth

One rainy Monday the boss drags my team into our largest conference room. The boss has exciting news: we’re switching to scrum! So begins our development crash diet.

The boss just read a great article about scrum. Unlike our waterfall process, scrum will allow us to ship more feature-rich, higher quality software in less time. Their friend at Initech uses it too, and the whole company adores it. Eighty three minutes and 16 slides later it’s official: we have a new methodology.

Fast forward two months.The visible parts of the process have changed: daily scrum meetings and org charts with scrummy titles and scrummy  shapes. Guess what? Our output may have had a brief spike, but productivity has settled back to the pre-scrum levels. It’s probably somewhat worse.

The team doesn’t work quite like the methodology says it should. Some team members have more political power than others, so they get to skirt the principals of scrum to varying degrees. Other exceptions to the scrum process occur because “we can’t do that here”. Our team also continues some old processes under the cover of borrowed vocabulary from the “new methodology”.

Where we do adhere to scrum, we feel anxious because we’re not comfortable with the new process. Because we make mistakes learning our new methodology, things take longer, and we receive frequent negative feedback. Sometimes the boss makes a mistake and reprimands us for following scrum. Relationships between QA, product management, developers, and management become strained. Everyone looks tired.

Some team members feel cynical about the change. They note that now team successes are attributed to the boss’s scrum decision while scrum failures land on the team or individual members. Others are starting to reply to recruiters.

Some individuals are confused or distracted by the chaos of a new process. Others withdraw, ignore scrum and just focus on their area of expertise.

The team suffers from change indigestion. New methodologies require a stack of new skills and habits. Each one requires practice and feedback to master. Teaching one person a new skill requires a lot of effort. Teaching ten people thirty skills all at once requires magic. Teaching ten people thirty skills while they continue their normal work lies beyond even magic’s abilities. We have too many new tasks — we can’t be good at them all.

The team resists scrum. Because a manager imposed the process without input from the rest of the team, we don’t feel committed, nor do we have the same expectation of benefits. Forcing such a large change steals autonomy. Crippled autonomy means less ownership and motivation. The reduced feeling of control over our work environment makes us less happy.

Methodologies Promise Better Teams But Only Deliver Better Tools

Like fad diets and nutritional supplements, the library of software development methodologies promise a cure for every potential team problem. This methodology helps you deliver more of what the customer needs? Perfect, that’s what we want to fix!

Just like protein powder doesn’t build muscle without exercise, sometimes you need improvements outside the methodology. Diets don’t address broken legs, how you handle stress, or gym time. Methodologies don’t address missing skill-sets, cynical attitudes, sloppy communications, or poorly defined roles.

Do members of your team have difficulty writing clearly? It won't matter if they are writing stories, tickets, features, or specifications. Poor writing makes poor feature descriptions, which causes confused development processes.

Does your leadership have trouble getting along with the developers? It won't change things if the leadership role is named team lead, manager, tech lead, scrum master, product manager, or sorcerer. The team will still feel mistreated, untrusted, or confused by their interactions.

Maybe your developers just build whatever they feel like. Will they become more obedient because you've gone from waterfall to agile? Nope. If they didn't obey before, they won't obey under your new methodology. Waterfall isn’t the issue: team dynamics are.

You’re not alone if you confuse great teamwork with great methodologies. Software teams talk a lot more about methodologies than team dynamics. We look at what other teams are doing and compare ourselves. We call ourselves agile. We debate code style and programming languages. But we rarely take the same time to discuss the human interactions that happen in our teams

Tactics For a Happy Productive Team

Review your team’s strengths and weaknesses. Is the code quality great, but the velocity too low? Is too much time spent in rework? Are the team members happy? Who needs help or training in what areas?

Examine your team’s fundamentals: communication skills, mutual respect, customer feedback loops, hiring practices, organizational skills, etc. Invest money and time teaching deficient skills and offering feedback. Tune the team. Help them do better planning, work on empathy, or practice writing.

Make individual roles clear to the entire team. Everyone should understand the work product of the other team members, including the leadership and business roles: product management, technical leads, and managers. The output of everyone’s work should be public and visible.

Once you address skills and habits at the individual level, you can work on changing the habits of the team itself.

If you eat two pounds of fries every day, you can't expect to switch to a kale diet overnight. That's crazy talk. You’ll crave those fries at every meal. You can resist for a while, but one day you'll snap. A tiny moment of weakness and you drain the neighborhood Wendy's of their entire potato supply. You'll sit in their plastic booth feeling sick, wondering why changing your diet is impossible.

Changing isn’t impossible when you set reasonable milestones to reach a larger goal. Success changing habits means catering to your psychology. Don't immediately drop the fries and pick up the kale. Measure your current fry consumption. Set a goal for a small but meaningful reduction. Adapt to it. Add a little kale. Figure out how you enjoy preparing it. Aim for the diet you want, move towards it. But don't trebuchet yourself into a new diet expecting an instant drastic change in your behavior. Catapults work against fortresses, not against behavior.

Perhaps your team designs and plans the wrong features. Just because you’re experiencing poor product-market fit, doesn’t mean you should toss your whole methodology in the trash. That's like selling your old car because the oil needs changing. The car isn't the issue, the oil is. To keep a car running, change the oil. To build the right features, you talk to your customers.

Yes, some methodologies directly address customer feedback. That doesn’t mean you have to start with the entire methodology! Pick one tactic from your favorite methodology and adopt it. Once it becomes a habit, pick the next tactic. Repeat until your customers send you love letters.

As you change things, use teamwork rather than pure authority. Habits are difficult to dictate. Collaboratively pick the “habit” to change with your team. Present your goals and seed some ideas. Maybe you’re consider changing how you review code, how features are prioritized, or how planning works. Whatever it is, socialize it. Get consensus. Present the change as an experiment to make it less threatening.

Allow the habit time to stabilize before making another change. Measure the results. Ask people what they think. Celebrate any successes, and quickly address any failures. If the experiment isn’t working, use the team again. Change the experiment. Make the process fun.

My team changed our code review process a several months ago. We started it as an experiment. We didn't just declare that we had a fix for code reviews. We discussed the shortfalls of our old process and tried some ideas that might fix it.

As we reviewed code over the next month, we reflected on the results of the experiment. Were the new habits sticking? Did the code quality improve? Are we learning from each other, and beginning to understand new areas of the project? Did we enjoy it?

Because we examined the results from the perspective of our goals, we felt free to make tweaks along the way. If we just imitated a methodology, we would have focused on imitation. We wouldn’t feel ownership of the outcome. Owning the change meant that we adjust it to get a better outcome, and that we felt responsibility for success.

If we made a mistake, we just said “oops” as a team and briefly discussed if there was something we should change. Because our mistakes were usually a forgetful flop back to old habits, nothing exploded. There was no breakdown of some huge new process. No other process  depended on a new code review process, so there couldn’t be a cascading failures. Our one mistake wouldn’t get us blamed for three more.

Messing up meant we reviewed code the old way; no big deal. That’s the cost of trying something new, and a useful opportunity to reflect on our new process. We didn't feel exhausted, lost, or overwhelmed.

As the team masters each new habit, add another to the stack. If you’re diligent, the team will develop a talent for mastering new habits. Your ability to improve the team will accelerate. Eventually you’ll make so many changes that you’ve landed in the new methodology. Perhaps you’ll build a bizarre Franken-methodology that you can brand and write a book about. Either way, congratulations. You have a product-building machine.

Stop switching methodologies like fad diets. Worry about the right methodology after you have the basics of running a development team under control. When your team runs smoothly you’re positioned to experience the benefits of an appropriate development methodology. When you do adjust your methodology, minimize risk by making gradual changes, letting the team own them, and by measuring the impact.

No comments: