The Daily Kanban Stand-up

Many teams transition to Kanban from other agile processes. And often,  they conduct the daily stand up just like they used to in their previous process. Since Kanban is an evolutionary method, in the beginning that's fine. However, very soon you'll find that changes are necessary.

Teams new to Kanban use the stand up like a status meeting - often taking turns to update the team on the status of  their "tickets".  Or, they keep it very Scrum like, typically answering the following questions, with some variation,  in a round robin format:

1.  What did I accomplish yesterday?
2.  What can I commit to doing today?
3.  What are the impediments in the way?

While that may have been appropriate for your past agile process, of the three questions, only the last one is relevant in a Kanban stand up.

(1) is unnecessary because the cards on the board tell the story. The WIP (Work in Progress) is visual.

(2) is irrelevant because work will take whatever time it will take to get done. Committing to accomplishing the work will not necessarily make it faster. Often, that leads to heroic effort. In addition, the focus on completing the work that has been "committed" to can actually come in the way of continuous improvement.

So,  what should we discuss in the stand up? I'd suggest:

1. Impediments: What's impeding us?
2. Flow: What's the flow like?
3. Kaizen - or "Continuous Improvement": What can we improve?

Let's take a deeper dive into these questions.

What's impeding us?

Impediments are in the way of getting work done. Impeded work items should be marked visually on the board - usually with a pink sticky on top of it. Team leads or managers should focus on getting these impediments removed as quickly as possible.

Note that you don't have to talk about which item is impeded - that's obvious from the board. The conversation should be around resolution.

What's the flow like?

Kanban looks at software engineering as a flow problem.  The board will show where the bottlenecks are and therefore what's preventing the team from accomplishing more. The conversation should be around smoothening the flow and what actions the team can take to relieve the bottlenecks.

What can we improve?

Kanban is an evolutionary method. It's success largely relies on a mindset that's looking to constantly improve. These do not have to be big changes and can be a series of small improvements. Having a brief conversation on this topic everyday will make it clear that it is everyone's responsibility to make the process better.  When suggestions for improvement come from the team (rather than from managers), there is greater ownership and more likelihood of success.

These are the core questions. Here are some additional practices for a successful standup:

Ensure the board is up to date before the stand up

Many teams also use electronic tools along with a board. It is important to keep the two synchronized. Before kicking off the stand up, ask the team if the board is synchronized.

Celebrate the small victories

The done column on  the board should ideally have two parts. Work done for the week (or an appropriate time period) and work done since the last stand up. You can now see what got done since yesterday. Call attention to what got done. Recognize the effort, if only for a moment. Bring your hands together. Yes - applaud and cheer.

Celebrate small victories and energize the team.  And move on.

Change facilitators

Encourage different team members to take on this responsibility. When you facilitate, you'll get a different perspective of the process. You'll also take more ownership and funnily enough, suggest more improvements.

End the stand up on a high note

 If you have a team song, team dance, a war cry - do it. It'll always elicit a laugh and keep good cheer.

 I'd encourage you to try these practices and improve on them. You should bring in your variations to the stand up around the core practices.

Remember - the goal is to continuously improve.

Kanban Introduction

Kanban Blog

WIP limits - The magic sauce in Kanban

A software development process can be viewed as a queuing system. Little’s law states:

Lead Time = Work In Progress / Throughput

Lead time is the time that a work item (represented by a card on a Kanban board)  spends in a system in various work states. We want to reduce that, so we can get more work done i.e. increase throughput. As you can tell from the equation, reducing WIP (work in progress) is one way of achieving that. 

Less is more

We've all heard that and have certainly experienced that. That's an earthy way of stating Little's law. We can get more done, when we are focused and working on fewer things.

Why?

We are not wired to work on multiple tasks at the same time. When we say we are multitasking, we are really switching our attention from one task to the other i.e switching contexts. Context switching is a waste. Studies have shown that with every additional task taken up there’s a 20% loss. If we are working on three different tasks, we’ve lost almost half the time in just context switching.

Grandma was right - less is more. But don't tell grandma that. Not unless you want her to limit you to just a couple of  freshly baked cookies instead of the usual half dozen. 

Ok, we should limit WIP.  But how do we determine the WIP limit?

We experiment.

If we set the limit too high, we will continue working on multiple tasks. If we set the limit too low, we may cause bottlenecks and affect the flow of work in the system.We want the WIP limit to be optimal so we can have a smooth flow.

But it’s not a science. The board will tell you. 

There are advantages to starting with a lower WIP limit

Yes, they’ll cause starvation and pain. But lower limits will expose potential problems in the system. The industry uses the analogy of the WIP limit with lowering the waterline. Lowering the waterline exposes the rocks (i.e. bottlenecks and constraints).  When the problems are exposed, the team can work to remove the bottlenecks and constraints until the work flows smoothly again. 

And so on and so forth.

It creates slack

When everybody is working  all the time, there's not much time to introspect, retrospect, experiment, innovate or even  have lunch. As a rule of thumb, lead times go up when utilization crosses around eighty percent.  High utilization is not good. We've probably experienced this when the highways begin to fill up with cars or when the CPU on the computer gets toward its capacity.

How do we create slack?

We could hire additional members on the team. Imagine going to the folks that handle the purse strings and telling them that we want ten developers in the team but we only want to keep eight developers working because we've heard that utilization  at above eighty percent is not good. Good luck with that :)

Or - we could apply WIP limits

If we have a team of ten and we've put an overall WIP limit of 8, and assuming only one person works on an item at a time, we've taken away work from a couple of developers. In other words, we've intentionally created slack (or  excess capacity). And slack is good because it will allow these two "idle" developers to do a few things such as pair up with other developers or help resolve issues at bottlenecks.

And funnily enough, this will help get more work done.

It's an enabler for a "Pull"system

A pull system is one where the downstream process pulls in work only when it's ready to process it. This will result in producing only what's needed, transferring work to a work station when needed and reduce inventories i.e. it will help result in a lean outcome.

When we have WIP limits, we cannot take on work unless  we have a "slot" available on the work station. We are consciously defining our capacity to take on work. Imagine if we did not have limits. Work would be "pushed" to us by the upstream process - whether we were ready or not. And  most likely, it would just wait  to be acted on. That's a waste.

WIP limits make Kanban a "Pull" system.

It's an enabler for Kaizen

It's not that we don't want to improve, but it's usually that we don't know where to start. Or what our bottlenecks/problems are. Lower work limits are a great enabler for continuous improvement (kaizen). When you run into the bottleneck, resist the temptation to immediately raise the WIP limit. Get a conversation started in the team and look to resolve the bottleneck. 

Pound the rocks to submission. And if you still have a bottleneck, raise the limit to enable flow.

Trust the board. The board does not lie.

BDD is not about tools

Many teams practicing BDD assume that the tool is the process. A lot of chatter is about: 

• What BDD tool (or framework) should I use?
• How should I use that tool?
• Is tool X better than tool Y?
• Etc., etc, etc. 

So much noise, that I’m afraid the intrinsic value of BDD is lost. Just because you are using a BDD tool, doesn’t mean you are practicing BDD. The tools are important, but not absolutely necessary. You could get a lot out of BDD without a tool. Here’s what I mean.

BDD helps build “Software that matters” 

In order to do that, you should capture requirements as behaviors resulting in a list of “behavior specifications”. In fact, that is what gathering software requirements is all about. You should define what the software should do (i.e. behavior) and frame it in the context of the business value.

But that’s hard.

Waterfall teams tend to write verbose documents, that developers find hard to understand. Agile teams err on the opposite side, writing terse user stories that often do not have enough information. BDD helps bridge the gap with the notion of User Stories that have two components: (a) Narrative and (b) Scenario.  The scenarios are the acceptance criteria and provide examples of how the software should behave. It helps to illustrate the narrative, providing a concise, yet vivid description of the expected business outcome.

You don’t need any new tools to do this.

It provides a language to document the behavior

The narrative is written in the format:

As A [role],

I want a [feature],

so that I [benefit])

The scenario is written in the format:

Given [context]

When [even]

Then [outcome]

This is a very structured way to express behavior.  “Given” describes the context or state of the system, “When” describes the user action or state transition and “Then” describes the outcome or expected behavior.  This expression almost feels like code. Yet it is in a natural language and more importantly, is in the language of the stakeholder. Because of the reduced ambiguity, you are likely to get what you are asking for. Especially, because you are also kind enough to tell the programmer/QA Analyst why a feature is required – a very important aspect of software requirements that is often ignored. Just adopt this format for your software requirements.

You don’t need any new tools to do this.

But it’s more important to discuss the behavior

Don’t just define behaviors, discuss them. Leverage the “Power of Three” i.e. the Business Analyst (a proxy for Customer or Product Owner), the QA Analyst and the Programmer in defining behavior. You can start with having the Business Analyst write the narrative and scenarios for the user story. But be sure to have the QA Analyst and the Programmer review the user story. Good QA people can elaborate the user story. Programmers can keep the specification realistic. Discuss it together, even if only for a brief time. These three stakeholders bring very different perspectives and expertise to the table. You’ll find that the scenarios get fortified with better acceptance-tests, including behaviors that were not originally considered. At the end of the conversation you will also have a definition of “done” for the user story.

You don’t need any new tools to do this.

It’s about baking quality in

Many defects occur because the requirements are ambiguous and subject to the programmer’s interpretation.  BDD fixes this through the scenarios with their rigid grammar. The scenarios provide the cues to the programmer to build the right features.  Defects also occur because the behaviors are not completely defined. This can be fixed by using the “Power of  Three” to inspect the user stories, discuss them and finalize them. This may take some time, but the investment will pay off. As the Toyota Production System says, “Inspection to find defects is waste. Inspection to prevent defects is essential”.

You don’t need any new tools to do this.

Engage the stakeholders

Leverage the “Power of Three” at every opportunity. Include the Business and QA analysts in the design and architecture conversations. Encourage the Programmer and the Business Analyst to review the test cases. When stakeholders, especially those that are more “outside” than “inside”, get involved in design conversations, they are more likely to drive a better understanding of the need and therefore influence the behavior.

You don’t need any new tools to do this.

However, you will need the BDD tools if…

You want to create “Executable Documentation” i.e. you want to record the behavior specifications in the BDD grammar in a BDD tool. Typically, these tools generate skeleton code, against which you write production code following TDD practices. Similar to the XUnit tools, initially tests will fail, and when the behavior is implemented correctly, the tests will pass.  This will provide traceability from the code to the business value.  More importantly, if you are disciplined, you will write just enough code to meet the behavior, thereby reducing waste.

In conclusion

A lot of the intrinsic value of BDD can be realized by:

-  Applying its User Story format

-  Documenting requirements using the recommended grammar

-  Leveraging the “Power of Three” to bake quality in the process

-  Engaging stakeholders to ratify and discover new behavior

And all of this can be done without BDD specific tools.

If you believe you are already following these best practices, then it's time to reach for the BDD tools. But not until then.

Stress in Agile teams

People working in agile teams sometimes complain that stress levels are higher in agile projects. Now, we all know that it is not supposed to be so. After all, don't agile projects distribute work more evenly for the duration of the project? Aren't agile teams supposed to be self organized and responsible for signing up for an appropriate load of work in a sprint (can't even blame management now)?

Agile experts are quick to point out that the increased stress, if that's even true, is most likely the fault of the agile coach / scrum master. Don't blame the agile methodology, they say. But then, there have always been good managers and bad managers and there certainly are good agile coaches and bad agile coaches. In many teams, the managers have now become scrum masters or agile coaches. When the same team, under the same manager that was practicing waterfall or iterative development is now practicing agile and is complaining about additional stress, could there be some truth to the statement?

Let's examine possible reasons for the stress and potential ways to mitigate them.

The Daily stand-up: This is the heart of many agile processes. This is also a potential stress point. Think about it. Day after day, you are asked to give an account of yourself in front of others. What did you do since your last stand-up? What do you plan to do today? Like it or not, you are subjecting yourself to intense scrutiny. There's no place to run and no place to hide. You cannot help wondering what your team mates think of you. You cannot help comparing yourself to Joe who always seems to get more done in the same time.

How to mitigate it: 
Emphasize to the team that skills and experience vary between individuals. Some are more skilled and experienced than others and may even be earning substantially more. Therefore, it is natural that they will produce more. Do not compare yourself with the performance of others. Just do the best you can.

Commitment:
 In agile, as in life, it's the many small promises that we make that adds to the pressure. In an agile process, specifically in the daily stand-up, we are making promises everyday. We call them commitments. It's human nature to want to meet those commitments. We are essentially honest people and we want to live up to our promises. Until we fulfill that promise, it's hanging over our heads like the proverbial sword of Damocles. And if we fail to keep the promise, we feel guilty. Even, if it's not entirely our fault.

How to mitigate it: 

Don't not use the word "commit". Instead, say "try". Don't say "I commit to do X". Instead say, "I'll try to do X". It's a subtle but important difference. Notice how there's no promise to deliver? Does that mean you are going to work any less harder? No. Remember, we are honest people. Our teams trust us.And we will do what we can to complete X. 

But we did not commit.So there is no expectation tomorrow that it will be done.We'll sleep better and perhaps even make it home in time for dinner.

Estimates:

When teams or individuals estimate tasks, they are implicitly committing to the hours. When we commit, we take on stress.

How to mitigate it:

Do not ask for estimates. Instead, attempt to "right size" the tasks using techniques like T-Shirt sizing. Measure the time taken to complete the task from the time it is started. The task will start when it does and complete when it does. As the team gathers these metrics they will also become more adept at "right sizing" the tasks. The eventual outcome is a steady velocity for the team.  And since the team is not providing estimate that they have to commit to, the stress is also less.And when the team is less stressed, they produce more.

Conclusion
Agile processes introduce stress in many subtle ways. Some of these are an outcome of the high visibility and transparency of the methodology - the very techniques that make agile so successful. This stress is not introduced intentionally - rather it is an unintended side effect. Being aware that the stress is real and tangible can help us mitigate it.

Unless we acknowledge it, we cannot manage it.

When Agile becomes Rigid

Agile teams are supposed to be agile. Yet, there are agile teams run by over zealous agile coaches, who in their quest to become agile have actually become rigid. In fact, very rigid. If you see the following signs, you are in one such team:

• It is the Agile coach's way or the highway. He/She will parrot prescriptions from various agile books on how it should actually be done. Forget the manual, coach. True agile is not supposed to be prescriptive. Good ideas can come from anywhere - include the team by your side that's still (Oh, my God!) practising iterative/waterfall.
• The agile coach will  not let a member of the team to participate in other important meetings/discussions in the organization, because "it will affect the team's commitments to the sprint!". Yes, the sprint is important but the organization is more important.
• You feel like you are followinng a cult rather than a software development methodology.
• The team is apparently "self organized" - or so the agile coach says. Repeatedly. And yet, all major decisions are from the agile coach.
• It's easier to stop the earth's rotation than to change the schedule of the Sprint ("we have committed these deliverables to the customer"). It doesn't matter that there are existing customer who probably want a fix urgently. Oh no, nothing can come in the way of the Sprint!
• All  the fun at work is lost. It's all about commitments and not letting your team down. And yet, ironically, there's a lot of talk about "the team is a family", "active socializing", "protecting the team member from context switching". etc. You didn't hear such talk in your "waterfall" team and yet you felt like it was a family. Strange, eh?
• Life is a series of Sprints. You are reduced to a Sprint Monkey going from one sprint to another. Even your schedule at home, revolves around these sprints ("I don't have time to throw the trash, dear. I'm in the middle of a sprint!").

If you are in one such team, remind your agile coach about agility. It's not enough to follow agile practises - you actually need to be agile.

Stop finding Defects !

Yes, please. Let's focus, instead, on "Preventing Defects".

We need to develop software faster and with better quality than ever before. We cannot afford to have an extensive "bug fixing" cycle. Finding defects towards the end of the software cycle is expensive. We all know that and yet many teams continue to do just that. This is perhaps the only industry where we tolerate that process and devote extensive time in the development cycle towards finding and fixing them. We wouldn't dream of a process like that while building houses or cars. Or even while cooking a meal at home. So why do we allow that while building software?

Could it be ignorance? Or complacence? Neither is acceptable in this day and age.

I'd like to suggest that we have tools and techniques at our disposal that will go a long way towards preventing defectings. What is lacking is belief that it's possible. Try it in the small and the belief will come.

Try BDD (Behavior Driven Development).

"Finding Defects" is an industrial age activity. "Preventing Defects" is a knowledge era activity. And in case you didn't notice, we are in the knowledge era.

BDD is more than “TDD done right”

BDD is more than just doing TDD right. Beginners in TDD struggle to understand that TDD is really about design and not testing. Similarly, newcomers to BDD may find it challenging to appreciate that BDD is also about collaboration between stakeholders and not just software behavior. You will find discussions on the internet on “BDD is TDD done right”. That’s certainly true, but there is much more to BDD. We will soon look at the additional value BDD offers, but first let us list why people think BDD is just TDD done right. 

1. TDD captures behavior or intentions just like BDD. You write a test case upfront to capture a behavior. However, the vocabulary is all test based so this is confusing. BDD fixes this by using the right vocabulary. Therefore, BDD is still TDD – but better. 
2. TDD captures low-level requirements (usually at the class/method level). BDD captures high-level requirements. Therefore, BDD is still TDD - but at a higher level.
3. BDD and TDD result in a suite of tests that we can run as a regression suite. Therefore, BDD is still TDD.

What then does BDD offer that is not in TDD? The value is largely in the collaborative process for defining requirements and the grammar and structure used for capturing them.

BDD captures requirements as user stories. A story has two parts – a narrative and scenarios. A requirement is not complete until it has both. We write Stories in BDD with a rigid structure. We write a narrative using a Role/Benefit/Value grammar and scenarios using a Given/When/Then grammar. Following a rigid structure like this can help adoption of a common vocabulary that leads to unambiguous requirements.

However, the benefit goes beyond that.

The crucial value that BDD offers is the process for creating the user stories. User stories are developed collaboratively by the key stakeholders – the Product Manager (or customer or Business Analyst), the QA resource and the developer. This level of collaboration results in:

1. Developing features that truly add business value.
2. Preventing defects rather than finding defects. Think how much easier it is to write code when you have scenarios that exemplify the positive and negative behaviors and make the intention transparent to the programmer.
3. Bring QA involvement to the forefront. This is great for team dynamics because in many agile processes QA personnel feel ignored.
4. Define executable, verifiable and unambiguous requirements.
5. Provide a better definition of “DONE” for agile development. 

However, the benefit goes beyond that.

BDD also helps you write “Software that matters”. What does this mean? Think of it as writing just enough code to meet a business requirement. You are probably thinking, “I could do that with TDD too”. Consider this. As you are writing Junit tests cases, is a stakeholder (Product Manage/Business Analyst/QA resource) with you? Probably not. Have you ever written code (probably excellent code, perhaps even following TDD practices), that was not required? Probably yes. You may not have shipped it, but you probably wrote it, found no use for it and deleted it. That is a waste. When you follow BDD, you are less likely to do this. You will reduce waste.

How can we write “Software that matters”?

The best way to do that is to leverage BDD and TDD. Here is an approach:

1. Write requirements as user stories using the BDD grammar/structure. Do this collaboratively with the key stakeholders.
2. Enter the User Stories (feature + scenarios) in a BDD tool.
3. Write code to map the User Stories to tests.
4. Write production code using TDD to make the tests pass. 

As you can see, BDD is not just TDD done right. You could use just the vocabulary of BDD to improve TDD but that would be like using only some of the benefits that BDD has to offer us. When we use the the strengths of both these techniques we will have “Software that matters” along with “Software that works”.

  

That is the promise of BDD.

Agile is not about speed

Studies have shown that agile methodologies shorten the software development cycle. Naturally, many assume that agile is all about speed. This is quite far from the truth.

What's the usual cause of delay in software projects? Quality. Or more accurately- lack of it. For example:

• Poor quality planning leads to project delays.
• Poor quality requirements, leads to poor implementation
• Poor quality implementation leads to defects which leads to delays
• Poor quality testing, leads to disaster

Agile methods infuse an intense focus on quality at all these levels.

• Planning is kept simple and straightforward to achieve sprint goals. Usually, the entire team participates in planning.
• Requirements are reviewed by all stakeholders and are kept minimal to achieve business value. The focus on business value ensures that only the “Minimal Marketable Features” make it to the sprint. Good requirements ensure that minimal code is written. We all know that the highest quality code is the code that isn’t written.
• Peer reviews, pair programming and Test Driven Development results in higher implementation quality.
• Quality is baked in at all levels. QA focuses on preventing defects rather than finding them.

A team that follows all this rigor will actually be writing fewer lines of code per day than other teams. Per conventional measurements, these teams are slower! However, the quality of the artifacts (all artifacts – not just the code) is much higher because it receives intense scrutiny. Enforcing quality at all levels has a happy side effect of speeding up projects. If you want to deliver more, and if you want to deliver it faster - slow down and focus on quality.

That's the essence of Agile methodologies.

Why are there so many Agile Coaches?

Makes me wonder.

Agile development must be hard. Why else would there be so many Agile coaches? When people were doing waterfall, I don't think there were that many "Waterfall Coaches". Think about it. How many of your contacts in the software world were "Waterfall Coaches"? None of my personal contacts were coaching waterfall. On the other hand, I know of at least half a dozen who are in this business - either fulltime or part time. And I'm running into many strangers who are Agile coaches. Or at least want to become one.

For a methodology that is supposed to be simple, why are there so many coaches? Perhaps it is not simple. Agile is supposed to simplify but that's not the same thing as being simple. Perhaps the highly adaptive nature of agile methodologies necessitates a coach.

Essentially - what does an agile coach do? If you are a team that's already practicing an agile methodology, she'll probably take a look at your methods and suggest improvements. Should I use the word prescribe changes? Funny. We choose a methodology that's not prescriptive for its ostensible benefits and then we hire a coach to prescribe methods.

The Rise of the BAQA ("BaaKwaa")

The role of the Business Analyst (BA) and Quality Assurance Engineer (QA) as we know it is changing. As agile methodologies become prevalent, they appear to have an overlap. I think of a person performing this new role as a BAQA (pronounced BaaKwaa). The need for people in agile teams to wear different hats is spurring this development. In recent times, this has received a boost with the development of frameworks supporting Behavior Driven Development (BDD) and Acceptance Test Driven Development (ATDD).

Agile teams, and in particular teams following BDD/ATDD, write user stories in place of verbose software requirements. The user story captures the essence of the requirement expressed in a "Role/Feature/Benefit" grammar. Subsequently, acceptance criteria are written as "Givens/Events/Outcomes" to exemplify the requirement. Together, they completely define the business requirement.While the user story requires a Business Analyst's skill, the acceptance criteria requires a Quality Assurance Engineer's skill. Essentially, we now have a need for a person who can play both roles.

Enter the BaaKwaa.

Let us see the typical responsibilities of a Business Analyst and QA Engineer in a traditional methodology, such as waterfall.

A Business Analyst:

• Understands the business domain
• Analyzes weaknesses and strengths and helps define new features
• Writes software requirements specifications for the development team

A Quality Analyst:

• Writes test cases
• Performs tests on software programs and finds defects
• Ensures the quality of the software

Here's a brief role description for a BaaKwaa in an agile team:

• Write the user story (thus capturing business requirements)
• Write the acceptance criteria (thus further expressing the requirements through examples)
• Write the acceptance criteria in a BDD/ATDD framework in the language of the framework (example: Fitnesses, Cucumber, StoryQ, etc)
• Verify that the implementation meets the acceptance criteria preferably through automated means.

I'm not suggesting that we are witnessing the demise of the specialist BA and QA. There is a role for the specialist and there always will be, just as we continue to have architects, DBA's and user interface specialists in agile teams. However, they will be required to drop their BA or QA hat quite often, and instead wear the BAQA hat.

Please welcome the BaaKwaa.

Agile: The Preacher and the Practitioner

In an Agile conference, you will find two types of attendees - the preachers and the practitioners. The preachers are the speakers, the trainers and the agile "experts". They believe implementing an agile methodology in a text book like fashion is the only way to do it. The practitioners are those who have started an agile process in their company, are struggling to make it work and are dealing with real organizational and people issues.They see value in the agile process but do not think it can be done in a textbook fashion in their organization.

Here is what, for example, some of the agile preachers believe:

• SCRUM-But is evil. Not implementing an aspect of SCRUM points to an impediment that must be fixed.
• You must have teams co-located.
• Autotmated testing (TDD), continuous integration, etc. is a must.
• The Product Owner is always available to answer the team's questions. 
• Team resources are fully dedicated to the sprint. 
• Deadlines can and should be stretched to acommodate quality.

Here is what, for example, some of the agile practitioners have experienced:

• SCRUM-But is a reality for organizations dealing with legacy products.
• Co-location is a luxury in today's era. Often, even in the same geographic location, some team members work from home.
• Automated testing, continuous integration is often not possible for legacy applications.
• The Product Owner (Product Manager in the real world) is often managing multiple projects. It is an illusion to assume the Product Owner is dedicated and available.
• Team resources - especially QA, DBA, Architects often work on multiple projects or agile teams concurrently.
• Deadlines are real. They are often final and products must meet them and make compromises in the process.

The Agile Preacher often comes off as a white collar intellectual mostly used to working in laboratories, where things are largely under control. The Agile Practitioner, comes off as a blue collar worker, just interested in getting the job done in the real world with real people and real challenges to face.

The truth, as usual, is in between. Agile Preachers must temper their language to factor reality. Agile Practitioners must try to change reality. And in this tension between the two groups is where we'll see real value being added to the Agile Process.

Agile Boston Open Space 2010

I attended Agile Boston Open Space 2010 (http://www.newtechusa.com/agileboston/events/AgileBostonOpen2010.asp) today. The event was at the Microsoft Building in Waltham, Boston. Of the many conferences that I have attended, I have no hesitation in saying this gave me the most value for money. Here's why:

1. Convenience: It was in my backyard - just a couple of miles from my workspot. It certainly helped that while the title had Boston in it, it wasn't in Boston. It was in Waltham (you can tell I don't like going to Boston).
2. Cost: Just $29. And that included breakfast and food. And yes - there was vegetarian food. It was just a wrap, but it was tasty. This is not a trivial matter. I'm a vegetarian and I have often found that some conferences that charge heavy fees don't pay much attention to the vegetarian fare. These people were thoughtful.
3. I got to hear Ken Schwaber for an hour.
4. I got to network with a number of smart people who were willing to teach, learn and share. I learn't about a number of misconceptions about Scrum. Just because it is highly adaptive doesn't mean that you shouldn't do some of the process that truly added value in waterfall. People didn't say that in so many words, but that was the gist of what I got from it. If it's valuable, keep it.
5. I learn't about "Open Space Technology" (http://www.openspaceworld.com/brief_history.htm) - a fabulous way to organize meetings very productively.

I think it was a day very well spent.

TDD is a Misnomer

We all know, at least those of us in the software community, that TDD stands for "Test Driven Development". That's a huge misnomer. Simply because TDD is much more than about tests. Yes - tests are a core part of TDD. But TDD is also about the following:

1. It's a way to capture behavior (think requirements)
2. It's a design tool (If it's easy to test, the resulting code will be elegant)
3. It captures high level documentation (think tools like agiledox)
4. It's a regression test suite.

The word "tests" in it leads to all kinds of confusion among TDD practitioners. What do I test? How do I test? What do I test first?

Happily, we now have Behavior Driven Development - which simply removes the confusion around TDD while taking it to a whole new level.

Yes - I'm beginning to like BDD a lot. It's a better TDD and more.

ScrumBut is not that bad

ScrumBut is a practice where teams implement partial apsects of Scrum. As in - "We did Scrum BUT we did not have daily standups". Scrum pracitioners look down on teams that practice ScrumBut and perhaps for good reason. As they argue, when a team does not implement a prescribed practice in Scrum, the team is allowing for a problem to fester.

For example, if a team finds that a Product Manager is unable/unwilling to participate in an agile team as a "Product Owner", using ScrumBut, the Business System Analyst may take on that role. The "Product Owner" is one of three key roles that Scrum prescribes. Not having the real Product Owner play that role is a violation of Scrum. In reality, there might be organizational hurdles, as in highly matrixed organizations, that prevent the Product Manager from participating in agile teams. Should the agile team give up their move towards adopting agile methodologies?

Perhaps not. ScrumBut is an excellent way to get started with agile methodologies. Agreed that it is not likely to give the same amount of return that Scrum promises. However, it is an incremental way to adopting Scrum. It is likely to result in less of a culture shock to new teams and may provide the breathing space to winning hearts and minds.

The end goal though must be to eliminate the "But" and have just "Scrum". Assuming of course, Scrum is what you want for an agile methodology.

Embrace “Automated Unit Testing” in Waterfall

Automated Unit Testing is a process where developers programmatically test their code using test frameworks such as the XUnit framework. Code is not considered complete unless it is accompanied by a test case. Agile teams have embraced unit testing. Waterfall teams can also do so without disruption to their process.

Introduction
Agile software development teams have reaped significant benefits by adopting “Automated Unit Testing” as a part of the development process. For teams that are considering becoming more agile, adopting “Automated Unit Testing” would be a good start. It can be easily embraced without significant impact on the process. “Automated Unit Testing” is a process where programmers write test cases for their code using testing frameworks such as the XUnit framework. Ideally, the test cases are written first followed by the implementation of the production code. When all the tests pass, the implementation is complete.


Challenges in “Automated Unit Testing”
More than the impact on the process, the challenge is in getting teams to adapt to this process. In waterfall process, development teams typically provide the implementation and throw it over the wall for QA to test. Asking programmers to prove that the code works before handing it over to QA is a significant paradigm shift. Writing tests is also hard – especially if the code has not been written in a testable way. Developers may have to spend as much time writing the test cases as the production code itself. To most developers, this seems like a waste. However, this pays off in terms of producing code with fewer defects.


Better late than never
It’s never too late to write tests. Tests are most effective if they are written at the time the production code is being written. However, you could introduce them even as defects are being fixed for existing code. When a defect is fixed, insist that tests be written for it. This allows for an incremental creation of test suites. Test suites allow for your code to be flexible. Their presence lets you re-factor with confidence.


Conclusion
Automated Unit Tests are a pillar of agile teams. For teams that want to adopt agile practices, this is a great way of adopting one of its pillars without disrupting existing processes. It’s never too late to adopt Automated Unit Testing. Embracing it in Waterfall also sets the stage for adopting the other important aspects of the Agile methodologies.

Embrace Upstream Testing in Waterfall

Upstream testing is a process where developers and testers work together from very early in the development cycle - even as features are being built. This is a flavor of an agile development methodology that can be embraced even in a waterfall process.


Introduction
Agile software development teams have reaped significant benefits by adopting the methodology over the waterfall process. However, there are significant challenges in moving from waterfall to agile. Most teams that make the change adopt a home grown hybrid model eventually leading to a complete agile methodology. If you are in a team that is practicing waterfall and you want to be more agile, consider adopting “Upstream Testing” in your current process.

How testing is usually done in waterfall
In ideal waterfall, features are delivered to QA after they are completely built - in one fell swoop. More commonly, features are delivered in interim builds to QA for testing. QA tests the interim build and reports defects on them. These defects are triaged, prioritized and assigned to developers for fixing in future builds. This is usually a slow and expensive process. Often times, the developers have moved on to other features building, sometimes, on the existing defects. Fixing the defect now is more expensive and risky. Additionally, in many situations QA is not able to complete the testing of features developed in the build. As additional builds are delivered, QA is often behind in testing. The later a defect is fixed, the higher the cost. Unfortunately, since the risk of fixing is also higher with the passage of time, many defects are deferred to a later release – which is effectively a “death sentence” for the defect. The reality is that it will never get fixed, resulting in an inferior product to the customer.

Embrace Upstream Testing
What can waterfall organizations to do in the short term to improve the quality of their deliverables? Adopt “Upstream Testing” in your process. “Upstream Testing” is a flavor of agile testing methodology that can be embraced without causing major disruption in existing processes. Very often it can be done without even the involvement of senior management.

In this process, QA works with software development even as features are being iteratively developed. As the developers are checking in code, the QA resources are testing the functionality on daily builds. The daily builds are not formal builds that are handed off to QA with all the bells and whistles. Instead, it is a build that is used by development to test the validity of their integration and check-ins. QA and development work off the same build. The real differentiator in this process is what happens when QA finds an issue. The tester simply reports this to the developer without registering the defect in the corporate defect tracking tool. The developer and tester agree on a fix and it is implemented right away. The tester tests the fix and the developer moves on to other features confident that not only has he unit tested it but that it has even been approved by a tester. When the formal build is done and handed off to QA for intensive and regression testing across many platforms, there are no surprises to QA and quality is already very high.

Note that there aren’t any significant deviations from how development and testing is done. The QA and development teams continue to report to their respective managers. The organization hierarchy is preserved while still injecting agility into the process. Apart from the development and QA teams, hardly any other organization is functionally affected – in contrast to a full agile methodology which has a ripple effect throughout the organization.

Benefits of Upstream Testing
Since the tester and developer are working together, the quality of the deliverable is being monitored continuously – almost on a daily basis. Defects are found and fixed early. It is extremely cost effective to the organization to have this done without going through defect reporting, triages, defect assignment, Change Control Board’s and such ceremonies. Defects that might get deferred are actually fixed – because it is possible to do so. The result is a higher quality product, delivered earlier in a more cost effective manner. The social engineering aspects of “Upstream Testing” cannot be underestimated. The tester gets to understand the complexity of development and the developer begins to appreciate the value of the tester in making the developer look good.
Development and QA start beginning to look like a team working towards a common goal.

Operational Challenges in Upstream Testing
Many organizations are set up functionally. Thus, the QA and development teams report to different directors who may ultimately report to the VP of Engineering. The very nature of this organization sets up different priorities for QA and development. Very often, one of the metrics used to measure the performance of QA teams is the number of defects filed per release of a product. It is not unusual to see colorful graphs of defects found outside the QA director’s office while the development director’s office proudly displays the defects fixed.

In “Upstream Testing” the defects filed are far fewer – because they are caught and fixed informally. The management of QA and development need to arrive at an understanding of a more mature and relevant measure of performance. This can often be challenging and needs good understanding and common goals between development and QA managers.

Conclusion
Upstream Testing is an agile practice that can be fairly easily introduced to teams following waterfall. It is not disruptive and fits in well with waterfall processes. It allows for defects to be caught earlier resulting in cost efficiencies. Apart from higher quality deliverables, the effect of the social engineering it introduces between development and test teams is high. If your team is following a waterfall process and looking to adopt more agile processes, consider “Upstream Testing”. It is an enabler for moving towards full fledged agile methodology in the future, while solving real problems today.

Embrace the daily stand-up in Waterfall

The Daily Stand-up meeting is the heartbeat of Scrum, an Agile methodology. This is a process that can be easily embraced by teams following the waterfall process.

Introduction
Agile software development teams have reaped significant benefits by adopting the Scrum methodology. However, there are significant challenges in moving from a Waterfall process to an Agile process. Most teams that make the change adopt a home grown Hybrid model eventually leading to a complete Agile methodology. It has been argued by many, that a swift change from one process to another is better than a gradual change. While there are merits to the argument, the reality is that most organizations and people take time to embrace change. It is better to take the time to win hearts and minds than to force adoption of a new methodology. Change that is willingly embraced is change that can be sustained.

Let’s say you want to adopt some Agile methodologies in your team that is currently following the waterfall process. What aspect of it should you adopt? My suggestion is to introduce the “Daily Stand-up” in your team. The “Daily Stand-up” is the heartbeat of the Scrum Agile methodology. See here for an explanation of this process:

http://www.mountaingoatsoftware.com/daily-scrum

Ask the team if they are willing to try this out for a couple of weeks. Also choose a time that is most convenient to them. Most teams will be willing to try this out. You can (and I recommend that you should) even offer the following incentives to the team, if it is in your power:

Cancel the weekly status reports
Most programmers dislike writing status reports. They’d rather write code. With the daily stand-up you probably don’t need weekly reports anymore. You are getting the status daily! You can use the information at the daily stand-up to update the project plan and produce a burndown chart to track progress.

See here for description of a burndown chart:
http://www.mountaingoatsoftware.com/release-burndown

Your team members will love you for canceling the weekly status report!

Cancel the status meeting
Most waterfall teams have weekly status meetings that last for an hour or more. Most programmers dislike long meetings. Since you are getting a daily update, you probably don’t need this meeting anymore. Cancel it.

Your team members will love you for canceling the weekly status meeting!

What’s the gain from the daily Stand-up?
As a Project Manager you have a daily update on the progress. This is better than the weekly update from the original status meetings. You have information early that you can act upon. There is less formality in a stand-up – programmers tend to like that. Information exchanged in a stand-up often results in offline warm & fuzzy communication.

Conclusion
The daily stand-up contributes heavily to the success of an Agile methodology such as Scrum. If the 80/20 rule were to be applied to Scrum, the daily stand-up would show up on the “20” side. It can be easily incorporated into the Waterfall process with all the resulting benefits. Embracing it in Waterfall also sets the stage for adopting the other important aspects of the Scrum Agile methodology.