When Agile becomes Rigid

Agile teams are supposed to be agile. Yet, there are agile teams run by over zealous agile coaches, who in their quest to become agile have actually become rigid. In fact, very rigid. If you see the following signs, you are in one such team:

• It is the Agile coach's way or the highway. He/She will parrot prescriptions from various agile books on how it should actually be done. Forget the manual, coach. True agile is not supposed to be prescriptive. Good ideas can come from anywhere - include the team by your side that's still (Oh, my God!) practising iterative/waterfall.
• The agile coach will  not let a member of the team to participate in other important meetings/discussions in the organization, because "it will affect the team's commitments to the sprint!". Yes, the sprint is important but the organization is more important.
• You feel like you are followinng a cult rather than a software development methodology.
• The team is apparently "self organized" - or so the agile coach says. Repeatedly. And yet, all major decisions are from the agile coach.
• It's easier to stop the earth's rotation than to change the schedule of the Sprint ("we have committed these deliverables to the customer"). It doesn't matter that there are existing customer who probably want a fix urgently. Oh no, nothing can come in the way of the Sprint!
• All  the fun at work is lost. It's all about commitments and not letting your team down. And yet, ironically, there's a lot of talk about "the team is a family", "active socializing", "protecting the team member from context switching". etc. You didn't hear such talk in your "waterfall" team and yet you felt like it was a family. Strange, eh?
• Life is a series of Sprints. You are reduced to a Sprint Monkey going from one sprint to another. Even your schedule at home, revolves around these sprints ("I don't have time to throw the trash, dear. I'm in the middle of a sprint!").

If you are in one such team, remind your agile coach about agility. It's not enough to follow agile practises - you actually need to be agile.

Stop finding Defects !

Yes, please. Let's focus, instead, on "Preventing Defects".

We need to develop software faster and with better quality than ever before. We cannot afford to have an extensive "bug fixing" cycle. Finding defects towards the end of the software cycle is expensive. We all know that and yet many teams continue to do just that. This is perhaps the only industry where we tolerate that process and devote extensive time in the development cycle towards finding and fixing them. We wouldn't dream of a process like that while building houses or cars. Or even while cooking a meal at home. So why do we allow that while building software?

Could it be ignorance? Or complacence? Neither is acceptable in this day and age.

I'd like to suggest that we have tools and techniques at our disposal that will go a long way towards preventing defectings. What is lacking is belief that it's possible. Try it in the small and the belief will come.

Try BDD (Behavior Driven Development).

"Finding Defects" is an industrial age activity. "Preventing Defects" is a knowledge era activity. And in case you didn't notice, we are in the knowledge era.

BDD is more than “TDD done right”

BDD is more than just doing TDD right. Beginners in TDD struggle to understand that TDD is really about design and not testing. Similarly, newcomers to BDD may find it challenging to appreciate that BDD is also about collaboration between stakeholders and not just software behavior. You will find discussions on the internet on “BDD is TDD done right”. That’s certainly true, but there is much more to BDD. We will soon look at the additional value BDD offers, but first let us list why people think BDD is just TDD done right. 

1. TDD captures behavior or intentions just like BDD. You write a test case upfront to capture a behavior. However, the vocabulary is all test based so this is confusing. BDD fixes this by using the right vocabulary. Therefore, BDD is still TDD – but better. 
2. TDD captures low-level requirements (usually at the class/method level). BDD captures high-level requirements. Therefore, BDD is still TDD - but at a higher level.
3. BDD and TDD result in a suite of tests that we can run as a regression suite. Therefore, BDD is still TDD.

What then does BDD offer that is not in TDD? The value is largely in the collaborative process for defining requirements and the grammar and structure used for capturing them.

BDD captures requirements as user stories. A story has two parts – a narrative and scenarios. A requirement is not complete until it has both. We write Stories in BDD with a rigid structure. We write a narrative using a Role/Benefit/Value grammar and scenarios using a Given/When/Then grammar. Following a rigid structure like this can help adoption of a common vocabulary that leads to unambiguous requirements.

However, the benefit goes beyond that.

The crucial value that BDD offers is the process for creating the user stories. User stories are developed collaboratively by the key stakeholders – the Product Manager (or customer or Business Analyst), the QA resource and the developer. This level of collaboration results in:

1. Developing features that truly add business value.
2. Preventing defects rather than finding defects. Think how much easier it is to write code when you have scenarios that exemplify the positive and negative behaviors and make the intention transparent to the programmer.
3. Bring QA involvement to the forefront. This is great for team dynamics because in many agile processes QA personnel feel ignored.
4. Define executable, verifiable and unambiguous requirements.
5. Provide a better definition of “DONE” for agile development. 

However, the benefit goes beyond that.

BDD also helps you write “Software that matters”. What does this mean? Think of it as writing just enough code to meet a business requirement. You are probably thinking, “I could do that with TDD too”. Consider this. As you are writing Junit tests cases, is a stakeholder (Product Manage/Business Analyst/QA resource) with you? Probably not. Have you ever written code (probably excellent code, perhaps even following TDD practices), that was not required? Probably yes. You may not have shipped it, but you probably wrote it, found no use for it and deleted it. That is a waste. When you follow BDD, you are less likely to do this. You will reduce waste.

How can we write “Software that matters”?

The best way to do that is to leverage BDD and TDD. Here is an approach:

1. Write requirements as user stories using the BDD grammar/structure. Do this collaboratively with the key stakeholders.
2. Enter the User Stories (feature + scenarios) in a BDD tool.
3. Write code to map the User Stories to tests.
4. Write production code using TDD to make the tests pass. 

As you can see, BDD is not just TDD done right. You could use just the vocabulary of BDD to improve TDD but that would be like using only some of the benefits that BDD has to offer us. When we use the the strengths of both these techniques we will have “Software that matters” along with “Software that works”.

  

That is the promise of BDD.

Agile is not about speed

Studies have shown that agile methodologies shorten the software development cycle. Naturally, many assume that agile is all about speed. This is quite far from the truth.

What's the usual cause of delay in software projects? Quality. Or more accurately- lack of it. For example:

• Poor quality planning leads to project delays.
• Poor quality requirements, leads to poor implementation
• Poor quality implementation leads to defects which leads to delays
• Poor quality testing, leads to disaster

Agile methods infuse an intense focus on quality at all these levels.

• Planning is kept simple and straightforward to achieve sprint goals. Usually, the entire team participates in planning.
• Requirements are reviewed by all stakeholders and are kept minimal to achieve business value. The focus on business value ensures that only the “Minimal Marketable Features” make it to the sprint. Good requirements ensure that minimal code is written. We all know that the highest quality code is the code that isn’t written.
• Peer reviews, pair programming and Test Driven Development results in higher implementation quality.
• Quality is baked in at all levels. QA focuses on preventing defects rather than finding them.

A team that follows all this rigor will actually be writing fewer lines of code per day than other teams. Per conventional measurements, these teams are slower! However, the quality of the artifacts (all artifacts – not just the code) is much higher because it receives intense scrutiny. Enforcing quality at all levels has a happy side effect of speeding up projects. If you want to deliver more, and if you want to deliver it faster - slow down and focus on quality.

That's the essence of Agile methodologies.

Why are there so many Agile Coaches?

Makes me wonder.

Agile development must be hard. Why else would there be so many Agile coaches? When people were doing waterfall, I don't think there were that many "Waterfall Coaches". Think about it. How many of your contacts in the software world were "Waterfall Coaches"? None of my personal contacts were coaching waterfall. On the other hand, I know of at least half a dozen who are in this business - either fulltime or part time. And I'm running into many strangers who are Agile coaches. Or at least want to become one.

For a methodology that is supposed to be simple, why are there so many coaches? Perhaps it is not simple. Agile is supposed to simplify but that's not the same thing as being simple. Perhaps the highly adaptive nature of agile methodologies necessitates a coach.

Essentially - what does an agile coach do? If you are a team that's already practicing an agile methodology, she'll probably take a look at your methods and suggest improvements. Should I use the word prescribe changes? Funny. We choose a methodology that's not prescriptive for its ostensible benefits and then we hire a coach to prescribe methods.

The Rise of the BAQA ("BaaKwaa")

The role of the Business Analyst (BA) and Quality Assurance Engineer (QA) as we know it is changing. As agile methodologies become prevalent, they appear to have an overlap. I think of a person performing this new role as a BAQA (pronounced BaaKwaa). The need for people in agile teams to wear different hats is spurring this development. In recent times, this has received a boost with the development of frameworks supporting Behavior Driven Development (BDD) and Acceptance Test Driven Development (ATDD).

Agile teams, and in particular teams following BDD/ATDD, write user stories in place of verbose software requirements. The user story captures the essence of the requirement expressed in a "Role/Feature/Benefit" grammar. Subsequently, acceptance criteria are written as "Givens/Events/Outcomes" to exemplify the requirement. Together, they completely define the business requirement.While the user story requires a Business Analyst's skill, the acceptance criteria requires a Quality Assurance Engineer's skill. Essentially, we now have a need for a person who can play both roles.

Enter the BaaKwaa.

Let us see the typical responsibilities of a Business Analyst and QA Engineer in a traditional methodology, such as waterfall.

A Business Analyst:

• Understands the business domain
• Analyzes weaknesses and strengths and helps define new features
• Writes software requirements specifications for the development team

A Quality Analyst:

• Writes test cases
• Performs tests on software programs and finds defects
• Ensures the quality of the software

Here's a brief role description for a BaaKwaa in an agile team:

• Write the user story (thus capturing business requirements)
• Write the acceptance criteria (thus further expressing the requirements through examples)
• Write the acceptance criteria in a BDD/ATDD framework in the language of the framework (example: Fitnesses, Cucumber, StoryQ, etc)
• Verify that the implementation meets the acceptance criteria preferably through automated means.

I'm not suggesting that we are witnessing the demise of the specialist BA and QA. There is a role for the specialist and there always will be, just as we continue to have architects, DBA's and user interface specialists in agile teams. However, they will be required to drop their BA or QA hat quite often, and instead wear the BAQA hat.

Please welcome the BaaKwaa.

Agile: The Preacher and the Practitioner

In an Agile conference, you will find two types of attendees - the preachers and the practitioners. The preachers are the speakers, the trainers and the agile "experts". They believe implementing an agile methodology in a text book like fashion is the only way to do it. The practitioners are those who have started an agile process in their company, are struggling to make it work and are dealing with real organizational and people issues.They see value in the agile process but do not think it can be done in a textbook fashion in their organization.

Here is what, for example, some of the agile preachers believe:

• SCRUM-But is evil. Not implementing an aspect of SCRUM points to an impediment that must be fixed.
• You must have teams co-located.
• Autotmated testing (TDD), continuous integration, etc. is a must.
• The Product Owner is always available to answer the team's questions. 
• Team resources are fully dedicated to the sprint. 
• Deadlines can and should be stretched to acommodate quality.

Here is what, for example, some of the agile practitioners have experienced:

• SCRUM-But is a reality for organizations dealing with legacy products.
• Co-location is a luxury in today's era. Often, even in the same geographic location, some team members work from home.
• Automated testing, continuous integration is often not possible for legacy applications.
• The Product Owner (Product Manager in the real world) is often managing multiple projects. It is an illusion to assume the Product Owner is dedicated and available.
• Team resources - especially QA, DBA, Architects often work on multiple projects or agile teams concurrently.
• Deadlines are real. They are often final and products must meet them and make compromises in the process.

The Agile Preacher often comes off as a white collar intellectual mostly used to working in laboratories, where things are largely under control. The Agile Practitioner, comes off as a blue collar worker, just interested in getting the job done in the real world with real people and real challenges to face.

The truth, as usual, is in between. Agile Preachers must temper their language to factor reality. Agile Practitioners must try to change reality. And in this tension between the two groups is where we'll see real value being added to the Agile Process.

Agile Boston Open Space 2010

I attended Agile Boston Open Space 2010 (http://www.newtechusa.com/agileboston/events/AgileBostonOpen2010.asp) today. The event was at the Microsoft Building in Waltham, Boston. Of the many conferences that I have attended, I have no hesitation in saying this gave me the most value for money. Here's why:

1. Convenience: It was in my backyard - just a couple of miles from my workspot. It certainly helped that while the title had Boston in it, it wasn't in Boston. It was in Waltham (you can tell I don't like going to Boston).
2. Cost: Just $29. And that included breakfast and food. And yes - there was vegetarian food. It was just a wrap, but it was tasty. This is not a trivial matter. I'm a vegetarian and I have often found that some conferences that charge heavy fees don't pay much attention to the vegetarian fare. These people were thoughtful.
3. I got to hear Ken Schwaber for an hour.
4. I got to network with a number of smart people who were willing to teach, learn and share. I learn't about a number of misconceptions about Scrum. Just because it is highly adaptive doesn't mean that you shouldn't do some of the process that truly added value in waterfall. People didn't say that in so many words, but that was the gist of what I got from it. If it's valuable, keep it.
5. I learn't about "Open Space Technology" (http://www.openspaceworld.com/brief_history.htm) - a fabulous way to organize meetings very productively.

I think it was a day very well spent.

TDD is a Misnomer

We all know, at least those of us in the software community, that TDD stands for "Test Driven Development". That's a huge misnomer. Simply because TDD is much more than about tests. Yes - tests are a core part of TDD. But TDD is also about the following:

1. It's a way to capture behavior (think requirements)
2. It's a design tool (If it's easy to test, the resulting code will be elegant)
3. It captures high level documentation (think tools like agiledox)
4. It's a regression test suite.

The word "tests" in it leads to all kinds of confusion among TDD practitioners. What do I test? How do I test? What do I test first?

Happily, we now have Behavior Driven Development - which simply removes the confusion around TDD while taking it to a whole new level.

Yes - I'm beginning to like BDD a lot. It's a better TDD and more.

ScrumBut is not that bad

ScrumBut is a practice where teams implement partial apsects of Scrum. As in - "We did Scrum BUT we did not have daily standups". Scrum pracitioners look down on teams that practice ScrumBut and perhaps for good reason. As they argue, when a team does not implement a prescribed practice in Scrum, the team is allowing for a problem to fester.

For example, if a team finds that a Product Manager is unable/unwilling to participate in an agile team as a "Product Owner", using ScrumBut, the Business System Analyst may take on that role. The "Product Owner" is one of three key roles that Scrum prescribes. Not having the real Product Owner play that role is a violation of Scrum. In reality, there might be organizational hurdles, as in highly matrixed organizations, that prevent the Product Manager from participating in agile teams. Should the agile team give up their move towards adopting agile methodologies?

Perhaps not. ScrumBut is an excellent way to get started with agile methodologies. Agreed that it is not likely to give the same amount of return that Scrum promises. However, it is an incremental way to adopting Scrum. It is likely to result in less of a culture shock to new teams and may provide the breathing space to winning hearts and minds.

The end goal though must be to eliminate the "But" and have just "Scrum". Assuming of course, Scrum is what you want for an agile methodology.