I’ve been thinking about the problems of applying XP to whole organizations. Turns out this is ground thoroughly covered by the lean production folks.

Here is a paper I found helpful: www.nwlean.net/no_harm.zip The premise is that every step of the transformation must pay for itself.

Getting people to understand that their problems are tractable can be hard, I’ve found. Rob Mee was talking to a guy about TDD on a gig of ours recently. "Sure, yeah, TDD is a great idea." "Why don’t you do it?" "It’ll make us go slower." "But it makes you go faster." "Yes, of course, but it’ll make us go slower." "But it’ll make you go faster" *iterate N times for annoying large N* "Okay, it makes people go faster everywhere but here."

As part of my interview, I was required to sit and pair program for about half an hour. We worked on an writing a unit test for an actual defect that currently existed on the project (although it could easily have been a real user story, or a contrived scenario if the project had not yet started).

I thought this was a great idea, and a great source of knowledge for both sides. I was able to show that I did indeed know how to program, write unit tests, knew my way around an IDE, had acceptable interpersonal communication skills, etc. I was also able to get a different perspective on what the team dynamics were like, which I could not have gotten from a formal interview setting.

The interesting thing is that both me and my partner (one of the interviewers) taught each other about some tools and approaches that we were not previously aware of.

Even though I didn’t get the job (the position was withdrawn), I kept in touch and became friends with the interviewer/partner, and the things we taught each other came in useful in our future development work.

This company also asked for code samples and a mini-presentation, which I also thought was a great idea for separating the wheat from the chaff.

Since I have had responsibility for helping interview, select and recommend job candidates in the past, I know for a fact that the best resume and interview performance in the world is inconclusive. You can still get a lemon, even though the lemon may be very good at piling on the BS.

From my perspective as a job candidate, I am confident in my skills and my abilities. I know that I can quickly adapt and excel in any position within my skill set. However, its very frustrating when I cannot convince the potential employer of this through only a traditional resume and interview.

In future interviews that I go for (which will hopefully only be with XP/Agile shops), I am going to suggest this as a way for the hiring company to get a better idea of my skills, knowledge and abilities, both technical and interpersonal. If I am ever part of a hiring team in the future, I will definitely propose that code samples and a pair-programming session be part of the interview process for candidates who make it to the final stages. This is admittedly very time-consuming, but probably much less net investment than being forced to live with (or try to get rid of) an employee who looked much better "on paper".

Thanks, Chad

 >Because it is important for the customer to have an idea of how much
 >> everything will cost ...

 In 99% of the times customer neither needs nor wants "everything"! One
 of the big dangers of BRUF is that it imposes the wrong feeling that
 "everything" is important. This is a major distraction that inevitably
 leads to scope-creep and eventually project failures. When we have the
 first meeting with a new customer we ask the following 3 questions:
 1. What is your biggest pain?
 2. If we solve this and only this pain, will your life get better?
 3. Are you willing to pay X amount of Euros to us to solve this pain.

 If any of these questions is answered with "no" we just thank for the
 coffee and walk away. If all 3 questions are answered with yes, we move
 to the first planning game...

 Put it in another way: I do believe it is extremely dishonest and
 incorrect behavior to conduct 9 months BRUF only to reach the
 conclusion that the customer does not have enough resources to
 continue. It is dishonest because as I wrote in an earlier posting,
 _all_ good developers I know are able to estimate almost immediately
 the scale of any software project without conducting BRUF.

For developers, XP allows you to focus on coding and avoid needless paperwork and meetings. It provides a more social atmosphere, more opportunities to learn new skills, and a chance to go home at a decent hour each night. It gives you very frequent feelings of achievement, and generally allows you to produce code that you feel good about.

For the Customer, XP creates working software faster, and that software tends to have very few defects. It allows you to change your mind whenever you need to, with minimal cost and almost no complaining from the developers. It produces reliable estimates so you can coordinate your schedule easier.

For management, XP delivers working software for less money, and the software is more likely to do what the end users actually want. It cuts risk in a couple ways: 1) It allows you to "pull the plug" on development at almost any time, and still have highly valuable code, and probably even a valuable working (if incomplete) application. 2) It reduces your dependence on individual superstars, and at the same time can improve employee satisfaction and retention.

The biggest disadvantage: It’s hard. It’s difficult to get many developers to accept the practices, and it takes a lot of discipline to keep doing them all. Customers may not like the idea of having to be so involved. Management may expect fixed-cost, fixed-scope estimates, which XP teams often refuse to create (because they are usually incorrect with any methodology).

Also, certain people may feel their jobs are being threatened, particularly architects, testers, and project managers. "Cowboy" coding "superstars" may dislike the reduction in fame, attention, and adreneline from "saving" the project at the last minute.

 I assume this popped up on everyone else's RSS aggregator as well, but...
  From Darrell Norton's blog [1] and Artima developer forum [2]: the
  current issue of CIO magazine is all about agility:
  http://www.cio.com/archive/081504/index.html

The Cook’s Tour offers an excellent howto to get yourself and your customers into the test-writing mode.

An intro article by Bill Wake.

Then I sit here for 3 hours trying to understand why the dammed computer does not do what it should. It’s that feeling I hate most. You waste time .. I mean I could as well go skiing or drink a bottle of vodka .. would be about the same productivity progress on my code and at least I would enjoy the sun.

After hitting my head long enough and starting to isolate the stuff .. I found it .. I had forgotten one return in a most trivial three line long method. Shame on me. Now I will go back to being test-infected. test-first is even better. Dammit .. sorry for the rant, but I doubt there are many systems like computers where one comma at the wrong place can make everything go boom. Oh well, try to modify the DNA and do not know what you are doing. :-).

Been coding hard now on SW and CFaR. I would really need some good vodka now before getting up early tomorrow morning to catch the train.

Hope you all had a good weekend, -A.

This vodka bottle reminds me that I am way behind on Futurometer. We will kick ass there soonish!

About the $200M Oracle-Ford Desaster: the management did not give them enough power to go fully agile

Chet Hendrickson:

 It was a very frustrating situation.  The team asked Don and me
 to help with preventing the Oracle consutants from making changes
 directly to the production system.  This was about $150 million into the project.

 We tried to sell them on a more agile approach (as you might imagine), but by
 this time they were pretty far gone.

 It was unfortunante that we were not operating at a level in the organization
 that would have allowed us to get the plug pulled sooner.

 chet

Georg Tuparev has a nice case story, too: speak out if you are put on a death march.

 Few years ago I was called to lead a huge team stuck in one and half
 year design phase. The team was supposed to build a control software
 for a network of telecom satellites. Cannot disclosed names and
 resources, but one could imagine ...

 Three days in the project I had a phone conference with the CxO's of
 both companies involved. Told them that the way it is going no
 satellite will ever fly and that I know a better way. After getting
 green line, the design document was burned with a small celebration at
 a BBQ, and 85% of the initial team members were sent to an indefinitely
 long vacation. With the rest (15%) of the team we had the first
 functioning version 2 months ahead of the schedule and 50% lower then
 expected expenses.

 So the lessons:
 - it is never late to change direction of a project in order to save it.
 - I do not agree with Kent that this is a sad story. If you are a good
 programmer put on a death march project you should speak out! If no one
 listens - walk away. There is just one very precious life in front of
 us - do not waste it! And if these folks wasted 2 years of their lives,
 well, it is their business ... but they should not expect my
 sympathies.

 BTW, Philip is right - the project I was telling about had a 9 months x
 40 people "Big Requirements Up Front"!!! Then the design started...

 Just my 0.02

 Georg Tuparev

Sabre Airline Solutions had many years of experience with its AirFlite Profit Manager, a big modeling and forecasting package that many airlines use to wring more income out of flight schedules. Even so, Release 8 of the software was four months late in 2000 after final system testing turned up 300 bugs. The first customer found 26 more bugs in the first three days of its acceptance testing, and subsequent joint testing by Sabre and the customer uncovered an additional 200 defects. www.computerworld.com/softwaretopics/software/story/0,10801,91646,00.html

 "Without test-first and refactoring, clients think
 they must assemble as many program requirements as
 they can afford to have written. This effort snarls
 all relative business priorities together, making
 Scope Control impossible. It obscures opportunities
 for simplification. Designing and implementing many
 features all at once is very hard, leading to our
 industry's reputation for very large failures. Putting
 tests in front of development's inner cycle permits an
 outer cycle of incremental feature growth. That
 relieves the Customer of the responsibility to predict
 the future and guess which complete set of features
 will maximize productivity."

.. Juergens answer:

 Nice description.
 Sometimes I try to explain that to non technical people
 with the following picture:

 If you have only one chance to get your wishes on a
 list, it is like Christmas for a child. You make sure
 you get every little wish on that list and hope for the best.

 If you are sure that you can get your wishes on the list
 at any time. You will just put the most important ones
 there which come to your mind easily.

 But anyway, when I last asked Kent what XP is, he said
 "XP is a community of software development practices based on
 values of simplicity, communication, feedback, and courage".
 That was about two or three years ago.

 I look forward to seeing the second edition as well.
 I'm sure it will be enlightening ...

When an organization wants more than the Agile process requires, it is easy to show the additional cost each iteration, and as the team builds credibility the customer/stakeholder(s) might be less inclined to want to spend money each iteration producing documentation.

So have some courage, tell your stakeholders that they can have it (whatever "it" is) if they want it and then make sure you follow up with giving them a good picture of the cost and value they get from "it". It’s their money - let them blow it if they want to. Let them slow the velocity by asking for non-code related items. All you can do is make it visible and help them down the path of figuring it out.

The sooner you start delivering some working code, the sooner you’ll have the credibility to address their real process issues. "It" will become less important the more iterations you deliver working tested code.

The hard part is getting code started, right? Maybe you can disguise the first few iterations as "prototyping" and use TDD as the process for your prototyping? :-)

1) Simplicity-the design must be simple, both in implementation and interface. It is more important for the interface to be simple than the implementation.

2) Correctness-the design must be correct in all observable aspects. Incorrectness is simply not allowed.

3) Consistency-the design must not be inconsistent. A design is allowed to be slightly less simple and less complete to avoid inconsistency. Consistency is as important as correctness.

4) Completeness-the design must cover as many important situations as is practical. All reasonably expected cases must be covered. Simplicity is not allowed to overly reduce completeness.

www.jwz.org/doc/worse-is-better.html

(Source: XP-mailinglist; thoughtful post by Ken Schwaber) Scrum is purely a project/product management process that can be applied to software projects, hardware projects, marketing projects, and any mix of the above. It does not contain engineering practices or any specific work practices. It is instead a way to maximize the ROI of work.

People who use Scrum in software development environment often adopt one or more of the XP practices to improve the engineering practices of the develpment organization. Scrum calls for an increment of potentially shippable product functionality every iteration. This means a fully cleaned up, refactored, tested, documented ready to go increment. Many organizations are incapable of this. It is easy to bring in XP practices because they are excellent and both Scrum and XP are pretty radically agile.

Project managers using Scrum are called ScrumMasters to differentiate the type of work they do ... they facilitate, manage the process, and optimize the team's productivity. They don't tell the team what to do, nor do they set of pairs of programmers, or parse out user stories. During the iteration, the team is entirely self- organizing ... whether it is doing software development or anything else related to the increment of functionality they are building.

XP seems to focus on team productivity... doing something the right way and as productively as possible. Scrum does this some, but instead focuses more on doing the right thing, getting ROI from building the 20% of the functionality that is necessary to get the value and maybe not building the rest.

We've implemented Scrum without telling the customer, users, or stakeholders. We've done this in one day. We seduce them into iterative, incremental development where they collaborate with us on what to do next. XP seems to require a steeper implementation curve with less acceptance from the users and customers. Scrum can't keep the customers away.

Estimating is a subtle diffrence that points out the XP and Scrum dividing point. XP works hard to estimate very finely defined stories, and to measure and improve these estimates. Scrum keeps the requirements more broad, more in general user terms that are analyzed during the iteration. Estimating isn't as important. The team does what it can, and gets better and better at figuring out how much it can do each iteration as it learns each other, the business domain, and the technology - iteration by iteration. We care more about delivering business value that having defensible estimates, which become meaningless in a collaborative setting.

Scrum also has a formal methodology that lays out how to scale Scrum to any sort of project with any number of people in any number of locations ... all based on the optimized 7 person team. This is an important mangement requirement, but not so important to an engineering discipline like XP.

I feel we are blessed to have such compatible practices and processes to apply to our software engineering projects.

This from a lean manufacturing consultant:

Find the simple path to what works and follow it, always looking for a simpler path.

Patrick D. Smith