Scott is a Toronto-native and long-time Smalltalker, though he has also done work on the dark side (Java). He give a similar talk to this at a database conference and some people walked out, they were so unwilling to even think about agile. Scott began by showing a picture of Mo (the Iraqi Information Minister) and asking if anyone in the hall sometimes felt they were working work for him :-)). A little picture of Mo appeared on all the subsequent 'XP doesn't work' slides as Scott's comment on the quality and honesty of some of the arguments he was rebutting. He sees a lot of lies about agility on the newsgroups; false assertions made by people who haven't a clue and have no clue that they have no clue. He aimed to present what he thinks the real story and talk about what else beyond XP is needed.
No bureaucracy does not mean no modelling. Agility does not mean code hacking. There are lots of low-end developers hacking code and saying they're doing XP. IEEE computer last month had an excellent article giving evidence that iterative minimal-bureaucracy development does work. Design is so important to agile developers that they do it every day, not in a swiftly-forgotten phase at the start. XPers do a lot of work planning in the planning game; they just don't waste time drawing an MS project gant chart because that's useless bureaucracy.
75% of project teams are less than 10 people so naturally most XP teams are less than 10 people. This is not evidence that XP, any more than traditional methods, has a problem scaling to large projects.
Agile methods focus on software. CMMI has lost this focus. It is preached by the software engineering equivalent of accountants who can't use a spreadsheet, i.e. by software 'experts' who haven't coded for years. Try finding a computer book these days that talks about not burning the money, not wasting the budget, i.e. that talks as if the money that pays for all this process were our money. Scott would bet his own money on an XP team against any CMMI level 6 six-sigma team. He does not think six sigma enthusiasts would do the same. He always asks six sigma method experts, "Do you develop software at home". Most do not but a few say yes: he has never found one that does six sigma on their own software at home. By contrast, plenty of XP enthusiasts (myself for one) use it at home.
CMMI six sigma appeals to bureaucrats needing to justify their existence but less to people with real work to do. It is possible to work that way but it usually fails. The Standish Chaos report states that project success rates are dropping: 65% fail blatantly (project is canned outright or its use is never attempted), 85% fail in fact (not valuable or not used). People either aren't doing traditional methods properly (i.e. those who must do them must dislike doing them too much) or traditional methods aren't working.
The agile manifesto says that we must value one set of things over another.
- Individuals and interactions are more important than processes and tools. People working well as a team and with stakeholders matter more than a process.
- Working software matters more than comprehensive documentation.
- Customer collaboration matters more than contract negotiation. An ironclad contract is no use without a good relationship with stakeholders; you will fail. (You'll have a good excuse when the failure goes to court; is that the game you want to be in?)
- Responding to change is more important than following a plan.
The agile manifesto also has 12 principles; he talked about some of them. Mary Popendike, "A changed requirement late in lifecycle is a competitive advantage, provided you can act on it." Out-sourcing and other non-agile methods deny you that advantage, but make life easy for bureaucrats.
Agile software development is not code and fix. It is not an excuse not to document. The software industry is arrogant and assumes the stakeholders are morons. It spends their money on documents we tell them we must write. In an agile method, you tell the stakeholders that it will cost $50,000 to write a system overview document. Let them decide whether to make that a requirement or not; don't decide it for them. No more "create a software overview document because RUP says we must do that, hold a review because RUP says we must do that." Always ask, what value does that add. Justify these things. (And, I would add, try to ask those who will actually get the document's value, or lack thereof, and be paying for it.)
Agile is not an excuse to ignore enterprise concerns. Enterprise people (the database, internal customers) must change their ways to work with agile otherwise the developers will ignore them, as in fact happens today under traditional methods. Never believe IT people who say they must work serially, not iteratively and interactively. They can; they just choose not to.
Out-sourcing is happening because the business community are sick of being burned. Now they will out-source and be burned at 25% of the cost, which is at least a step in the right direction. This decade will see that shake-out occur; you will be agile or out-sourced.
There are several agile methods:
- XP emphasises the planning game and the importance of tests (including customer/acceptance tests). An XPer's day starts with a quick paired design session followed by testing, coding and refactoring, with some integration (rerun of general test suite) checks. XPers talk to stakeholders frequently. There is not much of a niche for paper-pushers in this process.
- SCRUM is a project management/project coaching method (see also remarks in Joseph Pelrine's talk). It presupposes a project backlog, a pile of use cases or other list of things to do. You pull 30 days worth of highest-priority items off the stack and sprint for 30 days implementing them (using XP or another method; SCRUM does not say). It is a rule that there are no requirements changes to what you're implementing in that 30 days (change requests can be put on backlog stack). Every month-end there is a go/no-go decision for the team - i.e. the stakeholders decide whether to pay
more or less budget to the team - and the backlog's content, and that contents' priority ordering, etc. are revisited. SCRUM is for managers, not paper pushers; it asks for decisions, not reports. It has the problem that it does not help fabricate long-term annual budget estimates. In traditional methods, the accountants know that their next-year budget is in fact a fiction but they want to play the game of computing it, so may object to SCRUM because it interferes with the game.
- Dynamic System Development Method (www.dsdm.org): this may have value for a very pro-CMMI organisation because it has some of the traditional ceremonies while being agile-ish.
- Feature-Driven Development (www.featuredrivendevelopment.com): very like XP but with an explicit domain modelling activity up-front to drive the feature list (plus the model is further developed over time). The explicit model may make FDD more palatable to management.
- The Crystal Family (www.crystalmethodologies.com, Alastair Cockburn): a family of methods; you pick the one that suits your project.
- The Agile Unified Process is a marketing attempt to fit agile into RUP. The good side is that one can get close to doing XP in RUP. The bad side is that RUP speaks to bureaucrats so the people who want the RUP don't want agile and those who want agile don't want RUP. Thus a project of 10 people will allocate two people to the role of blocker - they do the RUP training, attend the meetings, create the stupid documents. He asked how many have done that (many hands) and how many have been caught (no hands). Not only do management imagine this stuff lets them manage, they're too stupid to catch us when we fake it. (He wrote an article on this a few months back; much feedback on it.) Have you been on a project where there was a big requirements document that developers didn't know existed or never read (several hands). This is the disconnect; traditional methods would work (not very well) if they were followed but they are not.
- Agile Modelling (www.agilemodeling.com): on a project some years ago they designed a great architecture and then went shopping (for large sums) on vendors' products to be the architecture's platforms. They could not get these products to work together. You must prove it with code, not architecture. Agile modelling just makes this point within other methods that may do some modelling.
(I saw the same thing happen in a major project at a former employer, supposed to be the basis for a key new offering. The third-party products' licence fees made every seat very costly, the system had a huge footprint and the parts were hard to make interwork and even harder to debug, both technically and administratively since every vendor began by claiming it was one of the others' problems. Many months of up-front analysis and design of this important system had failed even to notice this might be an issue. Once construction began, it soon became obvious to those involved but the plan was made, the analysis done, the design set: the perceptions of the skilled architects working on the project became a background noise of muttered warnings and moans that the project's CMMI process existed to suppress in the name of 'plan what you'll do, then do what you planned.')
When you have 20 configure-controlled models, you have 20 models to update when a change is proposed, a powerful incentive not to propose changes no matter how obviously needed. "How many people live on a street that is not on their road map" (several hands went up; I had encountered the phenomenon the day before travelling around Oakville with friends, and noted it as a difference between America and the U.K., where it would be most rare). Scott lives on an unmapped street but he will find his way home tonight. Q. How does the fire department find your house? "It's the one that's on fire!" (and the police? - "they manage" :-)). Barry Boem has spoken on the issue that traditional methods have risks as do agile methods.
- Test-Driven Development, like agile modelling, this is a method fragment, not a complete method; it just says how to do TDD.
- Agile Data Method (www.agiledata.org): how can data and enterprise developers work on an agile project? Operations people must be helped to get on board. ADM just defines principles to help this. (A fortnight ago, he talked to some data people who were convinced that there were less than 100,000 object developers in the world. Such 'living in 70s' people exist.)
Why does agile work. It is focused on quality every day, responding to change every day, working with stakeholders every day. You need people skills and some humility. The challenge is to get stakeholders to show up. For three generations, the IT industry has trained stakeholders to turn up at the start, sign off that we can burn this much of their money, then go away till acceptance test. Stakeholders must also be open-minded as we will be showing them our actual work state, not telling them all is fine till delivery.
Agile can shoot itself in the foot. So can its rivals. Agile shortens the feedback loop. If you believe that software is all about communication, then you want to work in the way that uses the best form of communication. Formal documentation is notoriously the worst form of communication. Standing round a whiteboard is better. Pairing on code is best. Even the best-run review is still a mistake. If you've invited people who are qualified to give feedback now, why were they not asked for their opinion earlier. If their opinions were not worth canvassing earlier, why are they asked now.
Software is like swimming; it's dangerous to do it alone. The heroes are the most dangerous people on your team: at best, they write good code no-one can understand, then they leave. (I was reminded of Edmund Burke's remark, after a lifetime's experience as a very skilled legislator, that, "I never saw any law that was not much improved by the comments of people far less able than those who had drawn it up.", from 'Reflections on the Revolution in France', quoted from memory.)
Specialists (e.g. a specialist use case modeller) are a serious problem in the
industry. Such specialists put information in their tools whereas it should be in the code. Have no specialists that can't read code and put information there. A requirements analyst that can't express information in code is like an accountant that can't drive Excel. (Scott has been in a meeting where sales people keen on web selling proposed delivering real physical products "as email attachments".) XP requires a degree of team work: the 'uber-techie' with no (bearable) personality is not wanted.
The 'Extreme Programming Perspectives' book has some success case studies and others are appearing in magazines. XP today is where objects were in 1990. It took a generation for objects to be accepted; XP will be the same. Scott works with researchers and notes that traditional methods have very little proof that they work. XP can get little funding for research; he knows frustrated researchers who would like to get their PhD studying the effectiveness of agile methods but can get no funding.
Why does the agile alliance work so well? Because most of them are smalltalkers.
Q. Selling to management? CEOs are as frustrated as the people in the projects. Their direct reports don't feed them agile for the reasons above. They are tempted to compromise (agile in RUP) as discussed above. Often you must wait till the company has shot itself in the foot several times before they will try XP.
Often middle managers prefer a larger team; it gives them more visibility. People who are not good at producing find political ways to justify their existence. To be visibly successful right away means that the politicians try to get involved. SCRUM states that 'the project manager's function is to be a shield, protecting the team from incoming flak'. Also, success can raise the level of expectation too much for others in the company. Scott worked in a team in a company that had not delivered in three years. His team delivered and were shown the door the day they delivered.
Q. If agility is where objects were 15 years ago, will we, 15 years hence, be using not agility but Javility: some dumbed-down version? Maybe; there's sure to be some, and indeed we're seeing some now.
Previous Topic
