The Artima Developer Community
Sponsored Link

Weblogs Forum
Traps & Pitfalls of Agile Software Development - A Non-Contrarian View

38 replies on 3 pages. Most recent reply: Jul 20, 2012 3:59 PM by Krithika Seetharaman

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 38 replies on 3 pages [ « | 1 2 3 | » ]
Sean Landis

Posts: 129
Nickname: seanl
Registered: Mar, 2002

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Jan 6, 2009 9:56 PM
Reply to this message Reply
Advertisement
> > That said, it seems reasonable for the employee to
> expect
> > to be respected, listened too, integrated into the
> > e decision process, etc. That's not a democracy but
> worth
> > consideration.

^^^This was the hint that I don't think 'proper' is a command-and-control relationship.

> Agile is entirely a two way street where as
> most traditional employer/employee relationships are one
> way with those nasty tire popping spikes that you see in
> parking garages and rental places thrown down on the road.

I can't speak to 'traditional'. I am sure it exists but I've never worked at a place like that. It's always been further on the spectrum toward two-way. Agile methods are definitely more likely to succeed in an environment where dialog is valued.

Jason Yip

Posts: 31
Nickname: jchyip
Registered: Mar, 2003

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Jan 7, 2009 3:41 PM
Reply to this message Reply
As a practitioner, the main trap/pitfall of Agile software development is thinking about Agile instead of about the specific situation

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Jan 7, 2009 4:19 PM
Reply to this message Reply
> As a practitioner, the main trap/pitfall of Agile software
> development is thinking about Agile instead of about the
> specific situation

So, if you have a methodology you aren't supposed to be thinking about, how does that differ from having no methodology at all? Other than the fact that it's hard to sell books about nothing, maybe?

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Jan 7, 2009 4:36 PM
Reply to this message Reply
> > As a practitioner, the main trap/pitfall of Agile
> software
> > development is thinking about Agile instead of about
> the
> > specific situation
>
> So, if you have a methodology you aren't supposed to be
> thinking about, how does that differ from having no
> methodology at all? Other than the fact that it's hard to
> sell books about nothing, maybe?

It's a Zen Koan. Once you figure it out, you have achieved Agile enlightenment.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Jan 8, 2009 12:45 PM
Reply to this message Reply
This interview is pertinent to the conversation:

http://www.infoq.com/interviews/Agile-Skeptic-Keith-Braithwaite

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Jan 9, 2009 4:09 AM
Reply to this message Reply
> > As a practitioner, the main trap/pitfall of Agile
> software
> > development is thinking about Agile instead of about
> the
> > specific situation
>
> So, if you have a methodology you aren't supposed to be
> thinking about, how does that differ from having no
> methodology at all? Other than the fact that it's hard to
> sell books about nothing, maybe?

Well, lets say that once he had booked into Agile he has committed to break the egg at the smaller end.

http://www.mat.upm.es/~jcm/jonathan-swift--end.html

Fireblaze .

Posts: 21
Nickname: fireblaze
Registered: Jul, 2006

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Jan 30, 2009 10:38 AM
Reply to this message Reply
I rarely find any team that follow the . Particular "At regular intervals, the team reflects on how
to become more effective, then tunes and adjusts
its behavior accordingly."

The tunes and adjusts bit is often not done at all since both developers and managers block the change. http://agilemanifesto.org/principles.html

Sean Landis

Posts: 129
Nickname: seanl
Registered: Mar, 2002

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Jan 30, 2009 11:50 AM
Reply to this message Reply
> I rarely find any team that follow the . Particular "At
> regular intervals, the team reflects on how
> to become more effective, then tunes and adjusts
> its behavior accordingly."
>
> The tunes and adjusts bit is often not done at all since
> both developers and managers block the change.
> http://agilemanifesto.org/principles.html

"Reflect and adapt" is possibly the most important agile value. Default human nature is to resist change so organizations must do what they can to establish an change friendly environment. A major part of that is hiring practice that brings in people predisposed to change.

It should be clear that if the predominant management philosophy is "command-and-control" then adaptation will be difficult. Agile teams to best when they are able to take ownership and accountability for their own success or failure. This requires management to give the teams room to adapt and room to fail sometimes.

I have observed another very interesting phenomenon: teams don't reflect and adapt because they BELIEVE management won't let them. I've seen this belief prevail even in change-friendly environments. It is truly fascinating.

John Labarge

Posts: 1
Nickname: jlblaw
Registered: Feb, 2009

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Feb 19, 2009 10:00 AM
Reply to this message Reply
Underdesign underdesign underdesign.

Agile sounds extremely rational, everything about it: iterations, emergent design, test driven etc. I was excited to try it. But what I found when I worked on an agile project was that the place that claimed to be agile was merely a hack shop. It was almost as bad as non-iterative methods.

Since Agile proponents like guidelines I'll give yo mine based on my experience:


(1) Agile as a concept is great; where it fails is that it under estimates the demands on fallible nature of management and overestimates the autonomy of developers.

(2) As a developer, you'll find when pressed for time, refactoring is the hardest thing to justify to management. It's not that you aren't an "adult". It's that you want to stay employed. It's not that you aren't competent, its that refactoring time is not built in to the schedule.


(3) Agile means to management - just code and the design will emerge. Any upfront design is often frowned upon.


(4) Without zero upfront design and near zero refactoring what we end up with is a hack shop.


And we are back to the proposition that another commenter mentioned, communism sounded great as a concept but in practice failed because ignored human nature.

The rebuttal to that was that employees are somehow insulated from human nature, because they are "professional" as it were and want the best for the company.

I beg to differ. Professionalism has nothing to do with it. At the time of the decision not to refactor, management may be right. And the developer's professionalism has nothing to do with it, because management communicates whats best for the company to developers.

Management is right because the expense of refactoring is not justified by greater demand or a greater price for the product when at that moment the new feature is so justified. Management will choose the new feature every time, because that in fact is better for the company. Oh there is the hidden expense of growing numbers of bugs; but at that point, refactoring is not the issue, the bugs are, so we patch the bugs. Developers are forced to go with management, that is what it is to be professional.

Agile isn't as much a methodology as a message. Methodologies will vary from group to group. Agile's message assumes too much; it assumes that developers are running the show or there is some mythical customer on site all the time who will agree that the code shouldn't be rickity. That is usually not the case.

To get the message right, Agile needs to add back a discipline of some amount of upfront design prior to coding.

At least as much time thinking about the problem as coding.
And by the way, this is true with almost every discipline, no matter how good you are at implementing, no matter how long you've been doing it. Some amount of thought time should be put in before you begin.

For example, papers turn out better when you outline first. Buildings turn out better when you plan them first ala blueprints. Interior decoration turns out better when you plan it first (watch any interior design show). Driving to a destination you've never been to (hell with traffic even that you have been to) turns out better when you think about how to get there first.

You think before you speak, I hope. You think before you write, I also hope. Why not think before you write code?

While this concept may be implicit in agile, it needs to be explicit, because today's agile managers are not getting that message.

Sean Landis

Posts: 129
Nickname: seanl
Registered: Mar, 2002

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Feb 19, 2009 11:59 AM
Reply to this message Reply
John,
I agree with some of the points, whereas some of the comments are why I wrote the article in the first place. Let me explain...

> Underdesign underdesign underdesign.
>
> Agile sounds extremely rational, everything about it:
> iterations, emergent design, test driven etc. I was
> excited to try it. But what I found when I worked on an
> agile project was that the place that claimed to be agile
> was merely a hack shop. It was almost as bad as
> non-iterative methods.

Just because an organization claims to do agile development, it doesn't mean they are. This is the kind of stuff I see too much of. It failed because the team failed, not because Agile failed.

> Since Agile proponents like guidelines I'll give yo mine
> based on my experience:
>
> (1) Agile as a concept is great; where it fails is that it
> under estimates the demands on fallible nature of
> management and overestimates the autonomy of developers.

I don't buy arguments like, "Agile fails because Management..." or "Agile fails because Developers..."
I personally think Agile is quite robust in the face of human nature. But that's not what I think you are getting at because of the comments that follow.

> (2) As a developer, you'll find when pressed for time,

The precondition here is "pressed for time." This is an environmental issue that is being responded to with some kind of behavior. Agile methods have a lot to say about how to best manage and avoid time pressure.

> refactoring is the hardest thing to justify to management.

This is a challenge for any software team. The challenge with agile is that it is hard for a team to hide what it is doing. This is the point of my #1. Agile teams can be allured into excessively tactical behavior and loose site of strategic behaviors like refactoring or design. I think the Agile Principles are pretty clear on how to address this and most agile methodologies speak to it as well. That doesn't mean teams practice the recommendations though.

> It's not that you aren't an "adult". It's that you want
> t to stay employed. It's not that you aren't competent,
> its that refactoring time is not built in to the schedule.

Sounds like a fear-motive. Reflect on that as it pertains to you job and your personal happiness ;-)

If refactoring isn't built into the schedule then it isn't going to get done - unless the team has a shadowy corner to hide work in. I'd say "adult" and "competent" do apply well here although they are somewhat loaded terms. Teams should plan their work. Refactoring should be done all the time and so it should be built into story estimates (or whatever your favorite vocabulary) just like bug rework and testing are.

> (3) Agile means to management - just code and the design
> will emerge. Any upfront design is often frowned upon.

"Agile doesn't work because Management..."
There is a huge body of work describing organizational and cultural preconditions where agile teams do best. Most organizations that aren't already succeeding with agile don't look that way. It's not a secret: Agile will force an organization to change. In an Agile-hostile organization, I think there are three options: 1) Try Agile expecting it to cause change and doing your part to help make the change, 2) Do something more consistent with the organization and culture, or 3) go work somewhere that has a less hostile culture and is amenable to change.

> (4) Without zero upfront design and near zero refactoring
> what we end up with is a hack shop.

A hack shop is full of hacks. The trap was identified but it's not in the nature of Agile discourage good software development techniques. It is important to note that Agile is NOT a set of software development techniques. The team is responsible for choosing to do the right thing; "adult" and "competent" come to mind.

> And we are back to the proposition that another commenter
> mentioned, communism sounded great as a concept but in
> practice failed because ignored human nature.
>
> The rebuttal to that was that employees are somehow
> insulated from human nature, because they are
> "professional" as it were and want the best for the
> company.

As I said, I think that agile respects and embraces human nature. Let's be sure do differentiate human nature from destructive human behaviors though.

> I beg to differ. Professionalism has nothing to do with
> it. At the time of the decision not to refactor,
> management may be right. And the developer's
> professionalism has nothing to do with it, because
> management communicates whats best for the company to
> developers.

This is a problem with a specific organizational and cultural situation that sounds quite Agile-hostile. If management is telling the team what to do, then the team is not being allowed to self-organize and do what it thinks is best. (By team I include the Product Owner). This is another case of "Agile doesn't work because Management..."

> Management is right because the expense of refactoring is
> not justified by greater demand or a greater price for the
> product when at that moment the new feature is so
> justified.

As I mentioned, one way to deal with paying down technical debt is to build its cost into estimates just like bugs. Is this organization you describe also hostile to allowing the team fix bugs? Does the team even estimate?

Another approach is to do the ROI analysis for the technical debt that accrues and in those terms, demonstrate the business value. There are other approaches. I don't think this example has anything to do with failures of agile development per se.

> Management will choose the new feature every
> time, because that in fact is better for the company. Oh
> there is the hidden expense of growing numbers of bugs;
> but at that point, refactoring is not the issue, the bugs
> are, so we patch the bugs. Developers are forced to go
> with management, that is what it is to be professional.

There is always a tension between tactical and strategic investment. In any scheme, I think it is the responsibility of teams to frame the trade offs so that sound business decisions can be made. This is professional and it is also prudent to the business. I can imagine situations where certain technical debt is forgone for tactical gain. At the same time, it is irresponsible to not frame the cost of deferring such debt. I am talking about debt that shouldn't be built into estimates so refactoring isn't what I mean. An example might be upgrading an open source tool, or rewriting a problematic module. If the decision is always made to not take care of technical debt then I think option 3 is a good choice because the software will eventually fail (and probably the company too).

> Agile isn't as much a methodology as a message.

Hopefully I made that clear.

> Methodologies will vary from group to group. Agile's
> s message assumes too much; it assumes that developers are
> running the show or there is some mythical customer on
> site all the time who will agree that the code shouldn't
> be rickity. That is usually not the case.

Maybe Agile assumes to much of the organization you are working in and option 1 isn't feasible. A software team has to be able to survive within its organization.

The thing I like about Agile is that it is clear what those assumptions are. It is irresponsible to adopt anything that requires change without mapping out the scope of that change. Adopting agile will require organizational change at all levels. That may be too big a mountain for an organization to climb.

Sadly consultants and writers and evangelists often don't point this out and they set teams and organizations up for big failures.

> To get the message right, Agile needs to add back a
> discipline of some amount of upfront design prior to
> coding.

I think there's a misconception here. Defining agile by how you've seen it done is like defining a religion by observing the behaviors of those who claim to believe. Go to the source and see if you find any indication that Agile discourages design. What it does discourage is a "Design Phase" where a team attempts to do all the design up front (BDUF) and then exit that phase with design finished. But that's not to say you don't design. In fact, nothing in Agile prevents you from doing BDUF if that works for your team. In most projects though, the reality is that you can't anticipate all design decisions up front so some design will evolve.

> At least as much time thinking about the problem as
> coding.

Do you believe Agile advocates not thinking about problems?

> And by the way, this is true with almost every discipline,
> no matter how good you are at implementing, no matter how
> long you've been doing it. Some amount of thought time
> should be put in before you begin.

I'd put this under software development or engineering principles not methodologies. I see the two as orthogonal to the extent they don't undermine each other. Agile says nothing about which engineering principles a team chooses to use.

> For example, papers turn out better when you outline
> first. Buildings turn out better when you plan them first
> ala blueprints. Interior decoration turns out better when
> you plan it first (watch any interior design show).
> Driving to a destination you've never been to (hell with
> traffic even that you have been to) turns out better when
> you think about how to get there first.

It sounds like you have the answer to your design problem in an Agile team!

> You think before you speak, I hope. You think before you
> write, I also hope. Why not think before you write code?
>
> While this concept may be implicit in agile, it needs to
> be explicit, because today's agile managers are not
> getting that message.

Like I said, agile isn't an engineering technique. And I wouldn't call any manager agile if they don't understand Agile in the first place.

Peace

Sean Landis

Posts: 129
Nickname: seanl
Registered: Mar, 2002

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Feb 19, 2009 12:39 PM
Reply to this message Reply
Just a couple other things...

> > Methodologies will vary from group to group.

A key principle of agile is to reflect and adapt which says that EVERY group will have a different methodology.

> > And by the way, this is true with almost every
> discipline,
> > no matter how good you are at implementing, no matter
> how
> > long you've been doing it. Some amount of thought time
> > should be put in before you begin.
>
> I'd put this under software development or engineering
> principles not methodologies. I see the two as orthogonal
> to the extent they don't undermine each other. Agile says
> nothing about which engineering principles a team chooses
> to use.

I don't mean to imply Agile is a methodology; it's a set of values and principles. That, and the methodologies derived from those values and principles (XP, Scrum, etc) don't speak to specific engineering principles like how much time to spend on design. Hopefully that is clearer.

Florin Jurcovici

Posts: 66
Nickname: a0flj0
Registered: Feb, 2005

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Apr 12, 2009 8:51 AM
Reply to this message Reply
A recent experience I had. A customer requirement to an already delivered and burried product required a change which I didn't really have a good idea how to implement. Anyway it required some significant refactoring - I thought.

It is my understanding that the agile way applied extremely would have meant rushing to code - after all, the user story was whoo hoo hoo how detailed and plain.

Since I didn't have any idea how to do it, I messed for some time with UML diagrams and some textual notes, rather than trying out various solutions in code. I quickly came up with a quite clean solution, which I coded quite quickly, compared to my initial expectations. I doubt rushing to code would have led me to a better solution in the same time, or to a comparable solution in less time. On the contrary.

My point: it's always cheaper, and in most cases doable, to get it right from the start, rather than through several iterations (agreed, you sometimes might need to take one step at a time, which would seem a lot like iterations, but isn't). You cannot completely avoid iterations, but by trying to get things right right from the beginning, rather than stick to rigid and frequent deadlines, you minimize the number of required iterations. Rather than making them a generally accepted part of your development process.

Florin Jurcovici

Posts: 66
Nickname: a0flj0
Registered: Feb, 2005

Project managers & Agile methods Posted: Apr 12, 2009 9:01 AM
Reply to this message Reply
PM theory, such as the one decsribed in the PMBOK, for instance, is mostly acquired from sources outside the software development domain. You couldn't build a large building using an agile project management method - think of it, refactoring at least parts of a large building during several iterations is unthinkable in terms of costs.

On the other hand, if an organization hires a professional project manager for a software project, they expect him to manage the software development process the same way a building project would be managed. I.e. to have at each point in time a clear view on completion, costs, status. Which is especially hard when using agile methods, which set out to build a working product at a time when not even all requirements are clear.

Although classical project management has learned to deal with iterations, it has not yet learned to deal with uncertain/unclear requirements/expectations/project goals. This is exactly the issue that agile methods try to cover. Which IMO is on one hand an attempt to solve organizational issues in the software development process, and therefore will not work, and on the other hand will never conveniently be mixable with classical project management as it is now.

Sean Landis

Posts: 129
Nickname: seanl
Registered: Mar, 2002

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Apr 12, 2009 9:07 AM
Reply to this message Reply
> My point: it's always cheaper, and in most cases doable,
> to get it right from the start, rather than through
> several iterations (agreed, you sometimes might need to
> take one step at a time, which would seem a lot like
> iterations, but isn't). You cannot completely avoid
> iterations, but by trying to get things right right from
> the beginning, rather than stick to rigid and frequent
> deadlines, you minimize the number of required iterations.
> Rather than making them a generally accepted part of your
> development process.

Your point is an excellent one and nothing in Agile principles suggests otherwise. This is Pitfall #1 on my list. People get it in their head that Agile encourages rushing to design.

Design is an engineering practice not an Agile practice. Use them together to you best benefit.

Florin Jurcovici

Posts: 66
Nickname: a0flj0
Registered: Feb, 2005

Re: Traps & Pitfalls of Agile Software Development - A Non-Contrarian View Posted: Apr 12, 2009 9:13 AM
Reply to this message Reply
> 11. Agile is too programmer-centric leaving it unclear how to balance work across an organization.

IMO it's not about work, it's about responsibility.

What my experience tells me is that unless you get the customer to feel responsible for his requirements early on in the project, and make it clear to him that bad requirements means high costs, the customer gets to develop an impression that your are cheating on him, or are way too expensive, or, even worse, unqualified to do the work, even if you deliver exactly according to customer requirements at the end of each iteration.

IMO, that's also the biggest pittfall with agile methods: by telling the customer you are going to do things right, even if requirements aren't well defined from the beginning, on one hand you take on responsibility for bad requirements, which is evidently bad, on the other hand you communicate a lie: that things are doable even if the customer doesn't bother too much to properly define requirements and expectations.

Also, when using agile methods, the role of documentation is minimized, and requirements are usually communicated informally between stakeholders and developers. Which IMO represents a capitulation without a fight against the challenge of properly documenting requirements (that is, short enough to be readable, detailed enough to be unambiguous, well refactored enough to be maintainable, well structured enough to be usable as working instructions by programmers and as a start for test cases for testers - I usually don't ask for more from such documentation). My impression is that agile gurus think this is either not doable with reasonable effort, or not doable at all.

Flat View: This topic has 38 replies on 3 pages [ « | 1  2  3 | » ]
Topic: JavaOne 2011 - Day 3 Previous Topic   Next Topic Topic: The Code Quality Myth


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us