Summary
I've spent most of my career in small companies, usually start-ups,
doing software development under less than optimal conditions. I've got 15 or 20 years worth of (hopefully instructive) horror stories aching for an outlet. This is my first installment.
Advertisement
guerril´la
A member of an irregular, usually indigenous military or paramilitary
unit operating in small bands in occupied territory to harass and
undermine the enemy, as by surprise raids.
The American Heritage Dictionary of the English Language, Fourth Edition
What is Guerrilla Development?
I've spent most of my career in small companies, usually start-ups,
doing software development under less than optimal conditions. So,
when Bill Venners asked if I'd blog on artima.com
I said yes! immediately since I've got 15 or 20 years worth of (hopefully
instructive) horror stories aching for an outlet. This is my first installment.
Choosing the clever name for my blog page was the second most difficult task
in all this (finding a picture of me that wouldn't frighten children was the
only thing harder). I toyed with several catchy phrases, but most of them didn't
convey the level of conflict and frustration that can be felt both inside the
software organization and between the developers and the rest of the company.
The more I thought about it, the more I realized the software department often
played the role of The Loyal Opposition, holders of customer interests,
keepers of the vision of the long-term company goals, and defenders of sanity
in general. We were insurgents in our own company. We were doing
Guerrilla Development.
Guerrilla Development
Software development in an environment unsupportive of the effort.
The adversity can take the form of active management encumbrances
(ridiculous budget constraints, time constraints, staffing constraints,
and the like), or in passive resistance such as stonewalling by
product management, marketing, sales, or training. Guerrilla Developers
find a way to win, despite the odds, and work to prevent their project
from becoming a
Death March.
Understanding the Impact of Low Maturity
All of this might sound a little melodramatic to those working in mature
software environments. If you are such a person, consider yourself lucky.
Estimates vary, but somewhere in the neighborhood of 80% or more of all
software development groups work at
Software Engineering Institute (SEI)
Level I Maturity. For those of you not familiar with the SEI maturity
levels, here is a handy table. (Please credit
SEI for crux of the information shown below.)
Level
Focus
Key Process Areas
5 Optimizing
Continuous process improvement
Continuous process improvement is enabled by quantitative feedback
from the process and from piloting innovative ideas and technologies
4 Managed
Measurements and analysis
Detailed measures of the software process and product quality are
collected. Both the software process and products are quantitatively
understood and controlled.
3 Defined
Process standardization
The software process for both management and engineering activities is
documented, standardized, and integrated into a standard software process
for the organization. All projects use an approved, tailored version of
the organization's standard software process for developing and maintaining
software.
2 Repeatable
Basic project management
Basic project management processes are established to track cost,
schedule, and functionality. The necessary process discipline is
in place to repeat earlier successes on projects with similar
applications.
1 Initial
Competent people and heroics make it all happen
I've read numerous software engineering books over the years and while
they might mention that most shops are at SEI Level I maturity, they
rarely give insight as to what that really means. The
vast inventory of software development knowledge is carefully cataloged
in these tomes: most shops are SEI Level I shops, projects should begin
with a requirements gathering phase, and here's a description of
McCabe Cyclomatic Complexity, for example. All of these interesting points do
not have the same importance. One of them, the fact that most shops operate
in an environment that can easily categorized as chaotic, is vastly
more important than all the other pieces of software engineering trivia.
Being at SEI Level I Maturity means that it is likely that your software
department, and likely your whole company, is a complete mess. Only heroics
and daredevil acts bring victories. There is no notion of control;
there is only risk mitigation. There is no process; there are only people.
Often, your goal is simply to survive to fight another day.
Consider the kinds of things I've heard over the years:
"Three tech writers? Why do I need one?"
"Quality? That's your problem!"
"We need a new demo for a customer. Put everybody on it. He'll be here in a week.
Oh, and your other schedule can't slip."
"I thought your estimates where high so I cut them in half before submitting
our bid. Consider it a 'management challenge'."
As you might be able to tell from the above, SEI Level I software development
shops often exist within SEI Level I mentality companies. This imposes a special
challenge for anyone wishing to make things better.
The Challenge of Leadership
Imagine the energy, planning, and determination it would take to bring
an SEI Level I shop to even Level II where there is some process, some
tools in place, and some measures to assist with scheduling, bug tracking,
and testing. This is the challenge that most small companies that build
software have.
Note that I'm not talking about software companies, necessarily. There is
a significant number of companies that have a software development function,
yet would not likely be described by their executive-level managers as a
"software company".
"We don't make software; we make mobile phones."
Yet, most of what makes a mobile phone and its network function is
sophisticated software.
"We don't make software; we make equipment."
It almost doesn't matter what kind of equipment we talking about anymore.
If it goes into a modern factory, it needs software. It needs to communicate
with the other pieces of equipment on the floor. Motion control is software
controlled. Its status is monitored remotely with software.
"We don't make software; we make cars."
Cars are now rolling networks with dozens or even hundreds of
embedded computers in them.
I've cited big things (phones, factory equipment, and cars) but the
problems associated with software being a second class citizen
in a company is actually more likely to occur in smaller companies
making smaller products like coffee makers, microwaves, toys, bicycle
computers, and other gadgets. Check out the book Embedded, Everywhere
to see how pervasive computers have become and where this trend is taking
us.
My Next Few Blogs
With the explosion of demand for software and a limited number of people
adept at software development, we can expect many of these new shops to
spring into existence at SEI Level I maturity and stay there for a while.
What can we do about it?
In my series of blogs I'll talk about the state-of-the-art at the low-end
of the software maturity scale. I hope to provide some insight into how
things are, how they got that way, and how folks (like me, when I was doing it)
try to introduce maturity and sanity into the system. You shouldn't write off
every department or project that has profound troubles at a given point
as a Death March. Quite the contrary, I believe that in every SEI Level I shop
there is a Level II or Level III shop trying to get out!
I've had my share of ideas gathered over the years that I hope to share
in future blogs. I look forward to hearing from the community on this.
Please feel free to post feedback here or write me directly.
While I very much look forward to you upcoming columns, I hope you use a spell-checker for them :-) Or is the odd spelling mistake meant to reinforce the point that development under unrealistic time pressure is unhelpful?
Thank you for your interest... and for the note that I introduced a typo in my last round of edits. While I was at it, I fixed a verb agreement problem elsewhere in the text I'd also found. Just like in software, it is those last minute, small, harmless changes that always get you! <br> -- Scott
About the SEI model ... I've found this model interesting, but perhaps not particularly useful. It shows levels of advanced organization, but it is not absolute.
If I recall correctly, you can't be level 2 unless you meet all the criteria for level 2, and likewise up the scale. Yet at the places I've worked at, there have existed varying attributes of all levels. I doubt that even one place would meet level 2 by strict definition. Even the place that had source control, reproducible builds, release cycles, requirements, use cases, conceptual model, etc., etc., etc.
The important aspect is not the CMM level of maturity, but the mix of features of those levels. It might be more productive to view a bar graph showing how much of each level the company achieves, though even that would probably omit important information. The various features of each level are appropriate in specific contexts.
The important aspect is not the CMM level of maturity, but the mix of features of those levels. It might be more productive to view a bar graph showing how much of each level the company achieves, though even that would probably omit important information. The various features of each level are appropriate in specific contexts.
These are excellent points. I would like to add that the CMM model fails to classify maturity in terms of ROI. It is understandable that we need objective criteria to establish measurement baselines, but at what cost. The CMM is stigmatized by a myopic view of the organization as a whole because it is born from the technology side of the house. What is needed is a more comprehensive way to view the maturity of an organization from the macro and the maturity of projects at the micro. An integration of the CMM with the Zachman model of an enterprise would be a good start.
I used to subscribe to the SEI model - I don't anymore. I don't think it works.
In my experience, the SEI model is essentially fantasy in the software world. That's not entirely true, the control software for the space shuttle is developed using this model and it does work - but nobody else can really afford to use it.
I find the ideas at http://www.controlchaos.com considerably more realistic and applicable to business software development.
That's an interesting argument, but I never quite understood what benefits the SEI model provides. I think sometimes very clever people try to define unifying theories of certain phonomena, whereas the phenomena themselves are quite simple to understand using plain common sense.
There are many fashionable theories to explain away failure, but the buck must always stop somewhere, and failure or "chaos" in software development almost inevitably boils down to some cognitive, epistemological errors made by an architect or developer. Such errors might be: failure to understand the specific objectives of a development task, failure to manage complexity, changes, using improper tools for a given task, etc. The epistemological errors plaguing ill-fated software development projects are not limited to software development, but are generally applicable to any purposeful human action.
Often, psychological factors underpin a developer's cognitive choices. For instance, not leaving broken windows behind is primarily a psychological choice, and all the thinking that must go into pursuing that objective are motivated by that psychological factor. The leadership of a software development organization should, therefore, primarily concern itself with establishing the right psychological conditions for the job. Given competent developers, the rest naturally follows. And even less competent developers can rapidly improve, given the motivating conditions. Successful open-source projects are an example of the primacy of psychological factors in software development; and failed projects - open source or otherwise - show that principle at work, too.
But motivating leadership is often lacking in software development projects. I also worked for smaller companies a lot, and what I found was that "chaos" almost always ensued as a result of a complete lack of leadership from the company's management. Perhaps larger organizations invest more into advancing the leadership skills of their managers - I'm not sure of that, though.
considering that most companeis that I have worked for ask what is SEI?
SEI does not have ROI measures but if you quit re inventing the wheel in repeating the process of paying twice and accept and use open source processes and/or code bases more..you find that SEI does make sense and havce a nice ROI..
But remember we live in a world where most programmers out of college only know consumer based software which is only 30% of the whoel in terms of overall software cde produced aroudn the worls..and thus have the myth that one must push out error pro0ne code to get the sale rather than bug free code for the long term sales of service support of a system..
How doe sone get rid of those type sof myths in programmign and managment staff wehile not trying to kill them ?? :)
I love the whole SEI thing and it certainly helps in judging the maturity of an organization etc.
Having said that, I've always had this nagging suspicion that a large number of software projects are death marches because various people simply aren't aware of the consequences of their actions.
Some of the causes seem to include:
(1) People don't know they don't know enough - they *think* they are able to make a particular decision little realizing they are very badly equipped and should pass the responsibility to someone else (and if there isn't such a person, find one!).
(2) Various people don't want to confront the complexity of developing software. It suits them to be naive of it, allowing them to make all sorts of statements and not be responsible for the outcome. i.e. They use their naive position to gain political advantage.
(3) People don't ever stop and ask themselves why a project went wrong. If they stop to consider it at all, they blame someone else (see point 2).
In conclusion, I think software development efforts suffer, significantly, from people type issues and, until we find some way to deal with these, death march projects will be the norm for most of us.
I hadn't heard of the term Guerilla Development until I read this article, but it is so very true. And so true of most of the places I worked.
I also hadn't heard of the SEI levels, but I think I'd just say there are two types of software companies. The first is where software development is isolated from the rest of the company's development processes. I think this could be how 80% of companies may do things. The second is where software development is integrated with the rest of the company. This might be how 20% of companies may do things. Probably less.
Being a software developer, I naturally think it is backwards. 80% of companies should have software development integrated with all other processes. 20% of companies may be doing the right thing to isolate software development from other processes as long as the requirements and expectations on software are pretty stable.
I'm sure many companies do not realize how broad software develpment has become. These days, artists may write software as part of scripting in everything from Flash to Maya. People considered not to be software developers may be using Perl or Python or other languages to get important work done. I think many companies may still consider those who use system programming languages to be software developers and those who use scripting languages to not be software developers. I think they are all software developers and are all building important tools that need to be collaborated at the same level.
I have 7 years of software development work, and 20+ years of overall development for my own edification and pleasure under my belt. I was thinking about software development when my peers were just trying to learn basic algorithms.
I studied and used various methodologies, and always came to the same conclusion: commonsense approaches to software development trump all scientific methods. At first glance this seems counterintuitive. However, I could not deny what my senses were showing me, time and time again.
The first thing I noticed is that classical methods of measuring productivity fail for software development. You can, and many times do, have a developer who produces large amounts of code that is completely useless. In the same time span you have another developer who produces considerably less code - code that consistently works. Quality needs to come into the equation, but is a factor that is generally unmeasurable by the bean counters, and thus ignored except for lip service.
Secondly, most groups that I have had the pleasure(sic) of working with only know and use a strict waterfall methodology. While this method may work for small projects, it is unsuitable for large complex system development. Instead of releasing early and often, putting incremental releases in the hands of users for real and direct feedback about the correctness of design assumptions, we spend large sums of money and time building a 'complete' release, that invariably fails - and requires even more time and money to correct over the lifetime of the project; in many instances never satisfying the vision of the end users at all.
Finally, the propensity of people to fall in love with more and more complex development tools has raised the process itself above the real purpose: providing useful tools for our user communities.
As a result, I have made it my mission to bring agile development methods into all of my projects. I have had modest successes that go above and beyond anything years of the 'process' quagmire has accomplished. However, you can't be afraid to embrace the idea that your original specification will change many times over, you will release 'incomplete' applications in the course of finding the correct solution, and your bean counters will not be able to quantify your work units - as you build really useful applications for your stakeholders.
A couple of years ago I took a job with a small-medium sized telco company. They were hiring Java developers to work on an equipment management tool (Swing/SNMP etc). The work had previously been outsourced but to cut costs the company had decided to bring in inhouse and hire 3 full-time developers (yes, that's right, to *save costs* :)
I was the first of 3 developers hired, and had a few weeks grace to get the feel of the existing code before we had to get stuck into performing the 300 odd outstanding bug fixes and change requests. As I started looking through the code, a cold sensation went down my spine. It was an absolute bloody shambles. 500+ classes, an amazing array of over-inheritance, heaps of "cut & paste" blocks replicated through-out the code, no comments, no UML, no docs. It worked, sort of.
The worse part was the exception handling. Heaps and heaps of try/catch all blocks with NO EXCEPTION HANDLING. They just caught the java.lang.exception base class and then did something like:
try { doSomething() } catch( Exception e ) { //should never happen, so ignore }
I did a search and found about 80 odd locations where this occured. I put in simple stack trace dumps to just verify that the exceptions were or were not occuring. I ran the program and the console went mad with exception dumps.
Management's plan was to rapidly extend this product and get it into production ASAP. I wrote up a detailed analysis, outlining all the inherient code & design problems and went to my manager, explaining my concerns with the quality of what we had taken over.
His response?
"Just make it work."
No discussion. No reading of my analysis. No re-evaluation of milestone dates. "Just make it work".
Not surprisingly, this company went broke, laid off 80% of it's staff and got sold to a compeditor for a pitance.
Previous Topic
