Summary
Complexity entails completeness! The software industry should admire and support a holistic approach to software engineering. Unfortunately, the industry does not herald a holistic viewpoint, since the software industry has a wrong treatment of complexity. This has to be changed!
Advertisement
Introduction
A week ago, I have talked in a webinar about software engineering and the IEEE Certified Software Development Professional certificate. The whole webinar was recorded, and it is hosted by the organizer Dr. Heinz M. Kabutz on his server. See the resources section for a download link, or click here.
My essential message was that the software industry should admire and support a holistic approach to software engineering. Such ‘people centric’ attitude is the sole way to guarantee success, and deliver high quality products to end users. People are the key in the software engineering profession, and this should be clearly emphasized! Certificates (like IEEE CSDA/CSDP) are one way to reach in a well guided fashion the desired level of proficiency in the field of software engineering.
Changes on the Horizon
Nowadays, we may hear voices about redoing things, i.e. changing current practices, in the software industry. Let me just give you a couple of hints:
The resurgence of parallelism: the authors (Peter J. Denning and Jack B. Dennis) nicely explain the reasons why the software industry today is hectically “reinventing the wheel” regarding parallelism, and what happened in that long forgotten epoch of parallel computation (during 60s and 70s in the last century). For more information see the resources section, or click here.
Expanding CS education: the author (Ed H. Chi) contemplates that it is time for reforming the computer science curriculums, as computer science is on a verge to recognize and submit new child sciences, since it got very large. This is a natural consequence of growth and expansion; all well established sciences had been through this process long time ago. For more information see the resources section, or click here.
Reinventing business: Bruce Eckel has started a project to even rethink how the industry should organize itself, and how should we build natural, friendly and lovely work environments. Otherwise, this web site is a gathering place of innovative ideas toward making a better future in the world of the software industry. For more information see the resources section, or click here.
Complexity Revisited
By my opinion, the software industry has come to the critical point where it needs to admit and correct errors made in the past. In other words it is a time for a change! I think, the biggest mistake in the software industry, and hence the urgent need for a change in this respect, is associated with the improper treatment of complexity. From the outset, the industry got intimidated by the vast complexity inherent in software development. However, it was not complexity what had caused great havoc in the field, but the amiss handling of that complexity. Instead of being proud on that complexity, the industry even now feels ashamed about it!
If we look into the past (not quite huge past actually) we may notice one general tendency in our field: a constant attempt to hide complexity and marketing that software development is affordable to everyone! What a waste of energy, I would say! Just imagine where medicine would be today, if they would had invested a stupendous time, money and resources to find a way to make “Hello World!” type surgery (like, appendectomy) doable by any fool! Would that be a nice world to live in?
That same complexity is also hidden from end users, too! All complexity what they in fact perceive about a software product is solely the complexity related to the user interface, i.e. to the usage of the software. They do not have a feeling what is under the hood! Take, for example, some other engineering discipline, like mechanical engineering. Let us consider a car, as one representative product. It surely has a very simple user interface; otherwise only few would be able to drive it. Nevertheless, people can always open the engine hood, and sneak a quick look at the underlying complexity. There is no such thing in software. As a consequence, lay people think that building software should not be so hard, after all. Imagine their standpoint, especially after they manage to write their first “Hello World!” program, and successfully run it on their computer! They immediately get courage even to start quarrelling with professionals!
I’m not saying here that complexity should not be reduced were possible in order to better cope with hard problems. I’m just emphasizing that the software industry should not foster blunders that developing software is easy! Software development is a complex task! This must be the starting point! The industry should not buttress anymore those directions whose sole purpose is to reduce complexity just in order to make software construction possible to everyone. The industry must change the course, and reduce complexity primarily for professionals to be able to cope with hard task! Of course, everyone will have been welcome to such a transformed industry, but the decision regarding who is capable to really contribute even a simple program would be in the hand of natural selection instead of sales people. Only the most fitted ones would be able to join!
What should be the next steps?
To accentuate loudly that software development is a complex endeavor!
To highlight the fact that complexity entails completeness!
To start educating software people that only a holistic approach to software engineering is the right way to go in order to triumph over the current “software crisis”!
To start educating software people that only a holistic approach to software engineering is the right way to go in order to triumph over the current “software crisis”!
So what is exactly meant by software crisis? How can you prove there is a software crisis? I mean we have better tools than ever to develop software (eclipse, programming languages like scala, java, continuous integration tools like hudson and so on), so I think the software community is on the right way. Using tools to ensure software quality is - imho - a sign of professionalism.
Posts: 18
Nickname: evarga
Registered: Feb, 2006
Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity
> So what is exactly meant by software crisis? How can you > prove there is a software crisis?
Your comment itself and your viewpoint are maybe the best proof! People do not even understand what software engineering is. This is among the biggest issues what we are facing today in the software industry.
> I mean we have better > tools than ever to develop software (eclipse, programming > languages like scala, java, continuous integration tools > like hudson and so on), so I think the software community > is on the right way.
Tools are just one small part in the overall picture. Yes, they are indeed an important piece, but without the proper knowledge from the other areas of software engineering their effectiveness are seriously hindered. It is absolutely evident that the majority of the community actually got trapped into a pitfall that tools will solve everything. What a blunder!
BTW: For those who sell tools this situation is a paradise! They probably pray each day for people to stay on the "right" way!
>> Your comment itself and your viewpoint are maybe the best >> proof! People do not even understand what software >> engineering is. This is among the biggest issues what >> we are facing today in the software industry.
okay, maybe you have a bigger brain or something
>> Tools are just one small part in the overall picture. Yes, >> they are indeed an important piece, but without the proper >> knowledge from the other areas of software engineering >> their effectiveness are seriously hindered. It is >> absolutely evident that the majority of the community >> actually got trapped into a pitfall that tools will solve >> everything. What a blunder!
>> BTW: For those who sell tools this situation is a >> paradise! They probably pray each day for people to stay on the "right" way!
First of all: Just because a company sells something it does not have to be necessarily bad. Just because a company sells bread bread is not bad. Think about your argument: If the topic was bread then you could have said the same: Companies which are selling bread would be happy to hear ... so the argument is deeply flawed.
Second: Languages are part of the toolbox too. So as languages evolve, as they enable us to express abstractions in an easy way the task of the programmer gets easier and he delivers products faster and also more reliable products. Same with continuous integration: If you use it errors in the code are exposed early, as a result software with less bugs is delivered.
I wouldn't say tools are the solution, but they are in most cases an important part of the solution. Of course without people nothing is possible. People have to be trained to use these tools. Would you use a text editor and C if you would have to implement a web application (say: web shop) or rather grails, java, eclipse..?
> To start educating software people that only a holistic > approach to software engineering is the right way to go in > order to triumph over the current “software crisis”! > > So what is exactly meant by software crisis? How can you > prove there is a software crisis?
Good questions, Fred.
I had a look on the only true source of information (Wikipedia) and found this description:
"Software crisis was a term used in the early days of computing science. The term was used to describe the impact of rapid increases in computer power and the complexity of the problems which could be tackled. In essence, it refers to the difficulty of writing correct, understandable, and verifiable computer programs. The roots of the software crisis are complexity, expectations, and change."
They list the following problems that appear due to this "software crisis":
* Projects running over-budget. * Projects running over-time. * Software was very inefficient. * Software was of low quality. * Software often did not meet requirements. * Projects were unmanageable and code difficult to maintain. * Software was never delivered.
I would say that quite a few of these problems are nowadays managed better due to our excellent tooling and agile processes. I think that continuous integration and unit testing solve a bunch of these old issues.
So I am on a quest here as well - trying to find out what the IEEE CSDP is and whether this will help us in the industry to produce better software. If the IEEE CSDP was modern and incorporated modern software engineering principles, then I think it could.
According to their web site, you don't need qualifications or experience if you join the IEEE.
Unfortunately, I can't join the IEEE because my surname "O'Sullivan" contains an invalid character. (Note to IEEE: No, it doesn't.)
Also, the username I chose contained lower case characters and only upper case ones are permitted. (Second note to IEEE: The inability to handle mixed case and non-alphanumeric characters on a simple web page is not a good advert for the quality of an organisation that sells software development certification.)
Posts: 18
Nickname: evarga
Registered: Feb, 2006
Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity
Ok, thanks for these really valuable feedback, I see know what is the main problem. I once again apologize for my earlier improper reactions, and I’m in the process of learning how to blog effectively!
What I mean by “crisis”?
In retrospect, I would now actually like to thank Fred for this question. I also appreciate Morgan’s comment, and will try to be succinct, as much as, possible. The video really got a bit long, and I know that nowadays people have little time to devote watching the whole stuff just to extract the essential info what they are looking for. On the other hand, the topic is complex so that there is always a danger of misunderstanding it if the explanation is too abridged.
I am a software engineer, and I always balance between serving my profession and serving the customers, who are using my product. In other words, I always try to follow the IEEE software engineering code of ethics (http://www.ieee.org/membership_services/membership/ethics_code.html). I am pretty aware that what I will say here will probably put me on the “black list” in many groups, nevertheless, I know that my intentions are honest, and serving the community.
There is a “crisis” in the software engineering profession, which directly or indirectly has impact on the software industry, as well. What I do not like today from the viewpoint of my profession? I do not like the position of a software engineer in the industry, and his/her treatment. In other words, I do not like who has the control over our profession, and our end products.
Which forces tear the software engineer profession? There are two main sources: disconnection between the management and the development department in companies, and disconnection between academia and the software industry.
On one hand, the management lives in his own virtual world, and thinks that the whole success of a project is associated with those "scientific" management hocus-pocus, and not because of the ingenuity of a software engineer. What they think is that coordinating design people, programmers, testers, etc. is the real art. What is then the role of a software engineer? Well, for them, they are just a bunch of replaceable resources whose sole job boils down to the usage of some superb tool, as some kind of operators. This is the reason why I put that section ‘Complexity Revisited’, and highlighted the fact that developing software is complex and mostly an intellectual task! A software engineer is not some bare tool “guru”; he/she is lot, lot, lot more than that! This is the principal reason why I always get so upset, when I hear that there are even software people who support this false view on their own damage.
On the other hand, there is another virtual world called academia. They are also shaping the software engineering profession. Unfortunately, due to the disconnection between their virtual world and industry, they try to push methods, which simply do not work. I have explicitly mentioned this in the Webinar (there is a separate addendum about this issue).
Who suffer directly? Both software engineers and their products. Who suffer indirectly? End users of those products.
Therefore, my article and my attempt for help here is for the benefit of us, software engineers. As it is the case with a software quality (that quality is the responsibility of all participants in the projects) this is also true for our profession. If we do not start to fight against all attempts to get from us respect, dignity and control then we should no wonder why software people are on the lower ladder then some “gurus” from the management.
What to do? This is the reason why I have written this whole article. To try to convince software people that it is time to appreciate the holistic view of our profession, and not that false view associated with high specializations (pure analyst, pure designer, pure programmer, etc.). I gave a fine example how this holistic view has direct impact on software quality (case study 2). To try to convince people that our profession is hard, and developing software is a complex endeavor. To stop supporting that false view that everything is easy today, since we have excellent tools.
Why certification? Because this is one way to get that desired level of proficiency in a well guided fashion. I’m not insisting that IEEE CSDP is the only route. I heard that there are also other similar certificates. The main point is that software engineering certificates are kind of reuse of the curricula for continued education.
Summary
I hope that I have now better explained my viewpoint, and tried to convince software engineers that the first important step is to boost their knowledge regarding our profession. Whether my appeal will be supported or not by the community, or be totally exposed to a public scorn, is not so important for me. Maybe I’m also totally wrong how to make the next step. However, my mission is honest, at least, I may say: “Well, I have tried!”
Addendum 1
J M has made another excellent remark regarding the other engineering domains. This comment has once again proved an old wisdom that a person should not review his own work. It is absolutely true what J M has mentioned that other disciplines also experience quite often changes in the requirements. What I wanted to say in the Webinar is actually the relative frequency of changes in other domains compared to those in the software industry. In this sense, we may say that the number of changes in the software is larger than in other domains, though.
Addendum 2
I had talked once with a lawyer, and she told me about new stringent rules who may qualify to become a member of the bar association. She added, “you know, if we lawyers do not protect our profession, who will?”
Mapped to our case, if we, software engineers, let others take care of us, I have no doubts whose interests will prevail!
> Which forces tear the software engineer profession? There > are two main sources: disconnection between the management > and the development department in companies, and > disconnection between academia and the software industry. : : > Why certification? Because this is one way to get that > desired level of proficiency in a well guided fashion.
You say that the two main problems are disconnection; with managers and with acedemics. How is this addressed by certifying software engineers?
Interesting explanation, Ervin. I think I'm slowly understanding what you are trying to say :-)
If I may paraphrase? Imagine a dinner party and we go round the table saying what we do. John sitting next to me says he is a Chartered Accountant. Jim sitting opposite me says he is a Mechanical Engineer. I say I am a programmer. Or do I say I am in IT? Or do I say that I am a Software Engineer?
For the first two professions, no follow-up questions are necessary, because they follow a well-defined charter, even though accountants differ wildly from each other. But with my "profession", I will be asked follow-up questions like, "what type of work do you do?" If I then say: "I am a Java programmer.", it is invariably followed with statements like: "Oh yeah, my brother's nephew is a wizz with Javascript."
So by having some certification, it is helping us define what a Software Engineer is. My personal viewpoint is that I'd rather be involved in deciding what this is, rather than have it forced on me at a later stage.
We are still in the wild west of software engineering, where anybody can write software and no one is help personally accountable for the errors that they produce. If a civil engineer makes a mistake on my house and it collapses, he is personally responsible.
> I've just read the eligibility requirements for the CSDP > and, despite 30 years of programming experience, it seem I > don't qualify!
Hi John, and despite writing tons of articles and speaking at about 10 events a year, I don't think I'd be able to retain my certification status beyond 3 years.
The whole CSDP is strongly biased towards academia and that needs to change if they want this certificate to have any meaning. I think they would need approximately 100.000 certified engineers for this to become a recognizable brand. At the moment there are about 1000 AFAIK. It was by pure accident that I noticed this title with my friend Ervin Varga.
For example, the Association of Chartered Certified Accountants in Britain has 140,000 members. A similar body in India has 150,000 members. Thus 100,000 would be an absolute minimum that they need for it to become recognized IMHO.
> First of all: Just because a company sells something > it does not have to be necessarily bad. Just because > a company sells bread bread is not bad. > Think about your argument: > If the topic was bread then you could have > said the same: Companies which are selling bread would > be happy to hear ... so the argument is deeply flawed.
I think there's a lot of misinterpretation here and that Dr. Varga is generally on the right track but hasn't made his points very well in this article.
The above comment is a good example of this. I don't see anywhere that the argument was that because something is sold that it is bad. The issue is that if you ask the baker whether bread is healthy, he will likely tell you 'yes' regardless of whether it's true. In fact, bread is a major contributor to sodium in a diet; far greater than the salt-shaker. Most bakers aren't going out of their way to tell you this.
I have reoccurring discussions with people (mostly non-technical) about tools. The general idea is that if we get the right tool, we will no longer need 'rocket-scientists' to do development. Often these tools are sold as 'no programming' because the primary interface is drag-and-drop. When I explain that while tools can have distinct and significant benefits, they don't really reduce the need for skilled staff, I'm usually told that vendor says I'm wrong and that the vendor knows better than me.
The vendors are saying this because this is what their customers say they want. Does this make the tools bad or the vendors bad? Absolutely not. If they didn't market these tools this way, they would lose business to competitors who do. The key here is that a salesperson is that last people to ask whether you really want or need what they are selling. It's not the vendors that need to change, it's the consumers.
> Hi John, and despite writing tons of articles and speaking > at about 10 events a year, I don't think I'd be able to > retain my certification status beyond 3 years. > > The whole CSDP is strongly biased towards academia and > that needs to change if they want this certificate to have > any meaning. I think they would need approximately 100.000 > certified engineers for this to become a recognizable > brand. At the moment there are about 1000 AFAIK. It was > by pure accident that I noticed this title with my friend > Ervin Varga. > > For example, the Association of Chartered Certified > Accountants in Britain has 140,000 members. A similar > body in India has 150,000 members. Thus 100,000 would be > an absolute minimum that they need for it to become > recognized IMHO.
A few things here. I don't think anyone has mentioned that there is no standard 'software engineering' field of study in colleges and universities. Engineers generally graduate from engineering programs before being certified as engineers. It's not clear to me that there is common agreement of what a software engineering program/certification should entail. And finally, to my previous point above, if there is no demand for certified software engineers, there will be no widespread adoption of the certification.
The last point goes to something I think is a major issue in software development, especially IT. There far too little technical understanding in management. I would wager that if you took teams of programmers and asked their management to rate them and then had the programmers give reviews, the results would tend to vary significantly. Additionally, so much of what is bandied about by management and consultants is harmless quackery at best and dangerously misguided at worst. I read something not too long ago from a "respected" management guru to the effect of 'IT doesn't matter because companies already have so much of it'. Just the way air and water don't matter. Other nonsense I hear (explicitly or implicitly) on a regular basis:
'we will change our business processes to work with our vendor's software'
'we don't want skilled developers because the less able developers we want to employ won't understand what the skilled developers did'
'developers from county/region X are better than developers from country/region Y'
'[any problem] can be addressed with more (or less) process or a different process'
'we bought a graphical programming tool and eliminated all our code'
Posts: 18
Nickname: evarga
Registered: Feb, 2006
Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity
> > You say that the two main problems are > disconnection; with managers and with acedemics. How is > this addressed by certifying software engineers? >
I would like to thank for these superb comments up so far! I would just like here to answer the question, which was put by Vincent.
Using a military jargon, we need certifications for securing the base camp. We need a much higher cohesion between software engineers regarding our common understanding, and appreciation of our profession. This is an important precondition before starting to regain control over our industry, and our products.
I’m not stating that this common ground cannot be achieved by other means, as well. I have chosen certification for the following benefits:
- Provides well established curricula for education. - Assures consistent and systematic way of knowledge verification. - Gives repute to holders, especially in case when the certification is issued by some well recognized authority.
P.S. According to G. Ford and N.E. Gibbs a recognized engineering profession is characterized by the “Registration of fitness to practice via voluntary certification or mandatory licensing.” In this sense, a certification even constitutes an integral part of the software engineering profession.
> I’m not stating that this common ground cannot be achieved > by other means, as well. I have chosen certification for > the following benefits: > > - Provides well established curricula for education. > - Assures consistent and systematic way of knowledge > verification. > - Gives repute to holders, especially in case when the > certification is issued by some well recognized > authority.
I think you are selling this to the wrong crowd. Even if you convince all the developers in the world that certification is a good idea, you still won't have created a demand for certified engineers. This is part of the problem with existing certification efforts.
You might also want to consider when a certified software engineer would be desired. The power of software is too compelling for only a chosen few to do it. Not every web-page needs to be created by an expert. How do employers make that distinction? These are some of the practical problems you face with this idea.
On a side-note, I've been largely anti-certification for quite a while but I've started to come around. There is definitely a value to having a 3rd party confirm technical ability. I'm not going to any non-certified doctors any time soon. I think it's weird for us (the geeks) to think about it but our work is extremely esoteric. Most people don't know a bit from a byte, let alone the difference between a PHP and a C++.
Flat View: This topic has 74 replies
on 5 pages
[
12345
|
»
]
Previous Topic
