Weblogs Forum - The Holistic Approach to Software Engineering as a Way to Handle Complexity

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
The Holistic Approach to Software Engineering as a Way to Handle Complexity

74 replies on 5 pages. Most recent reply: Dec 9, 2010 9:28 AM by Mike Swaim

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 74 replies on 5 pages [ « | 1 2 3 4 5 | » ]

John Wellbelove

Posts: 72
Nickname: garibaldi
Registered: Mar, 2008

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 24, 2010 5:32 AM

Maybe there's a software law to similar Newton's 3rd.

"Every advance in capability of tools has an equal and opposite advance in the complexity of the problems to be solved"

Posts: 18
Nickname: evarga
Registered: Feb, 2006

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 24, 2010 5:35 AM

Just an additional note to my revious post. Inside some well defined domains Software Product Lines may solve many reuse issues. However, reusing accross different domains is a major hurdle. This problem is referred as Resue at Large.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 24, 2010 6:42 AM

> I see now what you are basically trying to say. Actually,
> this phenomenon is a pretty known conundrum in the field
> of software engineering, and in the software industry in
> general, and it is called Reuse at Large. For more
> details, please, refer to the book (written by Robert L.
> Glass) what I have cited in the References section of the
> article. He succinctly explains the differences between
> Resue in Small and Reuse at Large, as well
> as, the associated problems.

I am not really talking about reuse at large; I am talking about reuse, either large or medium or small. The problem with reuse is that we are trying to reuse implementation instead of concepts. Knowledge should not be embedded in software, it should be embedded in a form that has no implementation details.

Posts: 18
Nickname: evarga
Registered: Feb, 2006

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 24, 2010 6:58 AM

>
> So we have a
> holistic concept to start with. What distinguishes writing
> practices is the ability to reason about correctness and
> apply objective criteria to its products.
>

Exactly! Thanks Kay for this very good and short recapitulation about everything what has been posted here!

As a side-note: Ideally, as Kay has mentioned, we could have a re-integration of software engineering curricula under one general umbrella. Unfortunately, this is an extremely hard task. The academic world is known to be very resistant to fundamental changes (even much smaller changes are difficult to succeed). This is the reason why I have chosen the B plan with certification. It is more of a tool for reaching that common knowledge, and “implementing” the holistic approach to software engineering. Certification may play the role to smooth the differences between various faculty educational programs. For example, in many countries in Europe there is a title called dipl. ing. (I also got that title) what you get when you finish your study at the university for engineering. It means something like graduated engineer. On the other hand, the situation in the USA is much different. There are even big differences between various dipl. ing. titles depending what university issued it.

This is one of the reasons why we have today such a “colorful” software engineering world, or as my friend Heinz has mentioned, a wild west of software engineering.

Posts: 18
Nickname: evarga
Registered: Feb, 2006

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 24, 2010 7:09 AM

>
> Knowledge should not be embedded in
> software, it should be embedded in a form that has no
> implementation details.
>

I'm puzzled here how you are going to reconcile your statement above with your earlier statement shown below:

>
> Programming languages, as they are today, make it hard
> to express that knowledge in a form that is immediately available to us.
>

Programming languages are exclusively associated with software construction, thus, they are the primary place where implementation details are kept.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 25, 2010 1:58 AM

> >
> > Knowledge should not be embedded in
> > software, it should be embedded in a form that has no
> > implementation details.
> >
>
> I'm puzzled here how you are going to reconcile your
> statement above with your earlier statement shown below:
>
> >
> > Programming languages, as they are today, make it hard
> > to express that knowledge in a form that is immediately
> available to us.
> >
>
> Programming languages are exclusively associated with
> software construction, thus, they are the primary place
> where implementation details are kept.

It has to be some form of programming language that is void of implementation details - it provides only the 'logic', the pure algorithms, if you wish.

Functional programming languages are quite close to that, actually.

Alex Stojan

Posts: 4
Nickname: staksi
Registered: Sep, 2010

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 25, 2010 9:17 AM

> > >
> > > Knowledge should not be embedded in
> > > software, it should be embedded in a form that has no
> > > implementation details.
> > >
> >
> > I'm puzzled here how you are going to reconcile your
> > statement above with your earlier statement shown
> below:
> >
> > >
> > > Programming languages, as they are today, make it hard
>
> > > to express that knowledge in a form that is
> immediately
> > available to us.
> > >
> >
> > Programming languages are exclusively associated with
> > software construction, thus, they are the primary place
> > where implementation details are kept.
>
> It has to be some form of programming language that is
> void of implementation details - it provides only the
> 'logic', the pure algorithms, if you wish.
>
> Functional programming languages are quite close to that,
> actually.

Are you talking about something similar to what Alex Stepanov and Paul McJones talk about in their book "Elements of Programming", where one uses logical reasoning and abstract algebra on various properties of objects to create generic algorithms? I think STL is also close to that.

Chris Dams

Posts: 14
Nickname: chrisd
Registered: Sep, 2009

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 26, 2010 1:39 AM

I doubt there is a programming language without implementation datails. It is not functional programming. When one programs for mathematics it is quite good, but current computer use is not very functional. The file system is not functional and neither is the network or the database. This leads to monads and whatnot. Lots of implementation details there. Also the question how one does in-place sorting in a functional language is quite "intesting" and certainly not devoid of implementation detail. One more example: the question what is lazily evaluated and what eagerly. Seems like an implementation detail to me. I guess these problems will never change as actually the world seems more object-oriented than functional to me.

The STL is a nice idea, but still quite full of implementation detail. C++ templates and implementation detail? Anyone?

Knuth comes to mind. He said something along the lines of "If somebody asks for a programming language in which you only need to specify what to be done, give him a lollipop".

In practice our programming languages are mostly already quite good. At least if one stays away from things like XSLT or worse (in general any of the many horror shows that the W3C created).

The bigger problem is that the people before us have generally already made a mess of the code that we need to extend. Human discipline is clearly the weakest link in the whole chain. Tools are not of much help with that and one might even argue that better tools have resulted in worse programming.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 26, 2010 5:11 AM

> Are you talking about something similar to what Alex
> Stepanov and Paul McJones talk about in their book
> "Elements of Programming", where one uses logical
> reasoning and abstract algebra on various properties of
> objects to create generic algorithms? I think STL is also
> close to that.

I haven't read that, but from the sound of it, I might say that it sounds close to what I am talking.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 26, 2010 5:17 AM

> I doubt there is a programming language without
> implementation datails. It is not functional programming.
> When one programs for mathematics it is quite good, but
> current computer use is not very functional. The file
> system is not functional and neither is the network or the
> database. This leads to monads and whatnot. Lots of
> implementation details there. Also the question how one
> does in-place sorting in a functional language is quite
> "intesting" and certainly not devoid of implementation
> detail. One more example: the question what is lazily
> evaluated and what eagerly. Seems like an implementation
> detail to me. I guess these problems will never change as
> actually the world seems more object-oriented than
> functional to me.

I did not say that functional programming languages will save the day. I said that programming languages lack the knowledge, and this knowledge could be supplied to them in the form of a language that looks like a functional programming language.

> Knuth comes to mind. He said something along the lines of
> "If somebody asks for a programming language in which you
> only need to specify what to be done, give him a
> lollipop".

Irrelevant to the discussion. No one will ever propose a language that is a description of what is to be done. The 'how' also needs to be catered for.

>
> In practice our programming languages are mostly already
> quite good. At least if one stays away from things like
> XSLT or worse (in general any of the many horror shows
> that the W3C created).

No, our programming languages suck. Big time. There is tremendous room for improvement.

> Tools are not of much help with that and
> one might even argue that better tools have resulted in
> worse programming.

I suppose you care to back up your (faulty) assertion with some examples...

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 28, 2010 8:47 PM

> No, our programming languages suck. Big time. There is
> tremendous room for improvement.

Then why is it that all that happens is, we keep creating "new" ones that are only syntactically different from existing ones? Adopting new syntax isn't progress. While we ignore the issue: users care about the data, not the language. We have a robust model of data, but not the one (hierarchy) which has bubbled up to popularity from the distant past of IT/SE. Why do we constantly babble about API's, but refuse to deal with data in a systemic way? All of these language zealots compete to become the dominant language (COBOL before, and java now, though possibly fading), so as to enforce the use of their particular API set. A tower of Babel is the result.

Yes, I'm talking about Dr. Codd's invention. There's no better data model (arguably the only data model), but coders keep re-inventing implementations of old algorithms in freshly minted syntax (those "new" languages). The only point to this charade is to keep churning the process, such that "old" skills are displaced by "new" skills; yet nothing semantic or fundamental has changed.

The whole multi-processor/parallel/concurrency meme is a perfect case in point. Today's young-uns believe that this is a new class of processor, and that it requires a new class of language to be able to solve old/existing problems. Functional languages are said to be the answer. But such machines were profligate in the 1980's, and, lo and behold, there just weren't many problems such machines could answer better than others. Language wasn't the stumbling block; appropriate problems was. Such machines are perfect for supporting multi-user server programs (ones which can parallelize individual queries to some data; running individual queries serially through a core/processor is a solved problem). No one, and I repeat NO ONE, has found a class of client/standalone problems which benefit from these processors. Until such is accomplished, this will be a fools journey.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 29, 2010 3:12 AM

> Then why is it that all that happens is, we keep creating
> "new" ones that are only syntactically different from
> existing ones? Adopting new syntax isn't progress.

It's only Haskell that is created because there was a theoretical advancement in computer science. All other languages are born due to economic reasons or fun projects: Java started as Oak because its creators couldn't program their embedded devices in C++, C# started because Microsoft wanted to rival Java, D started because Walter Bright wants to be a programming language pioneer etc.

> While
> we ignore the issue: users care about the data, not the
> language. We have a robust model of data, but not the one
> (hierarchy) which has bubbled up to popularity from the
> distant past of IT/SE. Why do we constantly babble about
> API's, but refuse to deal with data in a systemic way?
> All of these language zealots compete to become the
> e dominant language (COBOL before, and java now, though
> possibly fading), so as to enforce the use of their
> particular API set. A tower of Babel is the result.

Very very true. That is why I said: "make the computer information-aware. Computers are currently information-agnostic".

>
> Yes, I'm talking about Dr. Codd's invention. There's no
> better data model (arguably the only data model), but
> coders keep re-inventing implementations of old algorithms
> in freshly minted syntax (those "new" languages). The
> only point to this charade is to keep churning the
> process, such that "old" skills are displaced by "new"
> skills; yet nothing semantic or fundamental has changed.

SQL is fine and sweet, but its data types do not match what computers can currently do efficiently. The relational model is fine and sweet, but it needs to be the main way to storing information on a computer.

>
> The whole multi-processor/parallel/concurrency meme is a
> perfect case in point. Today's young-uns believe that
> this is a new class of processor, and that it requires a
> new class of language to be able to solve old/existing
> problems. Functional languages are said to be the answer.
> But such machines were profligate in the 1980's, and, lo
> o and behold, there just weren't many problems such
> machines could answer better than others. Language wasn't
> the stumbling block; appropriate problems was. Such
> machines are perfect for supporting multi-user server
> programs (ones which can parallelize individual queries to
> some data; running individual queries serially through a
> core/processor is a solved problem). No one, and I repeat
> NO ONE, has found a class of client/standalone problems
> which benefit from these processors. Until such is
> accomplished, this will be a fools journey.

Again true, but it would help if we can improve our methods of developing software, even if there is a limited number of problems that can be solved with parallelism/concurrency.

Morgan Conrad

Posts: 307
Nickname: miata71
Registered: Mar, 2006

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 29, 2010 9:20 AM

>No one, and I repeat NO ONE, has found a class of client/standalone problems which benefit from these processors.

Its still in it's infancy, but genetic analysis is starting to use of massively parallel processing:

https://nvidia.confreg.com/gputechconference/schedule/by-session/#Topics=Algorithms_~_Numerical_Techniques

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 29, 2010 11:29 AM

>While we ignore the issue: users care about the data, not the language.

Yes, they care about the data. They also want it filtered, analyzed, and packaged in a meaningful way. Power users want to do it themselves but don't want the messiness of coding. Business leaders want company processes encoded and supported by software. Coding is necessary. Modern languages abstract away messy parts of coding applications.

Two common and problematic coding errors - off-by-one and null pointer exceptions - are relatively infrequent now for organizations using modern languages. Even if we restricted the conversation to Java, I'll take its safeguards over debugging SIGSEGV's caused by a C program or scrambled data from an overwritten null terminator.

Application developers can now concentrate less on the machine and more on the application domain they've been hired to support. Scala and F#, just to take representatives from the Java and .NET platforms, give us the ability to achieve more with fewer "white noise" lines of code. If I'm working with a list of university application users in memory and want, for instance, all the students, why go through the bother of using an index variable and loop? What value does that add? More importantly, what potential errors does it introduce? Thanks to "for comprehensions" I don't need to remember if my language starts the index at 0 or 1; or allocating memory for each object in the new list; or what to call the index variable to make it through the code review police.

I'm not downplaying the benefits of SQL and a good relational model, but use of other programming languages is necessary, has been messy, and is IMO getting less so.

>The only point to this charade is to keep churning
>the process, such that "old" skills are displaced
>by "new" skills; yet nothing semantic or fundamental
>has changed.

This statement points to a problem with our industry. Tools get identified with
skills and that needs to change. Rather than identifying someone as a Java vs. C# vs. COBOL developer, we should identify by more meaningful indicators such as experience in a problem domain, in varying software project sizes, and in life critical systems. For instance, writing a large custom financial analysis application with a team of 50 takes soft skills that are less necessary when writing a data mining application in a team of 3 for genetic data. Both domains require specific terminology and knowledge necessary to support them.

>Why do we constantly babble about API's, but refuse to
>deal with data in a systemic way?

Would you please elaborate on what you would consider dealing with data in a more systemic way?

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: The Holistic Approach to Software Engineering as a Way to Handle Complexity

Posted: Nov 29, 2010 11:50 AM

> >No one, and I repeat NO ONE, has found a class of
> client/standalone problems which benefit from these
> processors.
>
> Its still in it's infancy, but genetic analysis is
> starting to use of massively parallel processing:
>
> https://nvidia.confreg.com/gputechconference/schedule/by-se
> ssion/#Topics=Algorithms_~_Numerical_Techniques

I'll grant that, and certain simulations: nuclear warheads and weather come to mind.

I should have been a tad more specific (I thought it obvious from this and earlier threads on the topic): "a class of... which confront a large number of users"; i.e. some (sub)class of the PC consuming population who constitute 99.44% of shipments of these cpu's. For those users, I believe my statement is correct.

The Connection Machines, et al, of the 80's never found any, and these were machines which could mine the scientific/engineering space as well. That space (as you've pointed out) is certainly still available, but won't move enough parts to make a decent bottom line.

Amdahl's Law still holds, especially in the mass market space which is where some are trying to find a way to morph applications into the parallel universe. The prospects, so far, are not good.

Flat View: This topic has 74 replies on 5 pages [ « | 1 2 3 4 5 | » ]

Previous Topic

Next Topic