The Artima Developer Community
Sponsored Link

Weblogs Forum
Java 7: Too Little, Too Late?

64 replies on 5 pages. Most recent reply: Jan 30, 2012 1:07 PM by Steve Carmeli

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 64 replies on 5 pages [ « | 1 2 3 4 5 | » ]
Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Re: Java 7: Too Little, Too Late? Posted: Aug 29, 2011 3:47 PM
Reply to this message Reply
Advertisement
>To the extent that IDEs, frameworks and other tools succeed
> in making it possible to work on a larger codebases than a
> team could otherwise manage, they often also hide
> significant bugs. As Dijkstra said about language features,
> many times these tools belong in the problem set, not the
> solution set.

Just to flesh this out; without an IDE, a project generally is limited to whatever one person can understand. Sure, it may take a whole team of programmers to write the thousands or millions of lines of code in the project; but as long as there are programmers able to actually grok the design, the project won't go entirely off the rails.

Tools can make the drudge work more appealing, or at least less depressing. I'm not opposed to tools. I am, in fact, fascinated by static analysis tools.

But to the extent that tools allow a team to fake understanding of the codebase, they really just delay the inevitable. They allow complexity to creep in without the team actually noticing or understanding it. At some point, the team realizes that they *are* in shit creek, and they haven't had a paddle for some time.

Tools are great, but they are no substitute for thinking.

Dick Ford

Posts: 149
Nickname: roybatty
Registered: Sep, 2003

Re: Java 7: Too Little, Too Late? Posted: Aug 29, 2011 8:17 PM
Reply to this message Reply
> "Use Java, and you, too, can ship projects that rely on
> thousands of lines of code that you didn't write, but that
> you will be responsible for finding bugs in!" Sorry,
> doesn't inspire confidence.

So are compilers that generates billions of lines of machine code impervious to bugs?

Sure, any tool that generates anything can be buggy, but developers use tools to generate GUI code. Emacs generates code if you want. Should we all use notepad to code because we're afraid our IDE is going to do something nasty behind our back?



>
> To the extent that IDEs, frameworks and other tools
> succeed in making it possible to work on a larger
> codebases than a team could otherwise manage, they often
> also hide significant bugs.

Everything is a balance, and more often than not they're not generating bugs.


> As Dijkstra said about
> language features, many times these tools belong in the
> problem set, not the solution set.
>

A tool is just a tool. Computers and software generate code that launch nuclear missiles. Do computers and software belong in the problem set now?

> Scala's promise is that the language improvements make it
> easier for a team of programmers to solve larger problems
> without hiding significant bugs. Scala may well lack
> tools to allow mediocre programmers to vomit out thousands
> of lines of code without actually understanding any of it,
> but I would consider that an improvement.

You instantly lose the argument when you start "spewing" out stuff like "vomit out thousands of lines of code".

I think the point that the OP was making is that you have to downgrade the effectiveness of a language like Scala when you're stuck with antiquated Emacs to code in it. You might not like it that Scala doesn't have good tooling, but you have to deal with the reality of the power of that our modern tools give us by letting us focus on the domain and not the plumbing.

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Java 7: Too Little, Too Late? Posted: Aug 30, 2011 10:55 AM
Reply to this message Reply
> But to the extent that tools allow a team to fake
> understanding of the codebase, they really just delay the
> inevitable. They allow complexity to creep in without the
> team actually noticing or understanding it. At some
> point, the team realizes that they *are* in shit creek,
> and they haven't had a paddle for some time.
>
> Tools are great, but they are no substitute for thinking.

I'm old enough to remember the late 80s and early 90s when database/4GLs were all the rage; one, Progress, still exists. Most of the others are either gone or maintaining very old code. The real programmers of the day, C and COBOL (depending on the platform) coders, derided the approach. Still do, largely. Yet with all the source generation (as distinct from compilation, and these are different exercises) going on in IDEs and frameworks and such, all those low level java coders are truly yearning for a nice and simple 4GL. Or a language without all the boilerplate that the IDEs and frameworks vomit (sorry, couldn't resist). But it is a valid complaint; at what point does the presence and use of such crutches deny the use of the base language?

The reaction to build higher level languages on the jvm is to have a clear semantic and syntax that eliminates the need for the boilerplate altogether. Let the compilers do what they're supposed to: generate efficient machine code.

It's worth mentioning that Meyer (the Eiffel one) has said, sort of, that C is a wonderful language, but not to write in; it's a universal compiler. java/jvm could well be headed the same way.

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Re: Java 7: Too Little, Too Late? Posted: Aug 30, 2011 11:32 AM
Reply to this message Reply
> So are compilers that generates billions of lines of
> machine code impervious to bugs?

Compiler bugs do exist ( http://blog.regehr.org/archives/539 ).

> Sure, any tool that generates anything can be buggy, but
> developers use tools to generate GUI code. Emacs
> generates code if you want. Should we all use notepad to
> code because we're afraid our IDE is going to do something
> nasty behind our back? ...
>
> Everything is a balance, and more often than not they're
> not generating bugs.

It's almost like I predicted this strawman response when I wrote "I'm not opposed to tools. ... But to the extent that tools allow a team to fake understanding of the codebase, they really just delay the inevitable. They allow complexity to creep in without the team actually noticing or understanding it. At some point, the team realizes that they *are* in shit creek, and they haven't had a paddle for some time."

But, to clarify, the tools don't *create* bugs, they mask bugs. Tools often convince developers that they have a deep understanding of the code when, in fact, they don't.

It's, sadly, not unusual for a project to have one or more modules that nobody wants to touch. People generally refer to those modules as brittle or poorly designed, but in nearly all cases the problem modules didn't start out that way. What caused the decline from a usable, robust, elegant module to something that people try their best to work around? In my experience, it's nearly always a series of patches by people who do not truly understand the code in the module. Anything that gives developers unwarranted confidence is a bad thing in my book.

> I think the point that the OP was making is that you have
> to downgrade the effectiveness of a language like Scala
> when you're stuck with antiquated Emacs to code in it.
> You might not like it that Scala doesn't have good
> tooling, but you have to deal with the reality of the
> power of that our modern tools give us by letting us focus
> on the domain and not the plumbing.

That may have been the original poster's point, but I think the poster is misguided. The ability to push buttons on a magic tool isn't much of a skill to be proud of. The ability to "not make a mess of it" (as Dijkstra says) is far more important.

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Tools Posted: Aug 30, 2011 11:45 AM
Reply to this message Reply
>But to the extent that tools allow a team to fake understanding of the codebase, they really just delay the inevitable.

Case in point, and it's not even a software tool: UML. Have you ever seen a UML activity diagram catch a race condition? A UML class diagram that helped somebody chase down a bug? A UML use case that uncovered an important UI problem? I haven't.

But I *have* seen a programmer who thought that having a sheaf of printed UML diagrams demonstrated significant knowledge of the software in question. It's hard to explain to somebody that they've just engaged in a colossal amount of busy work without getting any closer to understanding potential problems in the design.

Bruce Eckel

Posts: 875
Nickname: beckel
Registered: Jun, 2003

Re: Tools Posted: Aug 31, 2011 12:27 PM
Reply to this message Reply
> But I *have* seen a programmer who thought that having a
> sheaf of printed UML diagrams demonstrated significant
> knowledge of the software in question. It's hard to
> explain to somebody that they've just engaged in a
> colossal amount of busy work without getting any closer to
> understanding potential problems in the design.

Testify!

On occasion a good and useful tool appears. But it usually emerges from the grass roots, to solve a pain point that people are actually experiencing. And even the good tools risk being co-opted by marketing (the greatest thing about open source is that a product can excel or sink on its own merits; there being little financial incentive for marketing). I lived through numerous modeling wars and it felt like there were MBAs at the wheel.

Morgan Conrad

Posts: 307
Nickname: miata71
Registered: Mar, 2006

Re: Tools Posted: Aug 31, 2011 1:43 PM
Reply to this message Reply
I have found UML Sequence diagrams to be sometimes worth the time invested to make them, e.g. in cases where the interactions are complex. Sometimes you discover a good refactoring to simplify things. And a class diagram, if you leave out almost all the interior details, is occasionally worth the effort - at least to give a broad overview to maintainers. But, in general, I agree that UML isn't a solution.

And many of the tools stink and are tedious. Several years ago we used Rational Rose. The project manager got some good use out of it, it worked for him. But two of us somehow clicked differently, and, literally, within five minutes we nearly always crashed the application.

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Re: Tools Posted: Aug 31, 2011 2:18 PM
Reply to this message Reply
I feel like I've been all over the place, so I want to boil down my arguments:

Axiom 1

The biggest source of trouble in software is code that programmers don't understand.

There are several places to get such code, and it's useful to identify those sources and root them out as much as possible.

Axiom 2

Any code in your project is your responsibility, regardless of where you got it.

Axiom 3

Anything that convinces programmers that they understand code when, in fact, they do not, is a one-way ticket to the inner circle of Hell.

My complaint about auto-generated code comes from applying these principles. The fact that you know which version of a tool and which settings you used in generating code does not, in fact, mean that you understand the code in question. And, odds are, you will one day need to.

As an example; some projects I work on use Protocol Buffers. Protocol Buffers generates pretty solid code and has great documentation. Until you want to do something not mentioned in the documentation -- say serialize various NaN values or denormalized floats in one language and deserialize them in another. If you try to see if this will work by stepping through the auto-generated code, you can kiss your sanity goodbye.

I have to assume that the hairball of code that Protocol Buffers spits out is a hairball because it's easy to generate; but I can't call it anything other than a massive hairball (please note that the API that the generated code exposes in not a hairball; I'm referring to the internals of the generated code). The structure of code is far from obvious, there is no way to determine invariants, and the only comments in the files are -- not surprisingly -- statements that you should not edit the files if you ever plan on rerunning the Protocol Buffer compiler.

Alternatively, I know that I'm not the only programmer who's ever had auto-generated code throw a null pointer exception. And the only way to determine whether there is an error in the generated code or in my code calling it is to find out what details are being swept under the rug by the generated code (i.e., to understand the generated code). And, again, generated code is easy to generate, not necessarily easy to read.

On a different note, I'm fascinated by several tools that don't generate code; static analysis and modeling tools are prime examples. But most modeling tools seem to think of the problem at a level that's not useful for implementing a solution. They give people unwarranted optimism, and when things fall apart you have nothing to guide you out of the swamp you've been dumped in.

James Watson

Posts: 2
Nickname: banned
Registered: Sep, 2011

Re: Tools Posted: Sep 2, 2011 10:54 AM
Reply to this message Reply
I think there might be a misunderstanding of what is meant by tooling in Java. While this could mean tools that generate code, this is not the kind of tooling that comes to my mind. The kind of tooling that comes to my mind is things like "show references" and "open declaration". Just the other day I was sitting with someone who was doing text searches on Java code. In the time he did this and found what he was looking for, he could have imported the code into Eclipse times and done a "show references" 10 times and had better results.

Ultimately, we found that there was a call to an undocumented method that was at the root of the issue. He was stuck. If he had the code imported into Eclipse and linked to the source of the library, he could select "open reference" and we would have it in front of us in moments.

Other tools like findbugs, checkstyle, JDepend, are really useful but have nothing to do with code generation.

On a side note, while I tend to hate tools that generate java source code, it's good to keep in mind that compilers are tools that generate code.

John Zabroski

Posts: 272
Nickname: zbo
Registered: Jan, 2007

Re: Java 7: Too Little, Too Late? Posted: Sep 2, 2011 2:33 PM
Reply to this message Reply
> I'm old enough to remember the late 80s and early 90s when
> database/4GLs were all the rage; one, Progress, still
> exists. Most of the others are either gone or maintaining
> very old code. The real programmers of the day, C and
> COBOL (depending on the platform) coders, derided the
> approach. Still do, largely. Yet with all the source
> generation (as distinct from compilation, and these are
> different exercises) going on in IDEs and frameworks and
> such, all those low level java coders are truly yearning
> for a nice and simple 4GL. Or a language without all the
> boilerplate that the IDEs and frameworks vomit (sorry,
> couldn't resist). But it is a valid complaint; at what
> point does the presence and use of such crutches deny the
> use of the base language?

To answer your last question: Ideally, at no point. Have you ever watched Guy Lewis Steele's OOPSLA talk "Growing a Language"?

But there are people out there who realize that the code required to build operating systems today would be a 1 meter high stack of books, and that it would be nicer if we could create a novel. (And not the sort of novel John Updike would write! Hah!)

Check out Alan Kay's organization, VPRI. On their website are a number of research reports from their Foundations of New Computing (FONC) project. You can also e-mail me for more info, since I follow the project and am on the mailing list. Alan's goal is to build a complete operating system in 20,000 lines of code. For comparison, the smallest operating system he has built, Squeak, is 600,000.

Dick Ford

Posts: 149
Nickname: roybatty
Registered: Sep, 2003

Re: Java 7: Too Little, Too Late? Posted: Sep 4, 2011 9:02 PM
Reply to this message Reply
> > I think the point that the OP was making is that you
> have
> > to downgrade the effectiveness of a language like Scala
> > when you're stuck with antiquated Emacs to code in it.
> > You might not like it that Scala doesn't have good
> > tooling, but you have to deal with the reality of the
> > power of that our modern tools give us by letting us
> focus
> > on the domain and not the plumbing.
>
> That may have been the original poster's point, but I
> think the poster is misguided. The ability to push
> buttons on a magic tool isn't much of a skill to be proud
> of. The ability to "not make a mess of it" (as Dijkstra
> says) is far more important.

And as James Watson pointed out, you seem to be confusing tooling with code generation. When I'm talking about tooling, I'm mainly talking about IDE functionality - refactoring, inspection, navigation, completion....

David Pollak (inventor of the Lift framework) had an interesting post a few days ago http://goodstuff.im/.

He makes many interesting points - especially considering that he's a big Scala supporter and is able to step back a a bit from fanboyism and look at Scala from your average Java developer's point of view.

It's interesting that the main crux of your arguments seems to be of developers not understanding the code. The anti-Scala argument (I'm not one of them) is that by getting rid of the verbosity and introducing obscure Scala constructs, that you'll decrease understanding of the codebase. I would argue that understanding code is one of the least of developer's problems, but I'll leave that for another day. So you've increased misunderstanding of the codebase via the language instead of a tool "vomitting" out code.

Dick Ford

Posts: 149
Nickname: roybatty
Registered: Sep, 2003

Re: Tools Posted: Sep 4, 2011 9:25 PM
Reply to this message Reply
> I feel like I've been all over the place, so I want to
> boil down my arguments:
>
> Axiom 1
>
> The biggest source of trouble in software is code that
> programmers don't understand.
>
> There are several places to get such code, and it's useful
> to identify those sources and root them out as much as
> possible.
>

I disagree. I think the the biggest trouble in software is domain knowledge or requirements. I thought that was a pretty much a given. With a debugger it's not hard to figure out code...or even printfs if need be.


> Axiom 2
>
> Any code in your project is your responsibility,
> regardless of where you got it.
>

I'll agree there. Ownership of code and bitterness when constructive code review happens does nobody no good.

> Axiom 3
>
> Anything that convinces programmers that they understand
> code when, in fact, they do not, is a one-way ticket to
> the inner circle of Hell.
>

I'm sure I'm misunderstanding you, but in general we never understand any significant piece of code. We just don't. There's too many permutations of any significant piece of code to do that.


> My complaint about auto-generated code comes from applying
> these principles. The fact that you know which version of
> a tool and which settings you used in generating code does
> not, in fact, mean that you understand the code in
> question. And, odds are, you will one day need to.
>

Do you have a problem with ORMs then? Typically they autogenerate code.



> As an example; some projects I work on use Protocol
> Buffers. Protocol Buffers generates pretty solid code and
> has great documentation. Until you want to do something
> not mentioned in the documentation -- say serialize
> various NaN values or denormalized floats in one language
> and deserialize them in another. If you try to see if
> this will work by stepping through the auto-generated
> code, you can kiss your sanity goodbye.
>
> I have to assume that the hairball of code that Protocol
> Buffers spits out is a hairball because it's easy to
> generate; but I can't call it anything other than a
> massive hairball (please note that the API that the
> generated code exposes in not a hairball; I'm referring to
> the internals of the generated code). The structure of
> code is far from obvious, there is no way to determine
> invariants, and the only comments in the files are -- not
> surprisingly -- statements that you should not edit the
> files if you ever plan on rerunning the Protocol Buffer
> compiler.
>
> Alternatively, I know that I'm not the only programmer
> who's ever had auto-generated code throw a null pointer
> exception. And the only way to determine whether there is
> an error in the generated code or in my code calling it is
> to find out what details are being swept under the rug by
> the generated code (i.e., to understand the generated
> code). And, again, generated code is easy to generate,
> not necessarily easy to read.
>

We fought over whether to use the Entity Framework. We actually do Java and .NET development. The argument against the Entity Framework was that there was too much magic underneath. That's understandable. But now that EF 4.1 has come out we can trust the code generation. In java world, I'm assuming the same is the same in NHibernate world.


> On a different note, I'm fascinated by several tools that
> don't generate code; static analysis and modeling tools
> are prime examples. But most modeling tools seem to think
> of the problem at a level that's not useful for
> implementing a solution. They give people unwarranted
> optimism, and when things fall apart you have nothing to
> guide you out of the swamp you've been dumped in.

I'm a big, big, proponent of tool thats help BAs and developers narrow the conceptual and implementation gap, but I hear your point. EA sucks.

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Re: Tools Posted: Sep 6, 2011 4:06 PM
Reply to this message Reply
> I think there might be a misunderstanding of what is meant
> by tooling in Java. While this could mean tools that
> generate code, this is not the kind of tooling that comes
> to my mind. The kind of tooling that comes to my mind is
> things like "show references" and "open declaration".

This is a more subtle issue. While I like and use "show references," "open declaration," Intellisense and similar tools, I've started wondering if even these tools lull developers into thinking that they understand more code than they truly do. I know that I'm in the minority on that uneasiness. I prefer a project that is small enough I wouldn't *need* Intellisense for, but I'll use Intellisense anyway.

> Other tools like findbugs, checkstyle, JDepend, are really
> useful but have nothing to do with code generation.

Agreed. I believe that these tools are, in fact, useful in understanding the code in question.

> On a side note, while I tend to hate tools that generate
> java source code, it's good to keep in mind that compilers
> are tools that generate code.

That is true. I think there is an argument about how complex the generated code is. I've never heard of bugs in an assembler, but compiler optimizations will cause trouble once in a while.

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Re: Tools Posted: Sep 6, 2011 4:26 PM
Reply to this message Reply
> > The biggest source of trouble in software is code that
> > programmers don't understand. ...
>
> I disagree. I think the the biggest trouble in software
> is domain knowledge or requirements. I thought that was a
> pretty much a given.

How do you know if you've met your requirements if you don't understand the underlying code?

> I'm sure I'm misunderstanding you, but in general we never
> understand any significant piece of code. We just don't.
> There's too many permutations of any significant piece of
> code to do that.

It's entirely possible that we're talking about different levels of understanding. For one, I don't expect programmers to know the internals of their console output libraries when they write "Hello world." *But* I have seen trouble using C++ output streams, printf() and write() on the same output file (the internal buffering is hard to keep synchronized correctly).

I do believe it's possible to write significant code that is correct even with the number of code paths in question. Many standard practices come close to reaching that ideal, but none that I'm aware of actually manage to close the deal. For instance, unit testing can help just about any programmer improve their code quality, but a suite of passing tests isn't proof of software correctness ("program testing can be used to show the presence of bugs, but never to show their absence").

Identifying and avoiding common sources of complexity and defining and enforcing invariants seem to be not-so-standard practices that get programmers closer to correct code. I'm not sure how to get to the finish line, but I'm certain it isn't through the rampant use of autogenerated code.

> Do you have a problem with ORMs then? Typically they
> autogenerate code.

I haven't taken a vow of poverty w/r/t autogenerated code. I have come to believe that they aren't all unicorns and lollipops, though.

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Java 7: Too Little, Too Late? Posted: Sep 7, 2011 7:49 AM
Reply to this message Reply
> David Pollak (inventor of the Lift framework) had an
> interesting post a few days ago http://goodstuff.im/.

Example from the article:

flatMap [B, That] (f: (A) ⇒ Traversable[B])(implicit bf: CanBuildFrom[List[A], B, That]) : That


Is this type noise the major reason why Scala ( and other statically typed FPLs ) is perceived difficult?

Flat View: This topic has 64 replies on 5 pages [ « | 1  2  3  4  5 | » ]
Topic: The Principles of Good Programming Previous Topic   Next Topic Topic: First Impression: Scala Goodies


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us