Java Community News - Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Java Community News
Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

38 replies on 3 pages. Most recent reply: Jun 14, 2008 10:09 AM by John Zabroski

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 38 replies on 3 pages [ « | 1 2 3 | » ]

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 8:11 AM

Reply

Advertisement

> > > <p>So why should I be so happy about the future that
> > > hardware vendors promise?
> > Whether you are happy or not there is no alternative in
> > sight if you want more power than they can provide in a
> > single core.
> >
>
> I read him as being sad. Reading between the lines, his
> point was that Intel has a near (or may be a bit more than
> that) monopoly on architecture. Even if some hardware
> geeks came up with a new architecture (instruction set)
> that gets more work done with the available transistor
> budget and clock as a single cpu, it would never get a
> foothold due to the Wintel situation. This is much like
> what happened to mainframes in the late 1960s.
>
> The RISC thing has come and gone. We're left with an
> architecture that has reached its level of incompetence.

While I feel that there's likely a good bit more speed to squeeze out of single-threading, there's a pretty hard physical limit to how small you can make a processor and hence how fast it can be. Without some pretty major breakthroughs in physics, we are heading for a brick wall. It's just a matter of when we will hit it. It's not clear that quantum computers will become viable but even if they do, they can't (contrary to popular belief) provide benefits for a large number of problems.

Honestly, the idea that we can avoid ever going more parallel seems very dubious.

> At risk of upsetting some readers, there is one problem
> not mentioned in the interview that does benefit from
> parallel execution in multi-[thread|core|processor]
> machines. I'll note that DB2's latest (off mainframe)
> version is now fully threaded. Set processing is
> inherently faster than sequential processing. Set
> thinking is not common, nor relevant in many arenas. But
> where it is, it loves such machines.

What I don't get about the multi-core hullabaloo is that multi-processor machines are not new. I've been developing code for multi-processor systems for a decade. The only difference is that it's becoming cheaper. And in a lot of cases, the benefits of multi-core can be realized by simply splitting single-threaded processes across cores. There's still time to make multi-threading easier and for developers to learn how to use the tools that already exist.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 11:17 AM

Reply

> > "Extreme Programming is actually Experimental
> Programming,
> > where you 'design' a system by repeatedly coding stuff
> and
> > going back to the customer and saying "Is this what you
> > want?". "
> >
> > that's anarchy :)
>
> Well it's a lot better than developing something blindly
> and then telling the customer "this is what you want" when
> in fact, it hardly ever is.

There has to be a sweet spot in the spectrum between strict Waterfall and coding the whim of the day. My theory is that the sweet spot is a function of organization size and the rapidness of change to the organization's core business. Also worth considering is the level in the "abstraction stack" being coded for. Closer to the hardware and I think it gets easier coming up with hard requirements and designs. As you move up through abstractions, usually the requirements are malleable and cause rapid (often violent) changes to design.

With Dr. Knuth's considerable experience writing software, I'd like to think he believes in having an iterative approach but is turned off by the evangelization (by some) of Extreme Programming.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 11:40 AM

Reply

> Honestly, the idea that we can avoid ever going more
> parallel seems very dubious.

It's not avoidable, nor can I see a reason why anyone would want to avoid it, but judicious use of parallelization is something still being worked out. I think Dr. Knuth was just saying not to stop looking into optimization in more traditional ways.

> What I don't get about the multi-core hullabaloo is that
> multi-processor machines are not new. I've been
> developing code for multi-processor systems for a decade.
> The only difference is that it's becoming cheaper. And
> d in a lot of cases, the benefits of multi-core can be
> realized by simply splitting single-threaded processes
> across cores. There's still time to make multi-threading
> easier and for developers to learn how to use the tools
> that already exist.

There seem to be groups on polar opposite sides. One group sees parallelization as a savior for stalling improvements to code execution. Another group hasn't had to think about hardware, due to 4GL's and layers of abstractions, for a decade or so and is disturbed to think about it again. The latter group I think hopes for a new generation of compilers to take care of parallelization for them.

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 12:17 PM

Reply

> With Dr. Knuth's considerable experience writing software,
> I'd like to think he believes in having an iterative
> approach but is turned off by the evangelization (by some)
> of Extreme Programming.

The short iteration cycles advocated by Extreme Programming may not be well suited to the type of work for which Knuth is known. It certainly doesn't suit some of my work.

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 1:05 PM

Reply

> > Honestly, the idea that we can avoid ever going more
> > parallel seems very dubious.
>
> It's not avoidable, nor can I see a reason why anyone
> would want to avoid it, but judicious use of
> parallelization is something still being worked out. I
> think Dr. Knuth was just saying not to stop looking into
> optimization in more traditional ways.
>
> > What I don't get about the multi-core hullabaloo is
> that
> > multi-processor machines are not new. I've been
> > developing code for multi-processor systems for a
> decade.
> > The only difference is that it's becoming cheaper. And
> > d in a lot of cases, the benefits of multi-core can be
> > realized by simply splitting single-threaded processes
> > across cores. There's still time to make
> multi-threading
> > easier and for developers to learn how to use the tools
> > that already exist.
>
> There seem to be groups on polar opposite sides. One group
> sees parallelization as a savior for stalling improvements
> to code execution. Another group hasn't had to think about
> hardware, due to 4GL's and layers of abstractions, for a
> decade or so and is disturbed to think about it again. The
> latter group I think hopes for a new generation of
> compilers to take care of parallelization for them.

It's worth considering the sort of problems that Thinking Machines (of nearly 20 years ago) and Blue Gene of today are good at: weather simulation, thermonucular explosions, matrix arithematic (most statistical calcs) and the odd stock market manipulation; oh, and chess. Finding problems that clearly benefit from large scale parallelization wasn't and isn't simple. As the Intel chips gets closer to "large scale", just because nothing else can be done, identifiable performance increase will get incrementally smaller.

These machines are still von Neumann, just lots-in-the-box.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 1:35 PM

Reply

> It's worth considering the sort of problems that Thinking
> Machines (of nearly 20 years ago) and Blue Gene of today
> are good at: weather simulation, thermonucular
> explosions, matrix arithematic (most statistical calcs)
> and the odd stock market manipulation; oh, and chess.
> Finding problems that clearly benefit from large scale
> e parallelization wasn't and isn't simple. As the Intel
> chips gets closer to "large scale", just because nothing
> else can be done, identifiable performance increase will
> get incrementally smaller.
>
> These machines are still von Neumann, just lots-in-the-box.

Do you think areas like holographic imaging and graphics (not areas I know anything about) in general would be benefitted by parallelization?

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 1:37 PM

Reply

I'm a big fan of the literate-programming concept. That's why I favor languages like Ruby that excel at creating DSLs.

I'm also a big fan of refactoring. My professional experience has been nothing so much as watching an elegant design turn into a mass of spaghetti over time. Once I discovered refactoring (Fowler be praised), I found that it was time to *evolve elegance*, rather than being constrained to a prescient design that took into account all of the technological unknowns and unexpected requirements beforehand.

And I'm a huge fan of agile technologies, precisely because software development is mostly a *design and discovery* process, rather than a manufacturing or mechanical assembly process. (There are exceptions. There are people who have been doing the same thing for 5 years, who are somehow capable of doing it for several years more. I have no idea how they do that, but *they* are capable of doing a design that will has a good chance of remaining intact for the duration of the project. I know I don't.)

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 1:39 PM

Reply

> Do you think areas like holographic imaging and graphics
> (not areas I know anything about) in general would be
> benefitted by parallelization?

Many graphics algorithms are easily parallelised, often by merely subdividing the image into as many tiles as you have processors. Modern GPUs have many processor units which work in parallel. Games are big users of this technology.

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 1:42 PM

Reply

> mechanical assembly process. (There are exceptions. There
> are people who have been doing the same thing for 5 years,
> who are somehow capable of doing it for several years
> more. I have no idea how they do that, but *they* are
I have been doing algorithms related to vehicle routing for 25 years and hope to continue for a while yet!

George Sakkis

Posts: 14
Nickname: gsakkis
Registered: Jun, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 2, 2008 12:29 AM

Reply

Favorite quote: "With the caveat that there’s no reason anybody should care about the opinions of a computer scientist/mathematician like me regarding software development, ..."

Respect for his awareness that (theoretical) computer science and software development are very different skills, and for not pretending to be an authority on both. Extreme programming or unit tests are not really relevant to the sort of work he's been doing, and his opinion on such topics should not carry more weight than his opinion on, say, global warming.

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 2, 2008 12:44 AM

Reply

I seem to remember plenty of torture tests for TeX and components thereof.

Vincent O'Sullivan

Posts: 724
Nickname: vincent
Registered: Nov, 2002

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 2, 2008 1:04 AM

Reply

> Well it's a lot better than developing something blindly
> and then telling the customer "this is what you want" when
> in fact, it hardly ever is.

Indeed. But since nobody has ever proposed that as a development methodology it's just a strawman argument and not a valid basis for justifying for XP.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 2, 2008 6:05 AM

Reply

> > Well it's a lot better than developing something
> blindly
> > and then telling the customer "this is what you want"
> when
> > in fact, it hardly ever is.
>
> Indeed. But since nobody has ever proposed that as a
> development methodology it's just a strawman argument and
> not a valid basis for justifying for XP.

I'm not justifying XP. In fact I'm not sure that XP is really such a great thing. All I was observing is that writing code and repeatedly asking the customer if that's what they want is a much better strategy that what I normally see: Design an write a system based on some vague notions that slightly resemble requirements and don't show the customer until the very end of the available time frame. Then when the customer says "this isn't what we wanted" tell them there is no more time.

I'm not sure iterative development is "chaos" as it was described but even if it is, in my experience, the sooner the customer sees what you are doing the more likely you are to succeed. And my experience also tells me that big-up-front-design is largely a waste of time.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 2, 2008 7:55 AM

Reply

>
> I'm not sure iterative development is "chaos" as it was
> described but even if it is, in my experience, the sooner
> the customer sees what you are doing the more likely you
> are to succeed. And my experience also tells me that
> big-up-front-design is largely a waste of time.
>
Have to agree. On the other hand, I have to believe that succssful XP'ers are doing more long term design than they're admitting to. I like the idea that you don't *code* the cool architecture you have in mind. (If I had a nickel for every line of code I've poured thought into, never to see them executed, for which the machines no longer exist!) But by the same token, it's not as "knee jerk" a process as it is sometimes portrayed. After all, the way to get add(2,2) to succeed is to return 4. But to make it succeed for an arbitrary N,M, you have to work at a deeper level of abstraction!

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 2, 2008 8:21 AM

Reply

> >
> > I'm not sure iterative development is "chaos" as it was
> > described but even if it is, in my experience, the
> sooner
> > the customer sees what you are doing the more likely
> you
> > are to succeed. And my experience also tells me that
> > big-up-front-design is largely a waste of time.
> >
> Have to agree. On the other hand, I have to believe that
> succssful XP'ers are doing more long term design than
> they're admitting to. I like the idea that you don't
> *code* the cool architecture you have in mind. (If I had a
> nickel for every line of code I've poured thought into,
> never to see them executed, for which the machines no
> longer exist!) But by the same token, it's not as "knee
> jerk" a process as it is sometimes portrayed. After all,
> the way to get add(2,2) to succeed is to return 4. But to
> make it succeed for an arbitrary N,M, you have to work at
> a deeper level of abstraction!

I think it's good to have a strategy and a plan. How much that strategy encompasses and how you come to that conclusion is another topic. I think YAGNI can be taken to mean that you must solve each problem in isolation and never look at the big picture. But I definitely think you should never code something if you aren't sure it's going to be used in the very near future.

What I get really annoyed with is putting a lot of time into a really detailed design without any attempt to test any assumptions. A fairly high-level design of how chunks of an application will work with each other can be done very early (and should be IMO) but when you start getting down to designing every class and the methods in each one and detailed sequence diagrams of low-level interactions inside a logical abstraction, almost none of that will be correct at the end. Considering how long it takes to produce that kind of thing, it's extremely inefficient.

Again and again I see a lot of documentation about what the original design for a system was. But I hardly ever find documentation on what was actually implemented. The plans are really only useful up to the point you start implementing them. Knowing what was done is useful for the life of the system. Given this disparity, it seems crazy that, at least in my experience, the amount of effort applied to documenting the plan is so much more than the effort put into documenting the final system.

Flat View: This topic has 38 replies on 3 pages [ « | 1 2 3 | » ]

Previous Topic

Next Topic

Sponsored Links

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use