Java Community News - Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Java Community News
Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

38 replies on 3 pages. Most recent reply: Jun 14, 2008 10:09 AM by John Zabroski

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 38 replies on 3 pages [ « | 1 2 3 | » ]

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 3:32 AM

Reply

Advertisement

If you ignore the GPU, two cores seems to be more than adequate for most uses now and for some time to come. However if/when AMD/Intel start merging the CPU and GPU what will the total core count be and will all cores be the same or will capabilities differ (as already happens with the Cell systems). Taking best advantage of these systems could be even more challenging than simple symmetrical multicores (noting that a number of existing multicores already have NUMA).

Jiri Goddard

Posts: 54
Nickname: goddard
Registered: May, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 5:34 AM

Reply

look at the 8 cell ps3 thing - no one really knows how to use it all. it's academic.

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 5:48 AM

Reply

> it's academic.

No it's the real world; more complex than the models that academics prefer to study.

Vincent O'Sullivan

Posts: 724
Nickname: vincent
Registered: Nov, 2002

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 7:10 AM

Reply

> I would be interested to hear him flush out a little more
> about Extreme Programming. Does he believe iterative
> development is bad? Does he believe tight feedback loops
> are bad?

I don't think he's against iteration per se. I think that the problem is much of Extreme Programming is actually Experimental Programming, where you 'design' a system by repeatedly coding stuff and going back to the customer and saying "Is this what you want?". Tight feedback loops can be a symptom of (or an excuse for) not doing enough homework up front to have confidence in what it is that you're trying to develop.

Jiri Goddard

Posts: 54
Nickname: goddard
Registered: May, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 7:41 AM

Reply

it's on the market, so it's real. but except ps3 developers at sony and few universities no one is really promoting programming 8-cells. today games are using a fraction of its potential and the universities are trying to do a research what language and what methods could be useful for such architecture. from this point of view, it's academic even it's on the market already.

Jiri Goddard

Posts: 54
Nickname: goddard
Registered: May, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 7:44 AM

Reply

"Extreme Programming is actually Experimental Programming, where you 'design' a system by repeatedly coding stuff and going back to the customer and saying "Is this what you want?". "

that's anarchy :)

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 7:52 AM

Reply

> "Extreme Programming is actually Experimental Programming,
> where you 'design' a system by repeatedly coding stuff and
> going back to the customer and saying "Is this what you
> want?". "
>
> that's anarchy :)

Well it's a lot better than developing something blindly and then telling the customer "this is what you want" when in fact, it hardly ever is.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 8:11 AM

Reply

> > > <p>So why should I be so happy about the future that
> > > hardware vendors promise?
> > Whether you are happy or not there is no alternative in
> > sight if you want more power than they can provide in a
> > single core.
> >
>
> I read him as being sad. Reading between the lines, his
> point was that Intel has a near (or may be a bit more than
> that) monopoly on architecture. Even if some hardware
> geeks came up with a new architecture (instruction set)
> that gets more work done with the available transistor
> budget and clock as a single cpu, it would never get a
> foothold due to the Wintel situation. This is much like
> what happened to mainframes in the late 1960s.
>
> The RISC thing has come and gone. We're left with an
> architecture that has reached its level of incompetence.

While I feel that there's likely a good bit more speed to squeeze out of single-threading, there's a pretty hard physical limit to how small you can make a processor and hence how fast it can be. Without some pretty major breakthroughs in physics, we are heading for a brick wall. It's just a matter of when we will hit it. It's not clear that quantum computers will become viable but even if they do, they can't (contrary to popular belief) provide benefits for a large number of problems.

Honestly, the idea that we can avoid ever going more parallel seems very dubious.

> At risk of upsetting some readers, there is one problem
> not mentioned in the interview that does benefit from
> parallel execution in multi-[thread|core|processor]
> machines. I'll note that DB2's latest (off mainframe)
> version is now fully threaded. Set processing is
> inherently faster than sequential processing. Set
> thinking is not common, nor relevant in many arenas. But
> where it is, it loves such machines.

What I don't get about the multi-core hullabaloo is that multi-processor machines are not new. I've been developing code for multi-processor systems for a decade. The only difference is that it's becoming cheaper. And in a lot of cases, the benefits of multi-core can be realized by simply splitting single-threaded processes across cores. There's still time to make multi-threading easier and for developers to learn how to use the tools that already exist.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 11:17 AM

Reply

> > "Extreme Programming is actually Experimental
> Programming,
> > where you 'design' a system by repeatedly coding stuff
> and
> > going back to the customer and saying "Is this what you
> > want?". "
> >
> > that's anarchy :)
>
> Well it's a lot better than developing something blindly
> and then telling the customer "this is what you want" when
> in fact, it hardly ever is.

There has to be a sweet spot in the spectrum between strict Waterfall and coding the whim of the day. My theory is that the sweet spot is a function of organization size and the rapidness of change to the organization's core business. Also worth considering is the level in the "abstraction stack" being coded for. Closer to the hardware and I think it gets easier coming up with hard requirements and designs. As you move up through abstractions, usually the requirements are malleable and cause rapid (often violent) changes to design.

With Dr. Knuth's considerable experience writing software, I'd like to think he believes in having an iterative approach but is turned off by the evangelization (by some) of Extreme Programming.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 11:40 AM

Reply

> Honestly, the idea that we can avoid ever going more
> parallel seems very dubious.

It's not avoidable, nor can I see a reason why anyone would want to avoid it, but judicious use of parallelization is something still being worked out. I think Dr. Knuth was just saying not to stop looking into optimization in more traditional ways.

> What I don't get about the multi-core hullabaloo is that
> multi-processor machines are not new. I've been
> developing code for multi-processor systems for a decade.
> The only difference is that it's becoming cheaper. And
> d in a lot of cases, the benefits of multi-core can be
> realized by simply splitting single-threaded processes
> across cores. There's still time to make multi-threading
> easier and for developers to learn how to use the tools
> that already exist.

There seem to be groups on polar opposite sides. One group sees parallelization as a savior for stalling improvements to code execution. Another group hasn't had to think about hardware, due to 4GL's and layers of abstractions, for a decade or so and is disturbed to think about it again. The latter group I think hopes for a new generation of compilers to take care of parallelization for them.

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 12:17 PM

Reply

> With Dr. Knuth's considerable experience writing software,
> I'd like to think he believes in having an iterative
> approach but is turned off by the evangelization (by some)
> of Extreme Programming.

The short iteration cycles advocated by Extreme Programming may not be well suited to the type of work for which Knuth is known. It certainly doesn't suit some of my work.

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 1:05 PM

Reply

> > Honestly, the idea that we can avoid ever going more
> > parallel seems very dubious.
>
> It's not avoidable, nor can I see a reason why anyone
> would want to avoid it, but judicious use of
> parallelization is something still being worked out. I
> think Dr. Knuth was just saying not to stop looking into
> optimization in more traditional ways.
>
> > What I don't get about the multi-core hullabaloo is
> that
> > multi-processor machines are not new. I've been
> > developing code for multi-processor systems for a
> decade.
> > The only difference is that it's becoming cheaper. And
> > d in a lot of cases, the benefits of multi-core can be
> > realized by simply splitting single-threaded processes
> > across cores. There's still time to make
> multi-threading
> > easier and for developers to learn how to use the tools
> > that already exist.
>
> There seem to be groups on polar opposite sides. One group
> sees parallelization as a savior for stalling improvements
> to code execution. Another group hasn't had to think about
> hardware, due to 4GL's and layers of abstractions, for a
> decade or so and is disturbed to think about it again. The
> latter group I think hopes for a new generation of
> compilers to take care of parallelization for them.

It's worth considering the sort of problems that Thinking Machines (of nearly 20 years ago) and Blue Gene of today are good at: weather simulation, thermonucular explosions, matrix arithematic (most statistical calcs) and the odd stock market manipulation; oh, and chess. Finding problems that clearly benefit from large scale parallelization wasn't and isn't simple. As the Intel chips gets closer to "large scale", just because nothing else can be done, identifiable performance increase will get incrementally smaller.

These machines are still von Neumann, just lots-in-the-box.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 1:35 PM

Reply

> It's worth considering the sort of problems that Thinking
> Machines (of nearly 20 years ago) and Blue Gene of today
> are good at: weather simulation, thermonucular
> explosions, matrix arithematic (most statistical calcs)
> and the odd stock market manipulation; oh, and chess.
> Finding problems that clearly benefit from large scale
> e parallelization wasn't and isn't simple. As the Intel
> chips gets closer to "large scale", just because nothing
> else can be done, identifiable performance increase will
> get incrementally smaller.
>
> These machines are still von Neumann, just lots-in-the-box.

Do you think areas like holographic imaging and graphics (not areas I know anything about) in general would be benefitted by parallelization?

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 1:37 PM

Reply

I'm a big fan of the literate-programming concept. That's why I favor languages like Ruby that excel at creating DSLs.

I'm also a big fan of refactoring. My professional experience has been nothing so much as watching an elegant design turn into a mass of spaghetti over time. Once I discovered refactoring (Fowler be praised), I found that it was time to *evolve elegance*, rather than being constrained to a prescient design that took into account all of the technological unknowns and unexpected requirements beforehand.

And I'm a huge fan of agile technologies, precisely because software development is mostly a *design and discovery* process, rather than a manufacturing or mechanical assembly process. (There are exceptions. There are people who have been doing the same thing for 5 years, who are somehow capable of doing it for several years more. I have no idea how they do that, but *they* are capable of doing a design that will has a good chance of remaining intact for the duration of the project. I know I don't.)

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 1, 2008 1:39 PM

Reply

> Do you think areas like holographic imaging and graphics
> (not areas I know anything about) in general would be
> benefitted by parallelization?

Many graphics algorithms are easily parallelised, often by merely subdividing the image into as many tiles as you have processors. Modern GPUs have many processor units which work in parallel. Games are big users of this technology.

Flat View: This topic has 38 replies on 3 pages [ « | 1 2 3 | » ]

Previous Topic

Next Topic

Sponsored Links

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use