Registered: Sep, 2005
Re: word games
Posted: Aug 2, 2007 6:12 PM
> James Watson wrote
> > > That's right - a change to the specification requires
> > > change to the tests which may require a change to the
> > > formerly correct program before it could be
> > > correct once more, what's surprising about that?
> > I didn't say anything about a change to the
> If the spec didn't change then what do you mean "a correct
> program could be become incorrect without changing
> anything about it"?
I you define correctness as passing all tests, creating a new test that exposed an existing issue (with regard to specifications) would make the code incorrect.
> > The above is only true in the case where the tests are
> > e the specification. That is not how specifications
> > normally documented.
> How are implementations normally checked against
> specs? (I think acceptance tests are far more common than
> formal proof.)
> > > Verification checks conformance to spec -
> > >
> > > Validation checks what was really wanted - "meets all
> > > requirements".
> > And I'm the one playing word games. Unbelievable.
> Perhaps you aren't familiar with that simple distinction,
> perhaps you haven't come across those technical terms but
> they are well established terms (IEEE 729 "Glossary of
> Software Engineering Terminology" 1982) and commonly used.
> The authors of a 2002 IBM SYSTEMS JOURNAL article wanted
> to be "accessible to the students of software engineering
> at large" so they included some definitions for "relevant
> Thus, verification is the process of proving or
> demonstrating that the program correctly satisfies the
> specifications. Notice that we use the term verification
> in the sense of “functional correctness”...
> Validation is the process of evaluating software, at
> the end of the development process, to ensure compliance
> with requirements.
How is this incompatible with what I am arguing?
> > You are trying to side step the issue. It's very
> > possible, common even, to have a program that passes
> > tests that doesn't meet all requirements. Do you
> > such a program to be correct?
> There isn't an issue for me to side step.
> It's no more than a matter of definition, so what matters
> is that a definition is provided and agreed.
> Just Google correct or correctness and
> you'll find a wikipedia article that might be helpful -
I don't see where this is contradicting me in any way. Perhaps you should try to understand my point before arguing about it.
> When we use tests to check the program conforms to spec,
> and the program passes all tests, and we're using an
> ordinary definition of program correctness, then the
> program is obviously 'correct'.
If all tests pass, it doesn't mean that the spec is completely satisfied unless the spec is defined by the tests.
> If we were using the definition that you have come up with
> then the program is obviously not 'correct'.
In any other case other than that of which the spec is completely defined by tests all testing can do is prove that a program is incorrect for non-trivial programs. It cannot prove the program correct unless it tests all possible states of the application.
In practical terms, it's possible to test a program and have a good confidence in it's correctness. But you can never prove a program to be correct in the terms that Achilleas Margaritis describes as "proving correctness means to mathematically prove that a piece of software does what it should do." And this the commonly understood defintion between myself and Achilleas when you decided to start an argument about what correct means.