Weblogs Forum - Version Control is Undo

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
Version Control is Undo

83 replies on 6 pages. Most recent reply: Jan 27, 2009 4:17 AM by Dhruva Krishnamurthy

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 83 replies on 6 pages [ « | 1 2 3 4 5 6 | » ]

Fred Garvin

Posts: 52
Nickname: fredgarvin
Registered: Jan, 2008

Re: Version Control is Undo . . . or more

Posted: Dec 9, 2008 11:46 PM

@James

It appears you are dissatisfied not so much with the idea, but with the quality of an imagined implementation of the idea. So lets assume for a moment we have a sound working system, however far fetched it may seem to you, and maybe you'll find the concept a little less toady.

* no useful diff / merge (XML-based source is big offender)
The actual storage/format of the source doesn't matter. The idea is that the source control manager works directly on the parse element model; the raw storage has no impact, be it xml or binary. So quite the opposite is true -- the diff/merge tool becomes much more capable. It can do more than diff in terms of lines of text. It exposes changes in terms of language elements e.g., data member _foo changed to _bar; the expression in method baz() has conflicts. And you can edit/view changes and conflicts in your favorite code format.

* source that is difficult or impossible to read. If the source becomes corrupted, it costs a lot of time/money to repair the source if it's not human readable.
I understand, but lets assume a sound system.

* ultimate vendor lock-in (cannot see our own process logic or extract it from text)
I hear you, but lets assume a standard parse element model. This is the missing link I mentioned earlier.

* silent corruption of code by tool
Again, lets assume no corruption.

* lack of persisted format between tool versions (see previous)
Again, lets assume a standard parse element model.

* infeasible to write custom analysis, code generation, and refactoring tools
I believe the opposite is true. With a standard parse element model as the SOR, you can write tools in a much more straight forward manner. In fact thats a driving force behind the concept.

* inability to conform to change management (manual production deployments)
Are you referring to patches? Nothing much changes here. Object code deployment is not affected. Maybe you have something else in mind?

* integrated source control broken / unusable
Again, for now, lets assume a sound system.

* gui-based development much slower than text based (modal dialog hell)
Wha?? Writing code is still very much a textual interface. Nothing much changes there except, perhaps, the larger set of tools you have at your disposal.

Can I ask what IDE you currently favor? I ask only because, in my opinion, very few IDEs are worth using. I'll admit my strong bias toward IDEA. After using both Eclipse and IDEA on the same codebase, I have an even bigger appreciation for IDEA (something I thought not possible). I suppose what I'm saying is that if your IDE isn't serving you well, it's unlikely you'll be convinced that this nutty idea of using a parse element model and integrating source control has any merit.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 5:25 AM

I don't think the problems you describe are related at all to the argument I presented. Let me examine your examples:

> * difficult to impossible to incorporate into source
> control with other resources

You don't give any details as to the nature of the problem. If you tried to incorporate apples with oranges, it's only natural that the source control failed. Why do you blame the other format and not text?

> * no useful diff / merge (XML-based source is big
> offender)

You tried to apply a text-based diff/merge onto XML? that's why you got problems. A diff/merge tool based on the specific format the source code exists in would be much better and much easier to write.

> * source that is difficult or impossible to read. If the
> source becomes corrupted, it costs a lot of time/money to
> repair the source if it's not human readable.

It all depends on the corruption. You assume that the corrupted files have minor damage, and that the damaged parts can be inferred from the surrounding context. But this is hardly the case if the damage goes beyond 3 lines of code.

> * ultimate vendor lock-in (cannot see our own process
> logic or extract it from text)

If the data format is public domain, I don't see how there is a vendor lock-in.

> * silent corruption of code by tool

This means the tool is not correct. But it has nothing to do with the format of the source code being text and having an advantage over other formats. For example, Visual Studio has a bug which regularly corrupts my source code, which is in text format.

> * lack of persisted format between tool versions (see
> previous)

It can happen with text as well, if the new tool parses the text differently from the previous one.

> * infeasible to write custom analysis, code generation,
> and refactoring tools

Not infeasible at all. In fact, much easier, because the parsing step may be omitted.

> * inability to conform to change management (manual
> production deployments)

Not at all. Deployment methods are irrelevant. You need tools for the text format, don't you? ultimately, text is a collection of bytes which requires its own set of tools to work with.

> * integrated source control broken / unusable

Don't blame the format for that. Microsoft's Source Safe has big problems, although it is based on text.

> * gui-based development much slower than text based (modal
> dialog hell)

If you have a good IDE, development of GUI applications is much faster than in text.

>
> I'm not saying that a non-text based format cannot be
> written without these issues but unless all the benefits I
> get from text-based source are available in a new
> approach, I don't believe it is wise to use that approach.
> The downside risks are too high.

A new non-text based format would have all the benefits of the text format as well as many more benefits.

Don't blame the tools for their problems.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 5:44 AM

> > > What's needed in this grand scheme is a standard
> parse
> > > element definition and format so that tools are
> > > compatible. For instance, Java could specify a parse
> > > element definition and could directly compile this
> > > format... and pigs will fly.
> >
> > Actually, what is required is a universal data format.
> Not
> > for source code only, but for anything. A universal
> data
> > format could solve many issues in today's computing.
>
> Universal formats already exist. Text is one of them.
> The problem is that a universal format must be
> e constrained in order to be useful for defining program
> logic.

Text is not a universal data format. Text is a format for ...text. In order to represent something in text, it must have a textual description.

Saying that text is a universal data format is saying like a binary file is a universal data format. Just because something can be stored in a text file, it does not make text a universal data format.

A universal data format would be something that would allow a programmer to easily tell what's inside the data. Text does not have such a property (emphasis on 'easily').

>
> > One step in the right direction is XML.
>
> I disagree completely. XML is one of the worst formats
> for storing code.

What arguments do you have against XML for storing code?

> The only benefit it provides is that
> there are lots of libraries around for parsing it.

Which is because XML data are somewhat self-describing...at least, they have some sort of structure, and therefore they are easily parsed.

> The readability of XML-based code is very low, especially when
> XML gets embedded in the XML. And because equivalent XML
> can be formatted in many different ways, special diff
> tools to work with it.

Actually, writing a diff tool for code using XML is much easier than writing a diff tool for text. The reason is that XML is already structured, and the code's structure is reflected in the XML structure.

For example, no text-based diff tool can tell me today that two source files are equal when the only change is the re-ordering of declarations (i.e. the too files contain the same declarations, but in a different order). This is possible only with a structured format.

> What we need is better parsing libraries so that people
> don't feel the need to use XML for code. Parsing is
> really a pretty small part of the problem. The structure
> of the documents and the interpretation of them is a
> bigger problem.

No, parsing is a big problem.

For example, in order to make sense of a c++ program, for example, the parser has to be interleaved with the compiler: in order to understand what foo bar() means, information from the compiler must be fed to the parser. The c++ grammar is not context free, leading to all sorts of problems.

If the c++ code was stored in XML format, with a tag 'variable', then foo bar() would be unambiguous: it would be a variable declaration.

But this misses the point of my initial comment. I said that XML is a step in the right direction, not the solution. The advantage of XML is that it is a structured format, which is a quality that any universal data format should have. But it has many disadvantages (too verbose, too memory-hungry etc).

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 7:59 AM

> @James
>
> It appears you are dissatisfied not so much with the idea,
> but with the quality of an imagined implementation of the
> idea. So lets assume for a moment we have a sound working
> system, however far fetched it may seem to you, and maybe
> you'll find the concept a little less toady.

These are not imagined issues. I was asked what issues I have had. All of these are real things that I have struggled with in my career. I'm fed up with vendor lines about how their 'code-free' tools will improve productivity and reliability when the exact opposite is always the case.

What I am dissatisfied with is the tools that I have worked with that did not have human readable text as the canonical persistence format.

> * no useful diff / merge (XML-based source is big
> offender)
> The actual storage/format of the source doesn't matter.
> The idea is that the source control manager works directly
> on the parse element model; the raw storage has no impact,
> be it xml or binary. So quite the opposite is true -- the
> diff/merge tool becomes much more capable. It can do more
> than diff in terms of lines of text. It exposes changes in
> terms of language elements e.g., data member _foo changed
> to _bar; the expression in method baz() has conflicts. And
> you can edit/view changes and conflicts in your favorite
> code format.

I can see that. The problem I see is that there are lots of tools for doing diffs against text but as far as I know, any diff that works the way you describe here would have to have intimate knowledge of the structure of the model. If that's not the case, please enlighten me.

I would love to have that kind of diff. However, I don't want to be limited in what I can use. I want it all. I want that and the ability to do text based diffs if that's my desire. I see no reason why both cannot be supported.

> * source that is difficult or impossible to read. If
> the source becomes corrupted, it costs a lot of time/money
> to repair the source if it's not human readable.
> I understand, but lets assume a sound system.

That's the problem. I can't assume this because I know it will not always be the case. All software has bugs. Any argument based on an assumption that a given piece of software will never have bugs is going to ring hollow.

> * ultimate vendor lock-in (cannot see our own process
> logic or extract it from text)
> I hear you, but lets assume a standard parse element
> model. This is the missing link I mentioned earlier.
>
> * silent corruption of code by tool
> Again, lets assume no corruption.

Addressed above.

> * lack of persisted format between tool versions (see
> previous)
> Again, lets assume a standard parse element model.
>
> * infeasible to write custom analysis, code generation,
> and refactoring tools
> I believe the opposite is true. With a standard parse
> element model as the SOR, you can write tools in a much
> more straight forward manner. In fact thats a driving
> force behind the concept.
>
> * inability to conform to change management (manual
> production deployments)
> Are you referring to patches? Nothing much changes here.
> Object code deployment is not affected. Maybe you have
> something else in mind?

In IT enterprise IT, for example, change management processes often require software must be able to be delivered directly from QA to Production without human intervention. A lot of tools lack a modular way to persist code and/or binaries in a way that they can be delivered in such a way. This reliably results in process violations and ultimately production problems.

> * integrated source control broken / unusable
> Again, for now, lets assume a sound system.
>
> * gui-based development much slower than text based
> (modal dialog hell)
> Wha?? Writing code is still very much a textual interface.
> Nothing much changes there except, perhaps, the larger set
> of tools you have at your disposal.

So this is really the heart of my disagreement. If the tool is going to provide a textual format for the code and presumably be able to parse changes to that text back into the model, the work required to persist the model to and from text will have already been invested. Why then are you so adamant that this text can't be what is persisted?

I think all of the things that you are suggesting here make sense. What I don't understand is why you believe that a textual format is the limiting factor, especially if you are assuming textual formatting and parsing will be supported anyway? What difference does it make whether the text comes from a screen element or from a file?

> Can I ask what IDE you currently favor? I ask only
> because, in my opinion, very few IDEs are worth using.
> I'll admit my strong bias toward IDEA. After using both
> Eclipse and IDEA on the same codebase, I have an even
> bigger appreciation for IDEA (something I thought not
> possible). I suppose what I'm saying is that if your IDE
> isn't serving you well, it's unlikely you'll be convinced
> that this nutty idea of using a parse element model and
> integrating source control has any merit.

As I was saying in the paragraph above, it's not that I disagree with the ideas you are proposing. I think they are quite good. It's that you haven't shown me that you would have to jettison text-based files persistence in order to achieve these things.

Above you keep saying assume that the system is sound. Why should I have to? Just persist the model as text (something that should require almost no additional work) and give users a contingency. It's not important to me that I actually use this textual format, it's important that it's there in case I need it.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 8:02 AM

> A new non-text based format would have all the benefits of
> the text format as well as many more benefits.

The question I keep asking but have not gotten an answer to is: what benefits can be provided in a non-text format that cannot be provided in a textual format? One example would be a start.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 8:15 AM

> Text is not a universal data format. Text is a format for
> ...text. In order to represent something in text, it must
> have a textual description.
>
> Saying that text is a universal data format is saying like
> a binary file is a universal data format. Just because
> something can be stored in a text file, it does not make
> text a universal data format.
>
> A universal data format would be something that would
> allow a programmer to easily tell what's inside the data.
> Text does not have such a property (emphasis on
> 'easily').

I think you misunderstand what 'universal' means. It doesn't imply any that at all. Any universal format will have the same limitations as bytes and text. You will not be able to make use of it without assumptions about it's structure. This is the big lie about XML that I though was finally only believed by a minority. XML is useless as a persistence format without a schema and a description of what each element means.

> Actually, writing a diff tool for code using XML is much
> easier than writing a diff tool for text. The reason is
> that XML is already structured, and the code's structure
> is reflected in the XML structure.

Yet in my experience most diff and merge tools are unable to deal with it properly. I've spent a good bit of time searching for good diff tools for XML. Instead, I ended writing an alphabetizing pretty-printer to get usable diffs for XML documents using standard diff tools which worked brilliantly.

> For example, no text-based diff tool can tell me today
> that two source files are equal when the only change is
> the re-ordering of declarations (i.e. the too files
> contain the same declarations, but in a different order).
> This is possible only with a structured format.

It's dangerous to assume that reordering is always meaningless.

> No, parsing is a big problem.
>
> For example, in order to make sense of a c++ program, for
> example, the parser has to be interleaved with the
> compiler: in order to understand what foo bar() means,
> information from the compiler must be fed to the parser.
> The c++ grammar is not context free, leading to all sorts
> of problems.

Not all text based languages are C++.

> If the c++ code was stored in XML format, with a tag
> 'variable', then foo bar() would be unambiguous: it would
> be a variable declaration.
>
> But this misses the point of my initial comment. I said
> that XML is a step in the right direction, not the
> solution. The advantage of XML is that it is a structured
> format, which is a quality that any universal data format
> should have. But it has many disadvantages (too verbose,
> too memory-hungry etc).

The BNF grammar for a text-based language is more structured than XML. Moreover XML is text-based. So saying that XML is a step forward from text is like saying a Ford is better than a car.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 8:40 AM

> I would love to have that kind of diff. However, I don't
> want to be limited in what I can use. I want it all. I
> want that and the ability to do text based diffs if that's
> my desire. I see no reason why both cannot be supported.

Ok, but that's no reason to say that code represented in a special format can not be handled properly by diff tools.

> In IT enterprise IT, for example, change management
> processes often require software must be able to be
> delivered directly from QA to Production without human
> intervention. A lot of tools lack a modular way to
> persist code and/or binaries in a way that they can be
> delivered in such a way. This reliably results in process
> violations and ultimately production problems.

That's one of the best reasons not to use text as a storage format, actually. Writing tools that can manipulate models that contain structured information is much easier and much less error prone.

> I think all of the things that you are suggesting here
> make sense. What I don't understand is why you believe
> that a textual format is the limiting factor, especially
> if you are assuming textual formatting and parsing will be
> supported anyway? What difference does it make whether
> the text comes from a screen element or from a file?

It's easier to manipulate data when they are not in a flat text file. That's the big advantage.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 8:41 AM

> > A new non-text based format would have all the benefits
> of
> > the text format as well as many more benefits.
>
> The question I keep asking but have not gotten an answer
> to is: what benefits can be provided in a non-text format
> that cannot be provided in a textual format? One example
> would be a start.

It's easier to parse a structured model than a flat text format.

Example? there are many. I gave you one above, regarding source code.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 8:49 AM

> > > A new non-text based format would have all the
> benefits
> > of
> > > the text format as well as many more benefits.
> >
> > The question I keep asking but have not gotten an
> answer
> > to is: what benefits can be provided in a non-text
> format
> > that cannot be provided in a textual format? One
> example
> > would be a start.
>
> It's easier to parse a structured model than a flat text
> format.

Source code is a structured model.

> Example? there are many. I gave you one above, regarding
> source code.

Don't see it.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 8:50 AM

> I think you misunderstand what 'universal' means. It
> doesn't imply any that at all. Any universal format will
> have the same limitations as bytes and text. You will not
> be able to make use of it without assumptions about it's
> structure.

No. A universal format means that any program can tell what it is inside a data model. It does not necessarily mean that the data are useful to the program. The program does not need to make assumptions of what the structure is, simply because the structure will be available to the program.

> This is the big lie about XML that I though
> was finally only believed by a minority. XML is useless
> as a persistence format without a schema and a description
> of what each element means.

No. XML is very useful. There is no need for a description of what each element means; XML is a passive format, i.e. it does not have semantics, it only has structure. Semantics is given by the programs that process the XML.

> Yet in my experience most diff and merge tools are unable
> to deal with it properly. I've spent a good bit of time
> searching for good diff tools for XML. Instead, I ended
> writing an alphabetizing pretty-printer to get usable
> diffs for XML documents using standard diff tools which
> worked brilliantly.

Perhaps the blame is on the tools.

> Not all text based languages are C++.

The problem exists in any kind of data that can not be expressed as a context free grammar. For example, a Word document.

> The BNF grammar for a text-based language is more
> structured than XML.

No. BNF is not more structured than XML. Actually, they can't be compared. BNF is the schema, XML is a storage medium.

> Moreover XML is text-based. So
> saying that XML is a step forward from text is like saying
> a Ford is better than a car.

No. A text format, a simple sequence of characters is not equivalent to XML. XML has structure semantics, allowing the easy parsing of data models. Free text does not have this property.

The problem we want to tackle here is structural semantics, and a simple text format does not provide any structural information.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 8:50 AM

> > In IT enterprise IT, for example, change management
> > processes often require software must be able to be
> > delivered directly from QA to Production without human
> > intervention. A lot of tools lack a modular way to
> > persist code and/or binaries in a way that they can be
> > delivered in such a way. This reliably results in
> process
> > violations and ultimately production problems.
>
> That's one of the best reasons not to use text as a
> storage format, actually. Writing tools that can
> manipulate models that contain structured information is
> much easier and much less error prone.

Huh? The point is that no modification is allowed. Any modification of the model from one environment to another is a violation.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 8:56 AM

> Source code is a structured model.

Source code is a structured model if the structure is visible only after parsing the text with the custom parsing rules of each language.

A structured format would make the parsing step unnecessary.

>
> > Example? there are many. I gave you one above,
> regarding
> > source code.
>
> Don't see it.

Example...

The classic C++ ambiguity:


foo bar();
foo bar() {}

With a structured format, no parsing ambiguity:


<variable type="foo" name="bar"/>
<function type="foo" name="bar"/>

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 8:57 AM

> > > In IT enterprise IT, for example, change management
> > > processes often require software must be able to be
> > > delivered directly from QA to Production without
> human
> > > intervention. A lot of tools lack a modular way to
> > > persist code and/or binaries in a way that they can
> be
> > > delivered in such a way. This reliably results in
> > process
> > > violations and ultimately production problems.
> >
> > That's one of the best reasons not to use text as a
> > storage format, actually. Writing tools that can
> > manipulate models that contain structured information
> is
> > much easier and much less error prone.
>
> Huh? The point is that no modification is allowed. Any
> modification of the model from one environment to another
> is a violation.

Manipulate does not always mean 'modify'; it can mean 'process' or 'copy and modify'.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 9:03 AM

> > I think you misunderstand what 'universal' means. It
> > doesn't imply any that at all. Any universal format
> will
> > have the same limitations as bytes and text. You will
> not
> > be able to make use of it without assumptions about
> it's
> > structure.
>
> No. A universal format means that any program can tell
> what it is inside a data model. It does not necessarily
> mean that the data are useful to the program. The program
> does not need to make assumptions of what the structure
> is, simply because the structure will be available to the
> program.

universal: including or covering all or a whole collectively or distributively without limit or exception

http://www.merriam-webster.com/dictionary/universal

> > This is the big lie about XML that I though
> > was finally only believed by a minority. XML is
> useless
> > as a persistence format without a schema and a
> description
> > of what each element means.
>
> No. XML is very useful. There is no need for a description
> of what each element means; XML is a passive format, i.e.
> it does not have semantics, it only has structure.
> Semantics is given by the programs that process the XML.

You are affirming my point. XML cannot processed as source code without some definition of what the elements mean.

Tell me what this means (real world field names):


<rctcdp>
  <dqp2nb>123456789</dqp2nb>
  <dqoinb>999</dqoinb>
</rctcdp>

The above is no more meaningful than this equivalent data:


rctcdp {
  dqp2nb: 123456789
  dqoinb: 999
}

Or this:


rctcdp 
  dqp2nb = 123456789
  dqoinb = 999

The only difference between these is that the first one can be parsed using easily obtained tools in many different languages.

> > Yet in my experience most diff and merge tools are
> unable
> > to deal with it properly. I've spent a good bit of
> time
> > searching for good diff tools for XML. Instead, I
> ended
> > writing an alphabetizing pretty-printer to get usable
> > diffs for XML documents using standard diff tools which
> > worked brilliantly.
>
> Perhaps the blame is on the tools.

Great. When the tools are improved, let me know.

> > Not all text based languages are C++.
>
> The problem exists in any kind of data that can not be
> expressed as a context free grammar. For example, a Word
> document.

I'm not advocating using word documents for code. As I've said before, the only worse format for code than XML is Word ;)

> > The BNF grammar for a text-based language is more
> > structured than XML.
>
> No. BNF is not more structured than XML. Actually, they
> can't be compared. BNF is the schema, XML is a storage
> medium.

You are unable to see the forest for the trees.

> > Moreover XML is text-based. So
> > saying that XML is a step forward from text is like
> saying
> > a Ford is better than a car.
>
> No. A text format, a simple sequence of characters is not
> equivalent to XML. XML has structure semantics, allowing
> the easy parsing of data models. Free text does not have
> this property.

XML is a subset of free text.

> The problem we want to tackle here is structural
> semantics, and a simple text format does not provide any
> structural information.

I know, BNF grammars do.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Version Control is Undo . . . or more

Posted: Dec 10, 2008 9:05 AM

> The classic C++ ambiguity:
>
> With a structured format, no parsing ambiguity:

Not all source is C++. In fact you have proven that text based formats can be unambiguous given that XML is text based.

Why do you assume that the source must be C++?

Flat View: This topic has 83 replies on 6 pages [ « | 1 2 3 4 5 6 | » ]

Previous Topic

Next Topic