The Artima Developer Community
Sponsored Link

Weblogs Forum
Software Development Has Stalled

164 replies on 11 pages. Most recent reply: Mar 28, 2010 10:20 AM by Florin Jurcovici

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 164 replies on 11 pages [ « | 1 ... 4 5 6 7 8 9 10 11 | » ]
Rinie Kervel

Posts: 26
Nickname: rinie
Registered: Oct, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 1:39 AM
Reply to this message Reply
Advertisement
> > I believe that Kervel meant that the strict typing of
> > programs prohibits the scriptability of systems. In
> other
> > words, if you model a spreadsheet as a collection of
> > cells, you must refer to it as of that type at all
> times.
> > You can not manage it with some generic code.
>
> OK. I don't think that's true but I think I follow what
> you are saying.
It's not the strict typing. But the 'everything is an object' and all containers should be 'containers of objects' that leads to considerable overhead. So I am not a fan of ORM or ActiveRecord: may be nice for the record you have to examine in detail, but not for the total resultset.

And indeed you lack some generic code for general manipulation, like the lowlevel access that SlqAlchemy has.
In SQL I can use select/from/where on any table or view.
In Delphi I could use some generic code to display 'any table'. Most Ajax frameworks I need to define a column model for my data to display. So yes I miss a 'display *' with some sensible defaults.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 3:01 AM
Reply to this message Reply
> > In other words, we need a common language between
> programs
> > and systems.
>
> C
>
> Have a fun time.

No, C doesn't cut it. An abstraction over C is required for systems interoperability and unified scripting.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 3:04 AM
Reply to this message Reply
> > I believe that Kervel meant that the strict typing of
> > programs prohibits the scriptability of systems. In
> other
> > words, if you model a spreadsheet as a collection of
> > cells, you must refer to it as of that type at all
> times.
> > You can not manage it with some generic code.
>
> OK. I don't think that's true but I think I follow what
> you are saying.

When I said 'generic code' I do not mean templates, but truly run-time generic code.

For example, suppose I have a spreadsheet like this:


class Spreadsheet {
public:
std::list<Cell *> cells;
}


I cannot manage this in run-time from another language, written with another compiler, for another CPU. Even C++ itself is not compatible at binary level between different compilers.

The required abstraction for unified scripting should specify an ABI.

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Software Development Has Stalled Posted: Feb 10, 2010 5:10 AM
Reply to this message Reply
> I am still playing around to find alternatives.

Have you yet read and understood Date, Darwen, McGoveran?

http://www.amazon.com/Introduction-Database-Systems-C-J-Date/dp/0321197844/ref=sr_1_3?ie=UTF8&s=books&qid=1265807060&sr=1-3

http://www.amazon.com/Databases-Types-Relational-Model-3rd/dp/0321399420/ref=sr_1_5?ie=UTF8&s=books&qid=1265807060&sr=1-5

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 7:56 AM
Reply to this message Reply
> When I said 'generic code' I do not mean templates, but
> truly run-time generic code.

I had not thought of templates.

> For example, suppose I have a spreadsheet like this:
>
>

> class Spreadsheet {
> public:
> std::list<Cell *> cells;
> }
>

>
> I cannot manage this in run-time from another language,
> written with another compiler, for another CPU. Even C++
> itself is not compatible at binary level between different
> compilers.

OK. What does C++ have to do with anything?

> The required abstraction for unified scripting should
> specify an ABI.

Isn't that kind of the point of a virtual machine? I'm not really sure what problem you are trying to solve. Are you looking for a universal language?

Rinie Kervel

Posts: 26
Nickname: rinie
Registered: Oct, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 8:06 AM
Reply to this message Reply
> > I am still playing around to find alternatives.
>
> Have you yet read and understood Date, Darwen, McGoveran?
>
> http://www.amazon.com/Introduction-Database-Systems-C-J-Dat
> e/dp/0321197844/ref=sr_1_3?ie=UTF8&s=books&qid=1265807060&s
> r=1-3
>
> http://www.amazon.com/Databases-Types-Relational-Model-3rd/
> dp/0321399420/ref=sr_1_5?ie=UTF8&s=books&qid=1265807060&sr=
> 1-5

Date in college. Daytime job is a WMS system using every available native Oracle SQL and PL/SQL feature.
So I'd like to think I understood enough.

What is missing in my opinion is that the access protocols should be REST like, using HTTP and JSON.
Using a database from a webbrowser, browsing the schema like a SqlDeveloper/Toad/Tora schemabrowser, having an editor open for a hoc queries, nice grid display and export to excel with clean URL's is one of the things i'm playing with. Perhaps some small CommonJS/JSGI middleware for business logic and acces control/logging.

RDBMS is nice for table like JSON data. Not for files, blobs or deep trees. So MongoDB, CouchDB and QT QML are other things to play with...

Rinie Kervel

Posts: 26
Nickname: rinie
Registered: Oct, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 8:17 AM
Reply to this message Reply
>
> For example, suppose I have a spreadsheet like this:
>
>

> class Spreadsheet {
> public:
> std::list<Cell *> cells;
> }
>

>
That's just the internal representation inside you (blackboxed) code. How dou you store this on disk/in the cloud. Do you read all or just the cells you are interested in. Page can have several related rows (some cells you'd like a total of) and some unrelated rows, loose remarks or totally unrelated data.

So I would think of a spreadsheet as a few tabs each with some sparse regions of data, a few functions and probably some text (headers, comments) and graphics.

Just a list of Cells makes it hard to observe that a function-cell has relative references so you can apply (a copy of) the same formula to different rows.

Spreadsheet optimisation has to recognize the 'is a copy of' instead of just cloning the cel to another one.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 9:23 AM
Reply to this message Reply
> > When I said 'generic code' I do not mean templates, but
> > truly run-time generic code.
>
> I had not thought of templates.
>
> > For example, suppose I have a spreadsheet like this:
> >
> >

> > class Spreadsheet {
> > public:
> > std::list<Cell *> cells;
> > }
> >

> >
> > I cannot manage this in run-time from another language,
> > written with another compiler, for another CPU. Even
> C++
> > itself is not compatible at binary level between
> different
> > compilers.
>
> OK. What does C++ have to do with anything?
>
> > The required abstraction for unified scripting should
> > specify an ABI.
>
> Isn't that kind of the point of a virtual machine? I'm
> not really sure what problem you are trying to solve. Are
> you looking for a universal language?

Aren't we discussing universal scripting and a common communication language for computers, in the context of software development having stalled?

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 9:33 AM
Reply to this message Reply
> Aren't we discussing universal scripting and a common
> communication language for computers, in the context of
> software development having stalled?

I wasn't. I was asking what the proposal was for not thinking of a row as a collection.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 9:47 AM
Reply to this message Reply
> >
> > For example, suppose I have a spreadsheet like this:
> >
> >

> > class Spreadsheet {
> > public:
> > std::list<Cell *> cells;
> > }
> >

> >
> That's just the internal representation inside you
> (blackboxed) code. How dou you store this on disk/in the
> cloud. Do you read all or just the cells you are
> interested in. Page can have several related rows (some
> cells you'd like a total of) and some unrelated rows,
> loose remarks or totally unrelated data.
>
> So I would think of a spreadsheet as a few tabs each with
> some sparse regions of data, a few functions and probably
> some text (headers, comments) and graphics.
>
> Just a list of Cells makes it hard to observe that a
> function-cell has relative references so you can apply (a
> copy of) the same formula to different rows.
>
> Spreadsheet optimisation has to recognize the 'is a copy
> of' instead of just cloning the cel to another one.

I do not really understand what you are trying to say. Can you please elaborate?

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Software Development Has Stalled Posted: Feb 10, 2010 12:02 PM
Reply to this message Reply
> > > I am still playing around to find alternatives.
> >
> > Have you yet read and understood Date, Darwen,
> McGoveran?
> >
> >
> http://www.amazon.com/Introduction-Database-Systems-C-J-Dat
>
> >
> e/dp/0321197844/ref=sr_1_3?ie=UTF8&s=books&qid=1265807060&s
>
> > r=1-3
> >
> >
> http://www.amazon.com/Databases-Types-Relational-Model-3rd/
>
> >
> dp/0321399420/ref=sr_1_5?ie=UTF8&s=books&qid=1265807060&sr=
>
> > 1-5
>
> Date in college. Daytime job is a WMS system using every
> available native Oracle SQL and PL/SQL feature.
> So I'd like to think I understood enough.
>
> What is missing in my opinion is that the access protocols
> should be REST like, using HTTP and JSON.

Again, the view of a coder, not a data geek. Both Codd and Date offered up relational sub-languages; what we got was SQL, better than nothing. The notion that data can sit in an uncontrolled lump to be poked at by various and sundry "clients" is what's led to the Babel we have. The datastore has to be the point of control, otherwise it's just files, and of no additional value; MySql became popular just because coders were too lazy to write I/O routines and so used its primitive SQL syntax to move data in and out of files while ignoring any relational semantics.

If you want webservers (REST, HTTP, JSON, etc.) to taken on datastore control functions (IBM would love you for a client), good luck. It's not as easy as you think. Some very smart people, a few of which I've known, have been grappling with implementing Dr. Codd's model for nearly 40 years. SQL databases are still not fully relational. Part of the reason is that coders still insist on looping through records in application code (see here: http://www.simple-talk.com/sql/performance/writing-efficient-sql-set-based-speed-phreakery/ for an object lesson), so the vendors haven't been willing to make the effort to educate and build a truly relational data language; Chamberlin's SQL isn't. Part of the reason is that set operations are inherently parallel, and until recently, parallel operations in hardware were not widely feasible. Today, they are. Add SSD storage, and BCNF datastores are a piece of cake, if you want.

Shared, disciplined data is the point of the relational database (and to a lesser extent, its predecessors); this is what Codd and Date, et al, have built. Coders generally want to be able to scribble willy-nilly in the data lump without concern for others. Again, doing so is the main cause of the Data Babel.

You can't have it both ways: either Data Babel with each client program poking at some files with no concern for others, or a shared disciplined datastore. If the latter, then the database engine has control of update. That puts control in the hands of data geeks, not coders, leaving coders to display and accept user input. This is not inherently second-class citizenship, but coders (who are almost always ignorant of Codd) always think they know enough to specify databases. Kind of like insisting on doing neurosurgery because you can dismember a chicken in the kitchen.

> Using a database from a webbrowser, browsing the schema
> like a SqlDeveloper/Toad/Tora schemabrowser, having an
> editor open for a hoc queries, nice grid display and
> export to excel with clean URL's is one of the things i'm
> playing with. Perhaps some small CommonJS/JSGI middleware
> for business logic and acces control/logging.

There are lots of multi-database editors; on windoze my fave is AQT. If you're talking to a real database, you will need a compliant client; that can't be avoided. ODBC/JDBC provide a large common set of functions.


> RDBMS is nice for table like JSON data. Not for files,
> blobs or deep trees. So MongoDB, CouchDB and QT QML are
> other things to play with...

Every industrial strength database has the capability to reference/link external files and blobs, if you want. With CTE (SQL2), trees are not an issue, unless you're on Oracle, I believe you're still are limited to CONNECT BY.

Rinie Kervel

Posts: 26
Nickname: rinie
Registered: Oct, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 11:31 PM
Reply to this message Reply
> > >
> > > For example, suppose I have a spreadsheet like this:
> > >
> > >

> > > class Spreadsheet {
> > > public:
> > > std::list<Cell *> cells;
> > > }
> > >

> > >
> > That's just the internal representation inside you
> > (blackboxed) code. How dou you store this on disk/in
> the
> > cloud.
> I do not really understand what you are trying to say. Can
> you please elaborate?
I think you lose the structure of a spreadsheet if you view it just as a bunch of cells. IMHO it is just a representation of some user models (rows with data and functions, summaries etc). Improv tried to make these models explicit in the user interface, but never took off. I think we should discover the models and represent them in the code, without changing the the view that is presented in the GUI.

Rinie Kervel

Posts: 26
Nickname: rinie
Registered: Oct, 2005

Re: Software Development Has Stalled Posted: Feb 10, 2010 11:58 PM
Reply to this message Reply
> > > > I am still playing around to find alternatives.
> > >
> > > Have you yet read and understood Date, Darwen,
> > McGoveran?
> > >
> > So I'd like to think I understood enough.
> >
> > What is missing in my opinion is that the access
> protocols
> > should be REST like, using HTTP and JSON.
>
> Again, the view of a coder, not a data geek. Both Codd
> and Date offered up relational sub-languages; what we got
> was SQL, better than nothing. The notion that data can
> sit in an uncontrolled lump to be poked at by various and
> sundry "clients" is what's led to the Babel we have. The
> datastore has to be the point of control, otherwise it's
> just files, and of no additional value; MySql became
> popular just because coders were too lazy to write I/O
> routines and so used its primitive SQL syntax to move data
> in and out of files while ignoring any relational
> semantics.
It did the job for them and is better accessible than blackboxed binary repositories the 'serialisation' people are so fond off.
What Codd/Date did not have are Schema browsers with Excel like datagrids, URLs and favorites.
Using Toad, SqlDeveloper end users, not data or code geeks can access data and see what is going on.
Although this does not change anything theoretically it makes the data much more accessible, so they can point out errors in business logic (or in the data itself) without having to consult some guru.
>
> If you want webservers (REST, HTTP, JSON, etc.) to taken
> on datastore control functions (IBM would love you for a
> client), good luck. It's not as easy as you think. Some
> very smart people, a few of which I've known, have been
> grappling with implementing Dr. Codd's model for nearly 40
> years. SQL databases are still not fully relational.
I can live with SQL as JSON over HTTP. 3rd normal form is not my goal, but getting things done without too much regret in 3 years...
> Part of the reason is that coders still insist on looping
> g through records in application code (see here:
> http://www.simple-talk.com/sql/performance/writing-efficien
> t-sql-set-based-speed-phreakery/ for an object lesson), so
> the vendors haven't been willing to make the effort to
> educate and build a truly relational data language;
Sometimes pure relational wins, sometimes a PL/SQL loop gets the job done easier. It's like XML fanboys saying everything is a tree, where a lot of problems are happy with table solutions.
> Chamberlin's SQL isn't. Part of the reason is that set
> operations are inherently parallel, and until recently,
> parallel operations in hardware were not widely feasible.
> Today, they are. Add SSD storage, and BCNF datastores
> s are a piece of cake, if you want.
>
> Shared, disciplined data is the point of the relational
> database (and to a lesser extent, its predecessors); this
> is what Codd and Date, et al, have built. Coders
> generally want to be able to scribble willy-nilly in the
> data lump without concern for others. Again, doing so is
> the main cause of the Data Babel.
Our coders like the data as a relatively stable centre of the application. So different readers/writers/views acces the data and make migration of code and schema relatively painless.
>
> You can't have it both ways: either Data Babel with each
> client program poking at some files with no concern for
> others, or a shared disciplined datastore. If the latter,
> then the database engine has control of update. That puts
> control in the hands of data geeks, not coders, leaving
> coders to display and accept user input.
I have absolutely no problems with isolating data access / queries from the code and accept optimization changes from DBA's. Truth is, we should cooperate: Data geek, Business logic geek, Code geek and GUI geek. We should partition our mostly declarative JSON so that these professions can work together but also mostly individual.
>
> There are lots of multi-database editors; on windoze my
> fave is AQT. If you're talking to a real database, you
> will need a compliant client; that can't be avoided.
> ODBC/JDBC provide a large common set of functions.
I want JSON, HTTP an Webbrowser and an ExtJS grid.
So I can play with/browse data, queries and views and then copy/paste them in the business logic in a way that DBA and frontend can see what is going on and optimize/analyse their part.
>
>
> > RDBMS is nice for table like JSON data. Not for files,
> > blobs or deep trees. So MongoDB, CouchDB and QT QML are
> > other things to play with...
>
> Every industrial strength database has the capability to
> reference/link external files and blobs, if you want.
> With CTE (SQL2), trees are not an issue, unless you're on
> n Oracle, I believe you're still are limited to CONNECT BY.
Well the capability is there, but blobs suck because the may not have structure from an RDMBS point of view, but they do have from an application point of view (mp3 tags, jpeg exif tags). RDBMS (and JSON/HTTP) are not appropriate for video, music and photo's. Here I like Azures seperate Blob store more than the local RDBMS approach

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 3:28 AM
Reply to this message Reply
> > Aren't we discussing universal scripting and a common
> > communication language for computers, in the context of
> > software development having stalled?
>
> I wasn't. I was asking what the proposal was for not
> thinking of a row as a collection.

Isn't that question in the context of universal scriptability of systems? because the poster that brought that up said:

So my 2 cents are: we need a compact, mostly declarative way of 'scripting' programs and systems. Not try to model everything as (bottom up) objects: a spreadsheet page is not a collection of cell-objects, a sql row is not a collection of column objects.

So your question about sql row not being a collection of column objects is related directly to the need for a way to script programs and systems.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 3:31 AM
Reply to this message Reply
> > > >
> > > > For example, suppose I have a spreadsheet like
> this:
> > > >
> > > >

> > > > class Spreadsheet {
> > > > public:
> > > > std::list<Cell *> cells;
> > > > }
> > > >

> > > >
> > > That's just the internal representation inside you
> > > (blackboxed) code. How dou you store this on disk/in
> > the
> > > cloud.
> > I do not really understand what you are trying to say.
> Can
> > you please elaborate?
> I think you lose the structure of a spreadsheet if you
> view it just as a bunch of cells. IMHO it is just a
> representation of some user models (rows with data and
> functions, summaries etc). Improv tried to make these
> models explicit in the user interface, but never took off.
> I think we should discover the models and represent them
> in the code, without changing the the view that is
> presented in the GUI.

I still don't get it. In what other way can we view a spreadsheet? can you give me an example?

Flat View: This topic has 164 replies on 11 pages [ « | 4  5  6  7  8  9  10  11 | » ]
Topic: Heron Tackles the WideFinder Challenge Previous Topic   Next Topic Topic: Setting Multiple Inheritance Straight

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use