The Artima Developer Community
Sponsored Link

Weblogs Forum
When Designs Matter

29 replies on 2 pages. Most recent reply: Apr 18, 2006 5:15 PM by Bill Venners

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 29 replies on 2 pages [ « | 1 2 ]
Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Fear? Posted: Apr 11, 2006 11:42 AM
Reply to this message Reply
Advertisement
> > > So, is it just that you're afraid of losing your job
> or is
> > > there something else going on?
> >
> > How can people say "just that you're afraid of losing
> your
> > job"? Our company just had 25% of its staff cut after
> > being bought out for the second time in 3 years. I'm
> still
> > here, but try losing your job and then put the word
> "just"
> > in front of that statement.
>
> Been there, done that, have the debts to prove it. :-)
> :-(
>
> > Maybe you can afford to "just" lose your job. I don't
> > think most of us can.
>
> Ah, so, for many people, it is fear and the attendant
> aversion to the negative consequence which dominates the
> desire for integrity, quality, etc.
>
> So, if fear is the fundamental issue, what can we do about
> it? Or do you think that we're just doomed to be stuck in
> this hell forever?
>
>
> "If you haven't been fired often enough then you're not
> doing your job."
--JDM

Personally, I'm working on not needing to work for anybody else. I'd be happier in a hell of my own creation, I suppose.

John D. Mitchell

Posts: 244
Nickname: johnm
Registered: Apr, 2003

Changing Organizations Posted: Apr 11, 2006 11:43 AM
Reply to this message Reply
> I must either be particularly bitter today, have a really
> crappy place to work or have just given up because I read
> this and had to snort.

I'm reminded of a quote from Martin Fowler: "Change your organization or change your organization."

> Honestly, I'm tired of taking on
> the the responsibility for management not being able to
> figure out what they want. On the one hand management
> wants to treat the development staff like nothing more
> than a bunch of interchangeable parts. On the other hand,
> when the shit hits the fan we all get a "pep talk" about
> how we're supposed to just magically make everything ok,
> no matter what decisions have been taken out of our hands
> in regards to developing the software that is, supposedly,
> the life blood of our business.

Hmm... Is this an example where blame is given to people (the developers) who have no (organizational) authority to make things succeed? I.e., the managers will get any credit if the project succeeds but the developers get (most? all?) of the blame when things go down the toilet?

> For you few technical people that actually have a say and
> get some respect, God bless you and keep fighting the good
> fight. The rest of us, I think, need to go start our own
> businesses and simply crush the idiots we currently work
> for.

But how can people do that if they are afraid?

John D. Mitchell

Posts: 244
Nickname: johnm
Registered: Apr, 2003

Re: Communication, Trust, and Systemic Issues Posted: Apr 11, 2006 11:55 AM
Reply to this message Reply
> I think a lot of the problems come from the dot com
> bubble. There was a dearth of talent and mobs of
> charismatic charlatans in IT. We still carry the load of
> imcompetent people and the charlatans, though reduced in
> number, have left a indellible stain on our industry.

Ah, so perhaps we're (at least partially) suffering from Sturgeon's Revelation (http://en.wikipedia.org/wiki/Sturgeon's_law) -- i.e., 90% of everything is crap.

> Part of the problem is that the average upper management
> type has the computer skills of my 99 year old grandmother
> (well, maybe that's an exageration but you get the point.)
> They are completely unable to distinguish skilled
> technical minds from skilled bullshitters. I think a
> lot of them figure they have no real way to ensure success
> on their own so they might as well go as cheap as possible.

True. :-)

But don't we developers (as a group) have the same problem (i.e., hiring crappy developers)? I.e., don't we have a big responsibility to make sure that we are only hiring good people (and not the BS'ers)? I.e., isn't this another example of a need for us to "enforce" integrity of the team?

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Communication, Trust, and Systemic Issues Posted: Apr 11, 2006 12:14 PM
Reply to this message Reply
> But don't we developers (as a group) have the same problem
> (i.e., hiring crappy developers)? I.e., don't we have a
> big responsibility to make sure that we are only hiring
> good people (and not the BS'ers)? I.e., isn't this
> another example of a need for us to "enforce" integrity of
> the team?

(Toots own horn) I feel that when developers have been involved in hiring (like at my last job) we made rather good choices. The problem was all the people that were there from when developers were not invlolved.

At my new job, I 'need to learn how to interview' before I can do that and I would say my own technical interviews were fairly light. Nothing like what I used to do to put people through their paces. but I 'don't know' how to interview so I'm not involved with that.

It also tricky to get out from under people who are clearly in over their heads and desperately fighting to prevent that from being discovered. But I'm working on it.

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Changing Organizations Posted: Apr 11, 2006 2:50 PM
Reply to this message Reply
> Hmm... Is this an example where blame is given to people
> (the developers) who have no (organizational) authority to
> make things succeed? I.e., the managers will get any
> credit if the project succeeds but the developers get
> (most? all?) of the blame when things go down the toilet?
>

That's pretty much it. Maybe things will be different under new management, but as you said above, Change your organization or Change your organization.

Too bad change is so hard :-)

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Changing Organizations Posted: Apr 12, 2006 11:12 AM
Reply to this message Reply
well. my 2 kopeks.

Being a data guy from way back (before Codd), what I've seen happen since java, and xml, and SOA, etc. is that the application building melieu has returned to the bad old days before Codd. Thus we are revisiting old problems.

Here's the notion. BC, there was smart code (typically COBOL) and dumb data (typically VSAM, ISAM, son-of-SAM). Such an approach leads quickly to sealed applications (don't believe the crap about 'open APIs', it's crap). With the ascendence of java, and CS graduates who know nothing of SQL databases, much less the RM, we've ended up back in the 1960's. The java kiddies want smart code (java 'action classes') and dumb data (bean objects). For those with memories, there's no real difference between a bean object and a COBOL copybook. That's another reason those who view the current 'problem' with a increasingly jaundiced eye have no sympathy: the kiddies don't want to stand on the shoulders of the giants who gave them an answer. They choose to re-invent a square wheel. javBOL anyone?

SQL database driven development (admitting that SQL databases aren't fully RM) is, really, an object oriented way of treating the problem: the data and its behaviour are in one place. You gather the data from someplace (screen, wire, sphincter) and throw it at the database. Either it meets the constraints (so it sticks), or it doesn't (gets thrown back). Simple. Elegant. Not much code. Oh my!

Of course, kiddies who only know java get uncomfortable when shown a method which hasn't much need of them. Boo hoo.

Design the datastore. Generate the UI (if you need one) from that. Go home to the wife and kiddies. And, yes, there are off the shelf products which do this; both Open Sores and Fee.

John D. Mitchell

Posts: 244
Nickname: johnm
Registered: Apr, 2003

Re: Communication, Trust, and Systemic Issues Posted: Apr 12, 2006 3:42 PM
Reply to this message Reply
> In my experience the realistic estimate is not acceptable.
> We are continually pushed to cut corners. My most painful
> experience of this was dropping in a consultant written
> module into our product as is. I did the best I could with
> the two or so weeks I was given to work on it. To do it
> correctly would have taken about 12 - 13 weeks. Every day
> for two and a half years now I have paid for that decision
> made by management. I did everything I could to let people
> know that this was a REALLY BAD IDEA.

So, how much of this is bad communication and lack of trust between the managers and the developers?

How much do you think this is an issue of different expectations as to the actual need? I.e., is it really a problem where you're expecting that the needs are more that they are and the business people are expecting a lot less?

[...]
> My one big question about this great search for the
> underlying cause of systemic issues is, if we find the
> answer, will it really change anything? I think we've
> known the answer for more than three decades now. Still we
> don't acknowledge the simple fact that designing large
> robust software systems is a very arduous task that takes
> a lot of time and a lot of talent. All this other soul
> searching and focusing on process over talent is a smoke
> and mirros show to divert us from that simple truth. As
> long as there is a search for some underlying "trust"
> issue or process that can be tweaked we can conveniently
> ignore this fact, pay developers less than what they are
> worth, blame them when things out of their control go
> wrong and all in all continually dump on them for the
> simple reason that they'll take it.

I concur -- there are no silver bullets. IMHO, the truly fundamental piece of why anything sucks (or is great) is how much people care. Everything stems for the (lack of) caring. Processes/methodologies are doomed to failure when not enough people care to make them work.

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Communication, Trust, and Systemic Issues Posted: Apr 12, 2006 5:09 PM
Reply to this message Reply
> > In my experience the realistic estimate is not
> acceptable.
> > We are continually pushed to cut corners. My most
> painful
> > experience of this was dropping in a consultant written
> > module into our product as is. I did the best I could
> with
> > the two or so weeks I was given to work on it. To do it
> > correctly would have taken about 12 - 13 weeks. Every
> day
> > for two and a half years now I have paid for that
> decision
> > made by management. I did everything I could to let
> people
> > know that this was a REALLY BAD IDEA.
>
> So, how much of this is bad communication and lack of
> trust between the managers and the developers?
>
> How much do you think this is an issue of different
> expectations as to the actual need? I.e., is it really a
> problem where you're expecting that the needs are more
> that they are and the business people are expecting a lot
> less?
>

It certainly wasn't lack of trust or communication. Everybody, including the head of our development department that made the final call, acknowledged that the consultant written solution was a piece of crap. However, in the interest of expediency it was deemed "good enough" by people that don't have the task of maintaining or updating it.

Maybe there is a disconnect on needs as you say, but we do a lot of customizations around this particular component because it does not meet all the business needs. I'm not sure whose needs the component meets as is because I don't think we have a single customer that uses said component without massive customizations. Maybe we would still need the customizations but they could be a lot less painful.

> [...]
> > My one big question about this great search for the
> > underlying cause of systemic issues is, if we find the
> > answer, will it really change anything? I think we've
> > known the answer for more than three decades now. Still
> we
> > don't acknowledge the simple fact that designing large
> > robust software systems is a very arduous task that
> takes
> > a lot of time and a lot of talent. All this other soul
> > searching and focusing on process over talent is a
> smoke
> > and mirros show to divert us from that simple truth. As
> > long as there is a search for some underlying "trust"
> > issue or process that can be tweaked we can
> conveniently
> > ignore this fact, pay developers less than what they
> are
> > worth, blame them when things out of their control go
> > wrong and all in all continually dump on them for the
> > simple reason that they'll take it.
>
> I concur -- there are no silver bullets. IMHO, the truly
> fundamental piece of why anything sucks (or is great) is
> how much people care. Everything stems for the (lack of)
> caring. Processes/methodologies are doomed to failure
> when not enough people care to make them work.

I think too many people in a large organization care about too many different things. As a developer I care about the robustness and stability of my solutions. Consultants care about how customizeable it is to meet customer demands. Sales people want it to look all pretty so they can sell better. Support people want it to be as error free as possible. Middle managers want me to get my monthly status reports in on time. That's a lot of different masters to serve. As I write this I think this is on of the main reasons that small upstarts keep scooping up the big boys lunch. A smaller organization, simply by being smaller, will be better aligned to a single vision.

Here's a group of people that got something done

http://www.anetek.com/Microsoft1978.jpg

Compare that with, as of Sept 30, 2005, 63,564 employees. It's gotta be hard to steer that many people in one direction.

If nobody cared nobody would be having this conversation. I don't know how many times I've said "forget it, I don't care anymore." It makes me a bold faced liar because when I come back in the next day, somehow I give a rat's ass again. Caring is damn hard. I try to put on a thick coat of apathy each day before coming into work. Most days it wears off before I get in the front door.

John D. Mitchell

Posts: 244
Nickname: johnm
Registered: Apr, 2003

Re: Communication, Trust, and Systemic Issues Posted: Apr 12, 2006 6:56 PM
Reply to this message Reply
> It certainly wasn't lack of trust or communication.
> Everybody, including the head of our development
> department that made the final call, acknowledged that the
> consultant written solution was a piece of crap. However,
> in the interest of expediency it was deemed "good enough"
> by people that don't have the task of maintaining or
> updating it.

Ah, yes. Sounds like more of a case of different people with different ideas about what the "costs" are. Especially when there's a short-term, immediate payoff vs. a lingering cost over time (particularly when you're the one who has pay the cost).

[...]
> I think too many people in a large organization care about
> too many different things. As a developer I care about the
> robustness and stability of my solutions. Consultants care
> about how customizeable it is to meet customer demands.
> Sales people want it to look all pretty so they can sell
> better. Support people want it to be as error free as
> possible. Middle managers want me to get my monthly status
> reports in on time. That's a lot of different masters to
> serve.

Indeed. So then it comes to how much the group cares and how the group makes decisions.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Changing Organizations Posted: Apr 14, 2006 3:57 PM
Reply to this message Reply
> SQL database driven development (admitting that SQL
> databases aren't fully RM) is, really, an object oriented
> way of treating the problem: the data and its behaviour
> are in one place. You gather the data from someplace
> (screen, wire, sphincter) and throw it at the database.
> Either it meets the constraints (so it sticks), or it
> doesn't (gets thrown back).

The data inserted into the DB, therefore it's good? That's a terrifying assumption.

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Changing Organizations Posted: Apr 15, 2006 8:42 PM
Reply to this message Reply
> > SQL database driven development (admitting that SQL
> > databases aren't fully RM) is, really, an object
> oriented
> > way of treating the problem: the data and its
> behaviour
> > are in one place. You gather the data from someplace
> > (screen, wire, sphincter) and throw it at the database.
> > Either it meets the constraints (so it sticks), or it
> > doesn't (gets thrown back).
>
> The data inserted into the DB, therefore it's good?
> That's a terrifying assumption.

ah somebody bit.

the quote reveals that the assumption of databases being merely surrogates for files is not yet dead. this is why systems suck.

the point of the RDBMS is that it only allows data in that meet the constraints. so, the application code sitting out in the weeds has only one function: provide a nice format for input. it can also, if one chooses to, *redundantly* enforce constraints, which are stored in the RDBMS and read by a source code generator. the point is: *meets the contraints*.

the resurgence of the COBOL mind set (many posts from many people on the issue), that constraints can/should be enforced in client (application) code is why systems suck so bad. kind of like eating soup with 10 foot chopsticks; doesn't work all that well.

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Changing Organizations Posted: Apr 16, 2006 8:56 PM
Reply to this message Reply
> the quote reveals that the assumption of databases being
> merely surrogates for files is not yet dead. this is why
> systems suck.
>
> the point of the RDBMS is that it only allows data in that
> meet the constraints. so, the application code sitting
> out in the weeds has only one function: provide a nice
> format for input. it can also, if one chooses to,
> *redundantly* enforce constraints, which are stored in the
> RDBMS and read by a source code generator. the point is:
> *meets the contraints*.
>
> the resurgence of the COBOL mind set (many posts from many
> people on the issue), that constraints can/should be
> enforced in client (application) code is why systems suck
> so bad. kind of like eating soup with 10 foot chopsticks;
> doesn't work all that well.

The systems I work are are required to run against a variety of RDBMS systems and as such it makes much more sense to enforce the constraints in a middle tier. It's bad enough having to know all different nuances for simply setting up tables, enforcing permissions and optimizing performance between Oracle, SQL Server, MySQL, DB2, etc. without having to write stored procedures, triggers, and so on for all those platforms.

Our products have a middle tier that handles teh constraints, so we funnel all that logic through one place. Our applications, as you suggest, don't do much more than provide a nice place for the user to input data and run processes. But without that middle tier handling the business logic I don't know how we would be able to get the things done we do and support the number of platforms we do.

To say that systems suck simply because of the location of the business logic seems illogical to me. Why would it be any easier to write solid constraints in an RDBMS system than it would be application code? What makes writing directly against the RDBMS so special that it is able to stop idiots from doing idiotic things?

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Changing Organizations Posted: Apr 16, 2006 9:21 PM
Reply to this message Reply
> What makes writing
> directly against the RDBMS so special that it is able to
> stop idiots from doing idiotic things?

As I've (and lots of database developers; there is a current thread on the subject) said: put the constraints on the data with the data. This is the improvement that Dr. Codd gave us. Idiots can't change the data if the database constraints return: "You're an idiot. You can't do that. Go away". This is a really basic notion of relational (even, dare I say, SQL) databases. It also means that any application can access the data without messing it up.

I collect Quotes of the Day for my CubeLand e-mailer (really irritates the COBOLers). I'm not there, so I can't paste it in, but one goes like: "I'd rather learn SQL than 100,000 programmers' APIs".

The non-portablility of SQL is a myth. There is very little one needs to do which requires vendor specific syntax. And for those occasions, there's Swissql, which does a syntax transform. One could do the same with awk.

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Changing Organizations Posted: Apr 16, 2006 11:07 PM
Reply to this message Reply
>
> As I've (and lots of database developers; there is a
> current thread on the subject) said: put the constraints
> on the data with the data. This is the improvement that
> Dr. Codd gave us. Idiots can't change the data if the
> database constraints return: "You're an idiot. You can't
> do that. Go away". This is a really basic notion of
> relational (even, dare I say, SQL) databases. It also
> means that any application can access the data without
> messing it up.
>
> I collect Quotes of the Day for my CubeLand e-mailer
> (really irritates the COBOLers). I'm not there, so I
> can't paste it in, but one goes like: "I'd rather learn
> SQL than 100,000 programmers' APIs".
>
> The non-portablility of SQL is a myth. There is very
> little one needs to do which requires vendor specific
> syntax. And for those occasions, there's Swissql, which
> does a syntax transform. One could do the same with awk.

If non-portable SQL is a myth, why would one need tools to port SQL? In my experience, working on a multitude of platforms, it's the "very little one needs to do" that is usually the source of the most pain.

If your main concern is moving lots of data into and out of a database then I would totally agree with your quote. There is plenty of software that does a hell of a lot more than that, though. Your quote can equally substitute just about any language or technology if the application is narrow enough.

The constraints should be as close to the data as possible. Where we can we put it right on the data and let the DBMS take care of making sure bad data doesn't get into the database. However, there is a lot that we want to do with the data besides shuttle it into and out of the database. There seems to be a never ending debate about whether you should use the DBMS tools (stored procedures and triggers, mainly) to apply business logic to the data or whether this logic should be in some middle layer. I'm firmly in the camp that says this logic should be somewhere else. You'll need more than pithy quotes to convince me otherwise.

And, really, I've seen enough bad SQL/C/C++/C#/VB/awk/python/PHP/pick-your-favorite-language that I can't believe that just because somebody chooses to work in the database layer that they will miraculously write better stuff than somebody that works in a different layer. That's as bad as all the snooty application developers that believe people that choose to work in the database layer are all morons that can't do any higher level programming.

Bill Venners

Posts: 2244
Nickname: bv
Registered: Jan, 2002

Re: Changing Organizations Posted: Apr 18, 2006 5:15 PM
Reply to this message Reply
> The constraints should be as close to the data as
> possible. Where we can we put it right on the data and let
> the DBMS take care of making sure bad data doesn't get
> into the database. However, there is a lot that we want to
> do with the data besides shuttle it into and out of the
> database. There seems to be a never ending debate about
> whether you should use the DBMS tools (stored procedures
> and triggers, mainly) to apply business logic to the data
> or whether this logic should be in some middle layer. I'm
> firmly in the camp that says this logic should be
> somewhere else. You'll need more than pithy quotes to
> convince me otherwise.
>
I don't have a requirement to support lots of databases, though I do keep in mind we may want to upgrade from Postgres to Oracle or some other commercial database someday. Nevertheless, enforcing constraints on data is one area where I believe in violating the DRY (Don't Repeat Yourself) principle.

One of the things I thought was hacky from the Pragmatic Programmers' Agile Programming with Rails book was how they, in the name of the DRY principle, check for valid data (such as non-null, etc.) in the entity layer of the application. If there's a problem, they throw an exception with a message that gets shown on the UI to the user. So the entity layer is providing (non-localized in this case) user interface messages.

I think that error messages that show up in the user interface should really originate in the user interface layer. In our case these will be localizable. I think working with the user to get valid input is a conversation, a potentially different process than simply checking data for correctness before persisting. So we do checking in the user interface layer.

But I also think the entity layer should be checking too. Because it shouldn't let anything pass. Here I just throw exceptions, because a pre-condition of calling into these APIs is that the passed data is valid.

But I also really think it is useful to have those constraints checked again by the DBMS itself, both as an extra check in case there's a bug in a higher layer, and because that one application isn't the only way to get data into the database. The database may outlive that application. Other applications may be added that also access the same data. Admins may tweak things with an SQL interpreter, and so on. Right now in Artima's new architecture I'm only doing things like non-null, foreign key, or unique, easy things that don't require writing any stored procedures. But I'm tempted to use stored procedures for certain simple cases that we can declare in our entity DSLs, such as a positive number, or a valid user name, or valid email address. Bad data, when it slips into the database, can cause application failures and cost money to clean up.

These constraints are, in my mind, part of the business logic. So I believe it is useful to have some business logic there in the database, even if the vast majority of business logic is in the application itself. Certainly things like unique, non-null, and foriegn keys, which are very easy to use declaratively, cost little and help a lot.

Flat View: This topic has 29 replies on 2 pages [ « | 1  2 ]
Topic: When Designs Matter Previous Topic   Next Topic Topic: Bridging Static and Dynamic Typing


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us