Article Discussion
Type Checking and Techie Control
Summary: Bruce Eckel talks with Bill Venners about why he prefers Python's latent type checking and techie control of language evolution.
23 posts.
The ability to add new comments in this discussion is temporarily disabled.
Most recent reply: January 11, 2006 6:58 AM by Michael
    Bill
     
    Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
    Type Checking and Techie Control
    July 6, 2003 10:48 PM      
    Bruce Eckel says, "Once I get the program running, I feed it real data. Then I'm going to find some interesting errors. So the sooner I can do that, the sooner I can find those errors."

    Read this Artima.com interview with Bruce Eckel:

    http://www.artima.com/intv/typing.html

    What do you think of Bruce's comments?
    • Frank
       
      Posts: 135 / Nickname: fsommers / Registered: January 19, 2002 7:24 AM
      Re: Type Checking and Techie Control
      July 9, 2003 11:49 AM      
      I used to think that static type checking was a life-saving programming tool. I've been gradually changing my mind about this.

      First, the first big program I wrote (many years back now), was written in Perl. It started out as a few scripts on a Web server, and over time grow into a full-fledged enterprise application for a finance company. It had a huge uptime (better than 99.999%), and had been used non-stop by several 100 people for years. Perl's weak typing was never an issue in writing that app, because I made very small changes to the system over several years (mainly because the system was constantly in use), and each small iteration brought to fore the biggest bugs during testing. So we could just fix those bugs right away, and type-mismatch bugs never found their way into the production system.

      I think static typing is important when developers use tools that exploit them. With many IDEs, such as IDEA, I can just click on a variable that refers to a customer, and up pop all the methods I can invoke on that customer. I don't have to remember those methods, their parameters, or return types. I think it's a huge productivity booster, because I get to stay in one spot of the program and don't have to interrupt my flow looking at docs, etc (something Bruce had talked about in an earlier interview). I can just keep typing (pun intended).

      Static typing becomes even more important when we access network-based resources. However, few systems provide a strongly typed network object model (Jini is one exception). With static typing, if I want to access a inventory management system, I can just write code that says, in effect, "OK, at this point just find an object that implements the Inventory type." Since I know the methods of that type, I can continue programming to that interface without having to know how someone implemented that inventory system. In reality, I might not even be able to find out the implementation, even if I wanted to, since we're talking about an object that comes across the network from some remote location. So a strongly typed system allows me to have the object's type be the only prior agreement with whoever implemented that Inventory object. On the other hand, progamming in an untyped network environment is often a complete nightmare - having remember protocol messages, message order and content, etc., are a very unproductive experience.

      About Bruce's remarks on feeding real data into a program: I found that dynamic typing often breaks down during data loading. When I consume a text file, for instance, to feed someone's database into my program, I can use Python or Perl to read that text file. That Python program is going to parse the file, and give a variable that corresponds to, say, position 6 in the database file, which would, say, represent the sale price of an inventory item (e.g., a float). But a lot of databases out there are really often corrupt - position 6, for instance, might be blank, or represent some mistaken value. And that means that now *I* have write my own type checker. But I wouldn't have to do that if I could feed typed data into my program, i.e., a SaleItem. Because that object presumably conforms to all the contracts of what it means to be a SaleItem, I can take those guarantees for granted.

      In fact, one of the biggest productivity boosters I came across lately is to put a strongly typed service layer on top of data sources that my program has to rely on.
    • Jay
       
      Posts: 1 / Nickname: mcsquared / Registered: March 7, 2003 7:51 AM
      Re: Type Checking and Techie Control
      July 7, 2003 5:13 AM      
      I started using Python three weeks ago and I like it very much.
      I tried to convince my colleagues to switch to Python, but the biggest problem is the "type checking" issue. Runtime type checking just scared them away.
      Bruce Eckel's arguments are not so convincing to them(even not to me myself, and I like Python and Thinking in XXX).
      Hope somebody could make a top ten list about why runtime type checking is GOOD(or not that BAD).
      • Matt
         
        Posts: 62 / Nickname: matt / Registered: February 6, 2002 7:27 AM
        Re: Type Checking and Techie Control
        July 7, 2003 1:38 PM      
        > Hope somebody could make a top ten list about why runtime
        > type checking is GOOD(or not that BAD).

        I don't know about a top ten list, but when I first started learning Python, I was concerned about this issue, as well. The thing you learn as you begin to use it more, is that it is really a non-issue. Later, when using Java or C#, you realize how much a pain always working static types can be.

        I think both static and dynamic typing have their merits and are not mutually exclusive. I'd like to see a language that has both.

        As it is now, there are many tasks where Python is clearly superior to Java and many where the reverse is true.

        Until the perfect super language arrives (i.e. never), it is probably useful to know at least one scripting/dynamically typed language (Python, Ruby, Perl, etc.), one VM-based strongly typed language (Java, C#, Smalltalk, etc.) and C/C++ for the low-level stuff (via JNI and extensions, or just by building stand-alone tools).
        • Isaac
           
          Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
          Re: Type Checking and Techie Control
          July 10, 2003 0:39 PM      
          one VM-based strongly typed language (Java, C#, Smalltalk, etc.)
          Smalltalk is strongly typed, but at run-time not at compile-time - it's a dynamically typed language.
        • Isaac
           
          Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
          Re: Type Checking and Techie Control
          July 11, 2003 4:19 PM      
          I think both static and dynamic typing have their merits and are not mutually exclusive. I'd like to see a language that has both.

          Try these for both static and dynamic typing:

          Dylan http://www.functionalobjects.com/

          Claire http://claire3.free.fr/description.htm
    • Isaac
       
      Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
      Re: Type Checking and Techie Control
      July 10, 2003 0:21 PM      
      in Java we've also found out that we check things at both compile time and runtime
      Strange to have a discussion of static vs. dynamic typing without mentioning that some type systems are much more powerful than others.

      Haskell, Clean, OCaml (and potentially Nice) have type systems that are powerful enough to infer the types of functions and variables at compile time. The programmer simply doesn't need to do finger-typing. Let's not confuse the inadequacies of Java's type system with the capabilities of modern static typing.
      • Ross
         
        Posts: 1 / Nickname: rossjudson / Registered: July 15, 2004 6:45 AM
        Re: Type Checking and Techie Control
        July 15, 2004 10:55 AM      
        Isaac is absolutely right. Implicit typing is very powerful and probably the best way to do larger-scale development.

        Dynamic typing means you can't test it (code coverage etc). Static typing means that you sometimes have more typing to do, and that your program may be less able to adapt at runtime if the object model changes.

        The best world is implicit typing coupled with something like Eclipse. You certainly don't have to wait until you "compile" your program to find out if there are typing errors in it; this is a flaw in the editing environment, not the language. Good language support within Eclipse for implicitly typed languages would give you the best of both worlds. A "to-do" list could show you implicitly called functions there's no current match for, and allow you to write them quickly.

        The fact that Java 1.5's "templates" do not include the primitive types makes them borderline useless. I can count the number of times I've made a casting error while iterating over a collection class on one hand, in many years of Java coding. I can't count the number of times I've had to put together crappy object-based structures with wrapper objects or worse, just to handle primitives in a reasonable fashion (not to mention the ridiculous storage overhead this incurs).

        I figured Java templates would allow me to make a collection of longs to doubles, and do it efficiently. Guess we'll all still be waiting...
        • Isaac
           
          Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
          Re: Type Checking and Techie Control
          July 16, 2004 8:07 AM      
          I figured Java templates would allow me to make a collection of longs to doubles, and do it efficiently
          For example?

          This is the Nice programming language
          http://nice.sourceforge.net/

          void main(String[] args){
          let a = new ArrayList();
          a.add(30000.0d);
          a.add(30000.0d);
          println(a[0]==a[1]);
          }

          I:\Nice\Test>nicec --sourcepath .. -a t.jar -R Test
          nice.lang: parsing
          Test: parsing
          Test: typechecking
          Test: generating code
          Test: linking
          Test: writing in archive
          nice.lang: writing in archive

          I:\Nice\Test>java -jar t.jar
          true
    • Michael
       
      Posts: 2 / Nickname: mdaconta / Registered: February 11, 2003 11:58 AM
      Re: Type Checking and Techie Control
      July 9, 2003 7:55 AM      
      Having looked at Python several times, the lack of runtime type checking has always been a show-stopper. All developers who have been in the business awhile know how difficult it is to develop robust, error-free code so the sooner bugs are caught the better. This idea of "wait until runtime" to catch errors is unacceptable. The rule of thumb is to catch as many errors, as early in the process, as possible. The "saving finger-typing" argument holds no weight when compared to catching bugs. For mission critical and safety-conscious systems, run-time type checking catches errors *before* serious consequences occur.

      When productivity squares off against reliability, productivity should lose.

      - Mike
      • Isaac
         
        Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
        Re: Type Checking and Techie Control
        July 10, 2003 0:54 PM      
        For mission critical and safety-conscious systems... reliability
        The important thing here is what reliability means for each specific system. Dynamically-typed Smalltalk has been used to develop mission critical, reliable systems for many investment banks. What made the systems reliable is that they didn't crash & burn when there was a runtime error. They halted, paged a developer; and allowed the developer to debug the problem, correct data, and crucially resume from where the error occured. They were reliable because Smalltalk can be hot-fixed.

        How about 99.999% availability in dynamically typed Erlang?

        Ericsson AXD 301 ATM Switch control system,
        1 million lines of Erlang source code
        400,000 lines of C/C++ code
        13,000 lines of Java code
        200 person project

        http://www.erlang.se/publications/Ulf_Wiger.pdf
      • Matt
         
        Posts: 62 / Nickname: matt / Registered: February 6, 2002 7:27 AM
        Re: Type Checking and Techie Control
        July 9, 2003 11:05 AM      
        > Having looked at Python several times, the lack of runtime
        > type checking has always been a show-stopper. All
        > developers who have been in the business awhile know how
        > difficult it is to develop robust, error-free code so the
        > sooner bugs are caught the better. This idea of "wait
        > until runtime" to catch errors is unacceptable. The rule
        > of thumb is to catch as many errors, as early in the
        > process, as possible. The "saving finger-typing" argument
        > holds no weight when compared to catching bugs. For
        > mission critical and safety-conscious systems, run-time
        > type checking catches errors *before* serious consequences
        > occur.
        >
        > When productivity squares off against reliability,
        > productivity should lose.

        It seems that there is this very popular implicit assertion that compile time type-checking finds and eliminates (or prevents) a large number of programming errors.

        I agree with your final conclusion, for some cases, but I haven't seen any convincing data to show that runtime type-checking really eliminates that many bugs. Like any programmer, I make plenty of mistakes in the process of writing software. I don't think that type-checking would save me from any of them. I don't think my Python code is any more buggy than my C++, Java or C# code.

        That is, I think using a type incorrectly, or using the wrong type is a relatively rare mistake. Even in C, where it is most dangerous and easiest to do, I don't imagine that is the source of a significant number of programming errors.

        This is the same reason I began to hate the Hungarian notation convention with such a passion. If you are so ignorant of your code that you don't even know what type of variables you are using, then you are probably better off not touching the code. In many cases I've seen, the Hungarian notation usage is carried to such an idiotic extreme that all you know is the type of the variable and you are hard pressed to figure out what its purpose is.

        Incidentally, the ubiquity of editors with code-completion features (for statically and dynamically typed languages alike) further lowers the already slim chances of misusing types.

        Maybe there are some studies of this subject somewhere that show conclusively that programs written with compile time type-checking are overwhelmingly more reliable and of higher quality than those with runtime type-checking only? If so, please point me to them and I will capitulate.

        Finally I'll repeat my usual point that these programming tools are not mutually exclusive. (Yes, it is actually possible to run Python and Java on the same machine (really!) -- heck, it is even possible to run them both in the same virtual machine!).
        • Michael
           
          Posts: 2 / Nickname: mdaconta / Registered: February 11, 2003 11:58 AM
          Re: Type Checking and Techie Control
          July 11, 2003 9:00 AM      
          > > When productivity squares off against reliability,
          > > productivity should lose.
          >
          > It seems that there is this very popular implicit
          > assertion that compile time type-checking finds and
          > eliminates (or prevents) a large number of programming
          > errors.
          >
          > I agree with your final conclusion, for some cases, but I
          > haven't seen any convincing data to show that runtime
          > type-checking really eliminates that many bugs. Like any
          > programmer, I make plenty of mistakes in the process of
          > writing software. I don't think that type-checking would
          > save me from any of them. I don't think my Python code
          > is any more buggy than my C++, Java or C# code.
          >
          > That is, I think using a type incorrectly, or using the
          > wrong type is a relatively rare mistake. Even in C,
          > where it is most dangerous and easiest to do, I don't
          > imagine that is the source of a significant number of
          > programming errors.

          It is not necessary to try and quantify how many
          bugs are caused by the lack-of type-checking for
          three reasons:

          1. It may not matter based on the type of application
          or system your are building.

          2. The answer is greater than 0.

          3. I can program in languages with strong typing.
          I don't consider this overly burdensome. In fact,
          I would love it if Python included this as an
          optional feature.

          As an illustrative example, there has certainly been
          widely publicized bugs relating to bad data and
          mismatched types. One in particular in the
          Mars Lander -- see http://mars.jpl.nasa.gov/msp98/news/mco990930.html.

          Best wishes,

          - Mike
          • Merriodoc
             
            Posts: 14 / Nickname: brandybuck / Registered: March 24, 2003 6:00 AM
            Re: Type Checking and Techie Control
            July 11, 2003 11:56 AM      
            > As an illustrative example, there has certainly been
            > widely publicized bugs relating to bad data and
            > mismatched types.

            There are also many widely publicized bugs regarding bad data and perfectly matched types. Pick your favorite buffer overflow error. One can argue this all day either way. While interesting, it doesn't really get anyone anywhere.

            I figure that it doesn't pay quantify how many bugs are caused by lack of type checking because regardless of whether the language is strongly typed, there are bugs.

            Although, some (finger) type checking would have picked up this one :-)
            > 1. It may not matter based on the type of application
            > or system your are building.
          • Matt
             
            Posts: 62 / Nickname: matt / Registered: February 6, 2002 7:27 AM
            Re: Type Checking and Techie Control
            July 11, 2003 6:59 PM      
            > It is not necessary to try and quantify how many
            > bugs are caused by the lack-of type-checking for
            > three reasons:
            >
            > 1. It may not matter based on the type of application
            > or system your are building.

            Huh?

            > 2. The answer is greater than 0.

            I disagree. It has to be enough to justify the drawbacks. If you are going to write an program with 1000 bugs in one month using Java or C#, versus writing the same program with 1001 bugs in one week using Python or Ruby, which is better? You've got three extra weeks of testing to find one bug.

            Anyway, how do you know that compile-time type checking reduces the number of bugs? Maybe it just changes the nature of bugs.

            > 3. I can program in languages with strong typing.
            > I don't consider this overly burdensome. In fact,
            > I would love it if Python included this as an
            > optional feature.

            I would too. Interfaces would be a very nice addition too (see Frank Sommers' post above).

            But let's not confuse strong typing with dynamic typing; Python does have strong types, just not declarations and compile-time checking.

            > As an illustrative example, there has certainly been
            > widely publicized bugs relating to bad data and
            > mismatched types. One in particular in the
            > Mars Lander -- see
            > http://mars.jpl.nasa.gov/msp98/news/mco990930.html.

            I looked briefly, but I couldn't see where the problem was based on using a dynamically-typed language. Are you saying the lander software wasn't written in a statically-typed language and rewriting it in one would have eliminated that bug?
            • David
               
              Posts: 2 / Nickname: dkempster / Registered: April 14, 2003 2:10 AM
              Re: Type Checking and Techie Control
              July 21, 2003 8:06 AM      
              This discussion comes up over and over. What is interesting is the static checking must be done side of the debate has never done anything with a dynamically typed language while the pro dynamic typing people have almost always done both.

              The benefits of static typing are largely a myth. They were never there in the first place for safety. That was a justification used after the fact. Go back and see why C based languages for example have static typing in the first place. Also C is probably the least safe programming environment that there is.

              Also someone talked about environments like IDEA that make good use of static typing, and that is true. But some of the features mentioned have been in Smalltalk environments from the start. I can ask to Inspect any variable and what do you know it tells me specifically what the object is and what methods it has. How?!? This must be impossible without strong typing right? Well the fact is the objects are strongly typed. It is just you don't have to tell what the variable is. This allows for polymorphism. Statically typed languages get around the type system and allow for polymorphism (which remember relies on the code NOT knowing what type it has - only what messages it responds to) by coming up with various cheats like casting and having the language use a richer notion of types.

              I think if you are going to have a statically typed language you should have one like Eiffel that has a much better worked out theory of types and has generics so casting is not necessary.

              But the fact is I know of no one that has used dynamically typed languages for any length of time that finds they have more bugs than a statically typed language. Don't just go by what you read in your CIS 101 text book.
              • Isaac
                 
                Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
                Re: Type Checking and Techie Control
                July 21, 2003 11:03 AM      
                various cheats like casting and having the language use a richer notion of types
                Why is having a richer notion of types cheating?
              • David
                 
                Posts: 3 / Nickname: malven / Registered: July 2, 2003 2:32 PM
                Re: Type Checking and Techie Control
                July 21, 2003 5:02 PM      
                David Kempster wrote:
                "But the fact is I know of no one that has used dynamically typed languages for any length of time that finds they have more bugs than a statically typed language."

                The way the question is phrased makes it impossible to refute. There are many other features that contribute to the "defect rate" of a language than typing. For example, there are *many* features of C++ that are problematic for non-expert programmers, and removing static typing wouldn't help one bit.

                My own experience might be interesting to relate. For a long period of time, as a consultant I worked on projects in both Smalltalk and in C++. The overall defect rate in C++ was much higher and the productivity lower (as you might expect for general application programming). But we found that as components were evolved by different groups of programmers, relatively speaking many more integration errors crept into the Smalltalk code base over time.

                One programmer would change a class in some way, seemingly have everything working, and then it would break code elsewhere in the application or in another application using a library. There was no comprehensive way to see what a change might affect, and often these bugs lay dormant in code until much later.

                As Bob Martin and others have said, a comprehensive unit test suite can go a long way toward compensating for the lack of static typing. But lacking such a suite you are on dangerous ground as code gets large and multiple developers are working together.

                BTW, contrary to your original statement, I have used many different OO languages (both statically and dynamically types) and I prefer static typing for building large-scale systems.
          • Greg
             
            Posts: 1 / Nickname: gregjor / Registered: February 8, 2004 4:49 PM
            Re: Type Checking and Techie Control
            July 19, 2004 9:21 AM      
            > As an illustrative example, there has certainly been
            > widely publicized bugs relating to bad data and
            > mismatched types. One in particular in the
            > Mars Lander -- see
            > http://mars.jpl.nasa.gov/msp98/news/mco990930.html

            As I understand that problem, it had nothing to do with programming language types. I don't know the code, but it sounds to me like a variable named "altitude" was interpreted by one team as feet and another as meters. The compiler/interpreter just saw a numeric type that seemed to match. I don't see how static typing can help in such cases, where the wrong algorithms or interpretations are applied to data.

            If I had written that code I would probably have chosen a variable name like "altitudeMeters" to remind myself and others. That would work in either C++ or Python.

            Greg Jorgensen
            PDXPerts LLC, Portland, Oregon USA
    • Hermanns
       
      Posts: 2 / Nickname: jigen / Registered: January 28, 2003 1:08 AM
      Re: Type Checking and Techie Control
      July 7, 2003 0:04 AM      
      I missed one point in the discussion about runtime vs. compiletime
      type-checking and that is the self-documenting effect when I
      explicitly specify the types I use.


      I think this is much more communicating:


      public Configuration getConfiguration(File file) throws IOException


      than


      def getConfiguration(file)
      • Matt
         
        Posts: 62 / Nickname: matt / Registered: February 6, 2002 7:27 AM
        Re: Type Checking and Techie Control
        July 7, 2003 1:23 PM      
        > I missed one point in the discussion about runtime vs.
        > compiletime
        > type-checking and that is the self-documenting effect when
        > I explicitly specify the types I use.
        >
        > I think this is much more communicating:
        >
        >
        > public Configuration getConfiguration(File file) throws
        > IOException
        >


        Hmm... looks kind of redundant to me... Also, to increase the redundancy further, you may want to write several versions of this same method in Java; one that takes a BufferedReader, one that takes a String filename, etc.

        > than
        >
        > def getConfiguration(file)
        >


        What about this?

        def getConfiguration(configFile):
        'Reads the specified file and returns a Configuration object; throws IOException.'
        • Hermanns
           
          Posts: 2 / Nickname: jigen / Registered: January 28, 2003 1:08 AM
          Re: Type Checking and Techie Control
          July 7, 2003 10:51 PM      
          > > I missed one point in the discussion about runtime vs.
          > > compiletime
          > > type-checking and that is the self-documenting effect
          > when
          > > I explicitly specify the types I use.
          > >
          > > I think this is much more communicating:
          > >
          > >
          > > public Configuration getConfiguration(File file) throws
          > > IOException
          > >

          >
          > Hmm... looks kind of redundant to me... Also, to
          > increase the redundancy further, you may want to write
          > several versions of this same method in Java; one that
          > takes a BufferedReader, one that takes a
          > String filename, etc.

          To achieve the same thing in Python you would have only one method (wich accepts several types), with dispatching code at the beginning of the method. I think this is much worse.

          > > than
          > >
          > > def getConfiguration(file)
          > >

          >
          > What about this?

          > def getConfiguration(configFile):
          > 'Reads the specified file and returns a
          > rns a Configuration object; throws IOException.'


          This is exactly the problem I see with runtime-typed languages. Instead of specifying the types in the normal sourcecode, you begin specifying those things in informal comments!
          • Jiri
             
            Posts: 1 / Nickname: iceni / Registered: June 29, 2003 8:40 PM
            Re: Type Checking and Techie Control
            July 10, 2003 4:49 AM      
            > > >
            > > > public Configuration getConfiguration(File file)
            > throws
            > > > IOException
            > > >

            > >
            > > Hmm... looks kind of redundant to me... Also, to
            > > increase the redundancy further, you may want to write
            > > several versions of this same method in Java; one that
            > > takes a BufferedReader, one that takes a
            > > String filename, etc.
            >
            > To achieve the same thing in Python you would have only
            > one method (wich accepts several types), with dispatching
            > code at the beginning of the method. I think this is much
            > worse.
            >

            In the getConfiguration method:

            Either,
            1. you would accept an ancestor common to
            BufferedReader, String, File that you use for sequential reading, or

            2 will write overloaded versions of getConfiguration.

            Both things can be worked out in Python:

            add 1. fine, Python has virtual methods
            add 2. getConfigurationFromBufferedReader, getConfigurationFromString, getConfigurationFromFile, etc.


            Jiri
          • Michael
             
            Posts: 1 / Nickname: mtb / Registered: January 11, 2006 1:50 AM
            Re: Type Checking and Techie Control
            January 11, 2006 6:58 AM      
            I realize this is an old thread, but it seems worth pointing out:

            If you really care that much about types in your Python code, just handle the exceptions yourself and either run unit tests or run it through pychecker.

            For the most part, I've found as I embrace OOP more, the types don't really matter. That is to say, I have a function that parses and computes taxes on objects with a 'price' member which I convert to a decimal format before computation. I dont need to know what type of object it was, so long as it has a 'price' that can be converted to numeric (and Python's quite happy to convert strings for me too if they've been read in from files without converting).