The Artima Developer Community
Sponsored Link

Weblogs Forum
Adding Optional Static Typing to Python

49 replies on 4 pages. Most recent reply: Jul 22, 2010 10:29 AM by Dmitry Dvoinikov

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 49 replies on 4 pages [ « | 1 2 3 4 | » ]
Edward C. Jones

Posts: 3
Nickname: edcjones
Registered: Dec, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 27, 2004 11:40 AM
Reply to this message Reply
Advertisement
For quite a while, I have been interested in automatically wrapping C libraries with Python. I have written ugly piles of code which do this using Pyrex. It is clear to me that Python types like "C_unsigned_short" would make the task easier. I always assumed that this sort of type needed to be static. Not so. All static typing adds to Python is restrictions on the use of the names of certain objects.

I propose that a collection of types be added to Python that correspond to many of the standard types in C, C++, and FORTRAN. I chose these three languages because they are the most commonly used for writing libraries that someone might want to wrap with Python. These new types will simplify writing wrapper code for libraries. For example, in SWIG, C++ overloading would be easier to do. Examples are "Cshort", "Cunsigned_long_long", and "Cchar". I would also include an "arrayof" concept. The new types need not be static types.

Unfortunately, C types can have more than one meaning. The classic example is "char *" which can be an array of chars or a zero terminated string. There will need to be more than one Python type corresponding to "char *". Possible names might be "Cchar_array" and "Cnullterminatedstring".

Some questions:
1. What can be learned by studying the Pyrex, Psyco and SWIG internals?
2. Should parts of Pyrex, Psyco, and SWIG become part of Python?
2. Are there problems with allocating and deallocating memory?
3. Can SWIG be simplified if the only wrapper language is Python?

---------------------
Here is something I sent to the SWIG mailing list:

What features could be added to Python that would simplify using SWIG wrapped libraries in Python and wrapping C/C++ libraries without making Python difficult to read and write?

I think that it would be a good idea to have Python types corresponding to the C/C++ numerical and character types, arrays of these, and null-terminated strings. I don't think these need to be static types.

On the other hand, I think that all the C/C++ pointers, reference variables, etc., should remain buried out of sight of Python users.

Is any of this a good idea? Where should the line be drawn?

I am posting this to:

A blog by Guido van Rossum
Adding Optional Static Typing to Python
http://www.artima.com/weblogs/viewpost.jsp?thread=85551

The SWIG mailing list
http://mailman.cs.uchicago.edu/mailman/listinfo/swig

rahul

Posts: 2
Nickname: rahulgarg
Registered: Dec, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 27, 2004 9:30 PM
Reply to this message Reply
Hi.
I am adding more comments to clear up what i am saying.

I am assuming that optional type checking is being added for easier debugging only. So if 'expects' are turned on , python raises warnings(which do not halt the system) but not when they are turned off. These will enable easier debugging for new people(like me :)) while not affecting masters. Also,perhaps, it will be easier to accomodate till type checking mechanism is perfected(if it is implemented at all that is) so that python does not stop you when it is in fact python which
might be making some mistake.(This last statement is a guess only...)

It is similar to assert and __debug__=1 in a way.

So the crux is :
1.Expects is only a bridge between type checking and dynamic typing.

2.Type checking is done only as a tool which you are free to override if you want to.

3.The objective of type checking here is only to make debugging easier and not speed/optimization.

4.The point is not that 'expects' be a additional keyword.You can go like this also :
def (int a,int b): or whatever you like. Only that expects make it a bit clearer IMHO.

sincerely.,
rahul

Max Ischenko

Posts: 31
Nickname: maxi
Registered: Sep, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 28, 2004 3:44 AM
Reply to this message Reply
about "Compile Time vs. Run Time".

I 'd be completely happy if the type check would be performed at run-time, not on compile time or import time. After all, that's how a Python developer expects the things to be.

You may argue that's this doesnt buy you much but I disagree. ;-)

Firstly, it's a first and logical evolution from the current state of affairs. Secondly, it does provide some tangible benefits.

Currently, many developers are used to write something like this:


def foo(g,a):
assert isinstance(a, str), a
assert callable(g)
...


Strictly speaking, these assertions are not required, but they do help to detect errors as early as possible which is a sound engineering practice.

With the help of type interferencing or manifest typing we could free programmer from this tedious work, if he wants to.

Duck typing

What I really wanted to see in Python is an explicit support of DuckTyping, which is an integral part of type system anyway. The problem is IMO with a reasonable and helpfule implementation.

For some, a type interferencing (a sketch from the Haskell type system) may help. For instance if the foo() takes parameter a and then invokes method bar() on that parameter, compiler (or VM) could check (at least, at run-time) that the argument passed to foo() does have a callable bar attribute.

This also can be extended on operator overloading, such as __add__ or __lshift__.

And my point is, even with such a limited form (run-time checking of automatically derived type information), it could be a big step forward.

Optional type annotation (embedded in the source code) could be a next step.

Daniel Berger

Posts: 1383
Nickname: djberg96
Registered: Sep, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 28, 2004 6:47 AM
Reply to this message Reply
"On the other hand, it could make a lot of sense to introduce interfaces, which I've recently realized is a static type checking mechanism."

What advantage do interfaces have over mixins? With an interface, all you get is a method name (i.e. the beloved "contract" Java types always talk about). With a mixin, you get the method AND the method definition. Interfaces are a limited mechanism for static languages, and are almost useless for dynamic languages, except perhaps as some sort of odd static type checking mechanism, as you suggest.

Take a lesson from Ruby.

Dan

Ryan Paul

Posts: 2
Nickname: segphault
Registered: Dec, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 28, 2004 1:12 PM
Reply to this message Reply
I personally think that static type checking in a dynamic language is pointless. Changes in class state could easily nullify the benfits. In a dynamic language, methods shouldnt care about the type of the instance that they are passed, they should be more concerned with making sure that the instance is compatible with whatever operations the method will perform. As a result, I think that a DbC system, in conjunction with predicate based argument constraints that operate at run-time would be the best solution.

The following is an example that demonstrates argument constraint using an external (reusable) predicate:


pred Iterable:
hasattr(self, "__iter__")

def map(fn, lst:Iterable):
# ...


The following is an example that uses a condition block at the beginning of the method to perform the same check:


def map(fn, lst):
?:
hasattr(self, "__iter__")
# ...


I wrote an entry in my blog about it. It contains a link to a wiki page where I have started sketching out more syntactic details:

http://www.libervis.com/modules/weblog/details.php?blog_id=23&PHPSESSID=fc6d430a6433ea5933e236ed1517ae17

Michael Hobbs

Posts: 51
Nickname: hobb0001
Registered: Dec, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 28, 2004 2:47 PM
Reply to this message Reply
Some semi-random comments about static typing in Python:

Other Languages

O'Caml provides a nice example of how static duck typing can be done. The problem with it, though, is that writing the types explicitly can be quite tricky. The typing is best left to the type inferencer. If used in Python, I'd suggest that you don't allow for explicit typing, since it would make the code very unreadable. Of course, no type inferencer is 100% accurate, so if explicit types aren't allowed, the type checker would have to emit warnings instead of errors.

Dynamic Code

Code written like this would wreak havoc with any static type checker, both explicit and inferred:


def divide(x : int) -> int:
return x / 2

def redefine_divide():
def new_divide(x : str) -> str:
return x[0:len(str)/2]
global divide
divide = new_divide

redefine_divide()
divide("hello")


This is obviously a contrived example, but one of the useful features of Python is that it is so dynamic that you can redefine methods and functions at runtime.

Again, the solution to this is to have the type checker only emit warnings instead of errors, or perform the type-checking at runtime when the function is called.

Warnings vs. Errors

How, exactly, would a Python compiler perform static type checking without throwing an exception? (That is, only emit type warnings, not errors.) Currently, syntax errors are reported as exceptions when the module is first imported. How would type warnings be reported? Should the compiler be allowed to write a message to stderr and screw up any interface that might be displayed on the command line? Provide continuable exceptions? Maybe it just invokes some callback that can be replaced, like sys.settrace()? Anyway, it's a small detail, but...

Conclusion

If there is no acceptable way to emit only a type warning, static type checking really isn't feasable. In which case, the only proposed "improvement" is explicit type signatures on functions, which can only be checked at runtime. In that case, you might as well provide the entire DbC framework (including interfaces) and satisfy everyone.

Andre Roberge

Posts: 6
Nickname: aroberge
Registered: Dec, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 28, 2004 7:41 PM
Reply to this message Reply
As a python newbie, one thing really impressed me about the language: its readability, which makes it really easy to learn. (This has been mentioned so many times in various tutorials and other articles I have read about python, but I feel I must emphasize it from the point of view of beginners to the language).

I have also read about the versatility that its dynamic possibilities (e.g. "duck typing") adds and experienced it; thus, I wonder about the relative merits vs losses of adding optional static typing; human nature is such that people often use an "optional feature" of (computer language, program, electronic gizmo, etc.) just because they found out about it and feel somehow empowered about using it - not because it is really needed. I feel that the same could happen with static typing (and, unfortunately for relative newbies like me who try to read other people's code, with decorators as well).

As others have mentioned: what is the problem that is to be solved with this language feature? I would suggest that the chosen solution must somehow incorporate some information about the problem to be solved AND be annoying enough to use that it wouldn't be unless absolutely being required to. Here's an oversimplified example with a purposeful "ugly" solution to illustrate what I mean.

Suppose that one finds that the simple function

def gcd(a, b):
while a:
a, b = b%a, a
return b

is too slow. How does one knows that this function is too slow? (Remember, my hypothesis is that an "optional" feature that detracts to the readibility of the program must be used only when it is a true improvement; in this case, executing faster.) To be sure, one has to take the time to profile the function (within some test program). What if one could set up an option that would give the number of "operations" (opcode?) generated by the compiler and have to include it if the compiler is to take into account the "optional" static typing information - otherwise, it would give a warning to the programmer and not use the information. Furthermore, the compiler would first compile the function without the type information and compare with that as written by the programmer - and complain if the two didn't match.
Something like

@OPCODE(42) #because the answer has to be 42 ;-)
def gcd(a: int, b: int) -> int:
while a:
a, b = b%a, a
return b
@END_OPCODE

I realise that different python versions may change the number of "opcodes" generated and that the information would have to be updated before a program could make use of the static typing information; then again, new versions of python may negate the need for static typing of a given function.

I realise that my suggestion is rather "naive" and should not be taken literally. Rather, it is used to illustrate my belief that there should be a "penalty" to have to pay before having the possibility of using an "optional feature" which takes away from the beauty of python.

Just my $0.02.

André

David Vydra

Posts: 60
Nickname: dvydra
Registered: Feb, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 29, 2004 9:15 PM
Reply to this message Reply
> I wish programs were databases. We could look up gcd's
> constraints without having to look at them every time we
> look at it.


This is where good dev tools should help by constructing a dynamic view that hides certain details not of interest to me at the moment.

Nicolas Fleury

Posts: 7
Nickname: nidoizo
Registered: Dec, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 30, 2004 1:23 PM
Reply to this message Reply
First, I'll just comment the syntax, which I really don't like for the return value (->). Maybe a VB-like solution using as or another keyword (and I will use "as" in this message as a subliminal message;) ). The pyrex syntax seems better also. Anyway, I think the solution should be @decorators as a first step.

There's an important thing to me that is not mentionned in the article, the type of members. Right now, where I work we use something like:

def __init__(self):
self.names = {} # someType => someOtherType
self.count = 0
...


The dynamic nature of python make that self.names can at any time change of type, so if static typing is added, it should be possible to prevent that, or optimization and usefulness of typing is greatly reduced:

def __init__(self):
self.names as Dict(someType, otherType)
self.count as int = 0
...
def foo(self):
self.names = 7 # raise an exception


The same logic should be applied to arguments:

def foo(x as int):
x = "hello" # raise an exception


It means that typing would affect assignment to names and that additional information need to be saved in objects.

As others have pointed out, it's important to know what problems we are trying to solve. I'm not sure a C++-template-like mechanism is really useful. One of the things I like about Python is that manipulating types and instances is done at the same level, and does not involve complex metaprogramming like I'm doing in C++. What a

template T:
def min(a as T, b as T) as T: ...

brings more than using psyco (or even not using) is not worth the cost in my opinion. I really want Python to evolve to solve my day-to-day programming problems when working in team on a project, not improve performance which I already consider ok.

Regards,
Nicolas

Mike Meyer

Posts: 2
Nickname: mwm
Registered: Dec, 2004

Re: Adding Optional Static Typing to Python Posted: Dec 30, 2004 6:50 PM
Reply to this message Reply
> I personally think that static type checking in a dynamic
> language is pointless.

Common LISP shows this to be false. In Common LISP, declaring an
object to be of a type (which is optiona) says "I promise to only
attach objects of this type to this variable, so you can skip type
checking on it."

To me, that's what optional type declarations should be about. But
even that's not needed, as type inferencing can buy you most of that
performance gain.

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Adding Optional Static Typing to Python Posted: Dec 31, 2004 9:37 AM
Reply to this message Reply
In a language that I developed inside of AT&T Bell Labs for the 5ESS switch, I crossed this road too. I decided that the most explicit contract I could add without limiting runtime flexibilty was a simple type-check opcode evaluated at runtime. This provided a better indication to the test environment of what had happened and allowed the detection of incorrect types up front.

Basically, variables where declared either as

global a, b;


or

local c, d;


which provided scoping in context much the way that 'static' does in C (local acted like static).

When adding type limiting. I allowed a variable to be explicitly type limited at declaration as in:

global integer a, b;


In method/function declarations, types were also declarable on parameters and function/method variables.

When the compiler saw a typed variable being assigned a value, it would emit a type-check opcode with the expected
type name.

My language was smalltalk like in that a type only needed to have the appropriate methods defined to be compatible with use in any particular context. So, I, like you, Guido, did not want to add anything more explicit about type content. This would have greatly complicated the compiler too, which I did not want to do because of other reasons.

Good luck with finding a workable answer.

Maurice van Keulen

Posts: 1
Nickname: mvankeulen
Registered: Jan, 2005

Re: Adding Optional Static Typing to Python Posted: Jan 4, 2005 3:25 AM
Reply to this message Reply
Hi all,

A friend of mine directed my attention to your discussion on typing in Python, because I spent quite some attention to typing for object-oriented languages in my PhD research. Although I'm not that familiar with Python, I believe you can benefit greatly from the work of Guiseppe Castagna. He introduced what he calls ad-hoc polymorphism, an unobtrusive way of typing that supports all kinds of object-oriented features such as inheritance, function/method overloading, etc. I became especially enthousiastic about his work, when I read the superb article "Covariance and Contravariance: Conflict without a Cause" in which he provides a solution to this discussion on subtyping and overloading of functions/methods. I believe that his type system can be adapted and applied to Python and provides many of the things you would like in a language such as Python.

In case people are interested, please read his book "Object-Oriented Programming: A Unified Foundation" (http://www.di.ens.fr/~castagna/book.html; an extended and more readable version of his PhD thesis) or the paper I mentioned (see below). Guiseppe's homepage (http://www.di.ens.fr/~castagna/) also contains a link to a tutorial.

Paper: Giuseppe Castagna, "Covariance and Contravariance: Conflict without a Cause". In ACM Transactions on Programming Language Systems, volume 17, number 3, 1995. pp. 431-447.

Regards,
Maurice.
-- 
-----------------------------------------------------------
Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology
Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands
Email: m.vankeulen@utwente.nl, Phone: +31 53 4893688, Room: INF3039
WWW: http://www.cs.utwente.nl/~keulen

D H

Posts: 13
Nickname: dug
Registered: Jul, 2003

Re: Adding Optional Static Typing to Python Posted: Jan 4, 2005 7:00 AM
Reply to this message Reply
It wasn't mentioned by Guido or the commenters, but there already is a python-like language with static typing and type inference: http://boo.codehaus.org/

def gcd(a as int, b as int) as int:
    while a:
        a, b = b%a, a
    return b


See http://boo.codehaus.org/Gotchas+for+Python+Users

Guido van van Rossum

Posts: 359
Nickname: guido
Registered: Apr, 2003

Topic is closed! Posted: Jan 4, 2005 7:56 AM
Reply to this message Reply
Please read and add your comments to my "Part II" blog:

http://www.artima.com/forums/flat.jsp?forum=106&thread=86641

Jim Fulton

Posts: 3
Nickname: j1m
Registered: Jan, 2005

Interface declarations Posted: Jan 11, 2005 3:22 AM
Reply to this message Reply
I should note that there are a number of useful ways to talk about objects and interfaces. The most common is to say what interfaces a class implements. In addition, we often want to say that an object "provides" an interface.

After lengthy discussion we adopted the words "implements" and "provides" as follows:

An object provides an interface if the interface specifies the behavior of the object. For example, if an object provides an interface, then the methods defined by the interface are available on the object.

If a class implements an interface, then instances of the class provide the interface. We plan to generalize this to callables. IOW, a factory can implement an interface, meaning that the objects it produces provide the interface.

We have functions for declaring that individual objects provide interfaces.

Flat View: This topic has 49 replies on 4 pages [ « | 1  2  3  4 | » ]
Topic: Have Generics Killed Java? Previous Topic   Next Topic Topic: Reviewable, Runnable Specifications Ensure Software Quality

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use