> The question is, how does an interface specify the > relationship of one method to another? For example, how > would an interface say, "In order for your class to > properly implement the mapping interface, its > get(key, default) method should behave like > __getitem__(key), except that if the key is > not present it should return default instead > of failing."
This is an interesting problem, but scratches only the surface of a bigger issue, which touches on the associated invariants of an interface. (For example, after x.append(y), len(x) has increased by one, and x[len(x)-1] is y, etc.)
I don't know of any realistic programming language that even begins to touch this one, and I think it's an interesting Ph.D. topic, but not something we should try to solve today by adding semantics to Python's interfaces.
Mathematically, some simple data structures like sets, lists and dicts (and I suspect certain kinds of trees) can probably be specified with extreme rigor, but that would (a) exclude a lot of interesting implementations that cheat in some way (I can imagine all sort of useful mappings that violate some dict invariants) and (b) be impossible to validate. (Even in Java this would be impossible, and Python is a lot more dynamic. I expect one would quickly encounter the halting problem if one tried.)
What happens in practice is that you should trust that something is a mapping when it claims to conform to the mapping interface, and if it breaks, too bad.
I believe that Zope uses so-called "marker" interfaces which have no methods but are intended to convey to the user that an object has certain semantics that isn't expressible in method signatures. This, and documenting intended semantics, is IMO sufficient.
So, with respect, I ask you again to give this up. :-)
I just want to say I agree with the root message (Stop the Flames). I think the syntax should be used for duck-typing, and simplifying our day-to-day life and forget about compiler optimization is the way to go. I'm very happy with this direction.
I just hope that the chosen syntax will be scalable to specify types at additional places (and consequently enable auto-completion at more places). For example the iteration of a typeless container:
for i:MyClass in mylist: # cool, auto-completion-and-doc for i.
> I just hope that the chosen syntax will be scalable to > specify types at additional places (and consequently > enable auto-completion at more places). For example the > iteration of a typeless container: > > for i:MyClass in mylist: > # cool, auto-completion-and-doc for i.
That syntax would work (and it is one of the reasons why I like my syntax better than other proposals), although I'm -1 on adding it now; let's reserve it for the future.
But this reminds me of Java code where we have to declare the type of the loop variable even if it's totally evident from the type of the container. I'd much rather eventually evolve to declaring the (presumably) argument mylist as sequence[MyClass] or list[MyClass] or iterable[MyClass] or whatever is appropriate, so that i's type can be inferred without having to say it.
If Python were to allow assert: to introduce a bloc of code, instead of having to repeat assert at the beginning of each line and also allow the identification
as you (GvR) have suggested, one could (almost) write the following as valid Python code:
01 def gcd(a, b): 02 assert: 03 a: int 04 b: int 05 a != 0 and b != 0 # fake pre-condition 06 return c assert: 07 c: int 08 c != 0 # fake post-condition 09 '''Returns the Greatest Common Divisor, 10 implementing Euclid's algorithm. 11 Input arguments must be integers; 12 return value is an integer.''' 13 while a: 14 a, b = b%a, a 15 return b
The more general form would be:
01 def foo(...): 02 assert: . type information and/or . pre-conditions . return [...] assert: . type information and/or . post-conditions . '''docstring''' . ...body...
where we have three blocks at the same level: 1) the static typing information and/or pre/post-conditions 2) the docstring (as a visual separator) 3) the actual function body statement.
I have written about using a similar syntax in my blog (aroberge.blogger.com) and it has received some (limited) positive feedback.
Disclaimer: I am a relative newbie to Python and have no experience in programming language design.
Four years ago when I was a Python newbie I would have argued strongly for static types. I had felt that this feature was badly needed and wondered how much better the language would be with them. It's not that I thought Python was a bad language, I just believed it would be a better language with types.
Now I thought I had many good reasons why types should be added. For well over a decade I had bought into all the common arguments that were made to prove that types were a necessary evil. You know, basically the benefits out weigh the costs. After all, some really smart people came up with very convincing and sound arguments over the years and experiences I had seamed to back up there arguments.
Over the years I have programmed in countless languages and any time I used a language that lacked types it was always a painful experience. In languages where types were optional I would eventually use them for nearly all variables. I would make exceptions for some of them like i, j, k, and other common variables.
Something was different about Python, as it wasn't a painful experience. It was a pleasure and it even put the fun back into programming. Although, it took awhile for me to accept the fact that it was a pleasure. I had felt like I was breaking a taboo. Programming without the safety net of types. Have I sold my soul to the devil? Have I forgot the painful lessons I have learned over the years? Why could I not stop using Python? It made so sense. All I knew was that the programs I wrote in Python were more reliable than the ones I wrote in other languages. I was also far more productive. I also felt that I was becoming a better programmer at a faster rate. The light bulbs were turning on in my head far more often than in the past.
After more than 12 years of faithfully believing in types I realized that they just aren't that important in creating quality code. When you think about it, what type of protection do you really get from them. If you say you need an int, is that what you really need? Do you need the full range. Do you need just positive numbers (oh that's what unsigned int is for)? Maybe even or odd numbers? Maybe values from 0 to 100? Does it protect you from a divide by zero? I could go on and on here.
OK, so we add constraints. How complex do we make these constraints? After all, we can dream up so many possibilities. At some point isn't it just simpler to write the code out than use a complex constraint syntax.
In an earlier reply to this blog I had made a harsh comment. I didn't intend to be harsh but was too lazy to write a proper response. Guido slapped my hands and stated that unit tests, like static types were no panacea. I agree with Guido that unit tests are no panacea but I do believe they can play the role of protector that some programmers have come to expect from static types.
Now some will argue that it's crazy to define a unit test to test an int when a static type of an int would do the trick (assuming that what you really need is the full range of an int). To some degree I also agree but I also wonder what the consequence of adding static types would have on the language or better yet the users of the language. Will it add roadblocks to the learning curve to the many users.
I know I became a better programmer because Python lacked types. If they were optional I would have used them and I might not have realized how dependent I became on them and the false sense of protection that they gave me. How good it makes you feel when you compile code and it has no errors. Even better when there are no warnings. Even better yet when you set the compiler options to increase the number of warnings it will provide and none are given. Makes you feel real good but it doesn't really make much of a difference in the quality or correctness of the code. It's just an illusion.
Python, because of it lack of static types and the fact that it is a great language made me come to this realization (made the light bulb turn on). I wonder just how many others out there had a similar experience as I have had. Could this be the reason some are strongly against adding static types to the language? Do others feel that it may be a disservice to all those programmers that haven't quite got the point that static types are not always the best protection or that they only provide solid protection in limited cases. I know this is not the only reason why some are against static types but it's the reason I would have against it.
For the record, I'm not 100% against adding static types. I'm just more against it than for it at this point in time. If they can be added in such a way that they are truly optional and I don't mean it in the sense that you don't have to use them but that the syntax is such that it somewhat discourages their use unless they are needed. Needed for reasons other than for protection like being used for optimization in cases where performance is an issue.
You may ask what is syntax that discourages it's use unless it is needed. I believe decorators fit this bill. It may not seem like it at this point in time as everyone under the sun is trying to find ways to use and abuse decorators but this will change when decorators are no longer new toys. After all, many of use like to play with new toys. In time though only the usefully uses of decorators will prevail.
Unfortunately I'm not convinced that adding static types will follow this pattern. If the syntax of static types are not enough of an inconvenience to use they will likely be used and abused forever. This is why I believe it may be a disservice to add static types.
Now, are there times that I wished Python had static types. Sure. Not for their false sense of protection but for their compact form of documentation or the intent that they show (what a variable/function/method is for or how it is to be used). I miss that that kind of documentation/intent is missing from Python. It's a good thing that Python code tends to be very clean and compact so normally you can quickly skim the code to get that same level of understanding that you would get from types. It's just that it is more work than would be require if types exist. Not that I believe that this is a good reason to add types.
I believe that there is something fundamentally missing from Python that would take care of this issue but I have no idea what it is. Some would say that's what doc strings are for but I'm not convinced that the lack of a syntax to address such an issue is the best solution. Doc strings can certainly play this role but if everyone uses a different format I'm not sure it's best for the common good.
I want to see this intent in the code I write so bad that I find myself abusing __slots__. I add __slots__ to, far too many of my classes. I even add them when I know they will cause me to write more code as I feel that the intent they show out weighs the cost of adding them. I wonder just how many others do the same?
> Four years ago when I was a Python newbie I would have > argued strongly for static types. [...]
Nice post John. You might want to subscribe to the python-dev list and follow the, um, lively PEP 246 discussion there.
Your admitted overuse of __slots__ is interesting. I've seen this in others too, though I almost never use them myself. It is similar to the "cast of characters" printed at the start of a play -- a helpful reference for the reader.
Maybe my "name: type = value" syntax could take over this role. It would be nice if we could also work a docstring in that syntax, but the proposal done earlier here (put the docstring on the next line) is too ambiguous to work, and neither does putting the docstring on the same line. Bonus points to make the same syntax work for arguments.
> > Maybe my "name: type = value" syntax could take over this > role. It would be nice if we could also work a docstring > in that syntax, but the proposal done earlier here (put > the docstring on the next line) is too ambiguous to work, > and neither does putting the docstring on the same line. > Bonus points to make the same syntax work for arguments.
I'm going to assume that for attributes x:protocol is syntactic sugar for x=typed_prop(protocol), where:
I just would like to add that I like how this whole topic morphed from static types to something much more useful, and that my own opinion about types has changed drastically over the last years when I was using Python.
As John, I thought types were something very valuable, and it would be great to add static typing to Python, especially since I'm so keen for optimization, but also because certain errors are known at compile time. I still remember those Python 1.6 discussions.
Meanwhile I learned that this is nonsense.
Speed: There are better ways to get speed than static typing.
Correctness: I agree completely, this is an illusion. The "no errors encountered" during compilation means just that you didn't make some silly errors, and that you followed the rules that the language forced you to obey. But these things are imposed by the language which tells me it needs (correct) types. But this helps me almost None in writing good code.
Well, I don't want to repeat things others have said before, this is just to express my happiness how this whole thing has evolved. When the Blog started, I was really in fear of what this would do to Python. This is gone, now.
I finally got around to reading all Guido's blogs and the replies. Here's the sum of my comments. Some apply primarily to the second blog, but they should all be fairly relevant.
Simplicity - Motivation:
Very good summary. The timing of Oliver Steele's blog entry is a very funny coincidence. I also think a retrospective on adding bool to Python is in order. There was a huge outcry about how bool was going to ruin Python (yes, I'm exaggerating, but only a bit). Many brought up dire consequences of how there would be all kinds of compatibility problems. While there are always compatibility issues with changing versions, I have not heard about any major issues. Nearly all of the concern in change was misplaced. I believe adding "Optional Static Typing" will be more of the same. Certainly, this change would be much deeper and will lead to more changes in Python software. However, many of the fears are overly cautious.
I think there are two major hurdles for people to get past when considering "Optional Static Typing". I've grown to dislike this term because of the animosity and assumptions that go along with it. This is also not exactly how I think about it.
I would prefer to call it "Conformance Checking". (It would still be optional.) I think this is a more descriptive term than "duck typing". My view is that conformance checking is primarily about checking an interface, rather than checking a type. If I declare a function:
def foo(x: int) -> float: ...
I am not thinking about int and float as concrete types, but rather more like abstract interfaces. int and float are merely a short-hand, concise, and convenient way to name the concept (interface) that I mean.
I have not used Eiffel, nor do I know a great deal about Design By Contract. However, I view adding type information as a very specific and terse use of DbC, basically a limited short-hand. For a more complete approach to using DbC, decorators can be employed. This may not be maximally optimal, but should be good enough until we can learn how to better implement DbC in Python. (Heh, I wrote this last sentence before reading Guido's third post.)
I wonder if we should use function attributes for pre/post conditions. This might lead to better re-use. Either way, there is the problem of choosing magical names.
I'm wondering if we should have a list of potential keywords in the future. We seem to want this often enough. This would really just be for guidance, although we could enforce it (with a warning).
The two words I recall are: as, interface. But I know there were others.
I like the | and & proposals in "Unions and Such". The syntax reminds me of set notation (even if they are really C operators) which is how I think about it. I'm not sure I like * though. I don't really like the idea of using words like intersection or cartesian for the same reason, they are too verbose.
On the other hand, | and & are very C-ish to me. Perhaps we should not try to borrow from it. But if not C, then what?
I wonder if it might not be better to require a command line switch to allow type declaration syntax? This would add a hurdle for newbies who try to use types by default.
def pseudomedian(seq: sequence) -> any: if not seq: raise ValueError, "median of empty sequence not defined" index = medianindex(seq) return seq[index]
Now, this code works when adapt() is implicitly called, because of the implicit conversion. But if adapt() isn't called, then suddenly we get "TypeError: list indices must be integers"
Some reasons that adapt() may not be called are, 1. -O switch or some other switch that elides run-time type-checking 2. back-porting code to older pythons (perhaps mechanically)
IMO, it would be better to call a type-checking function rather than a type-coercion function -- one that can only raise TypeErrors
With type "adapting" there is no TypeError in my example; with type "checking" the TypeError is raised in correct spot (the return statement of medianindex).
Automatic coercion has risks. Being able to know that working programs will work without type checking is a nice property. For both of those reasons, it seems to me that the conservative approach would be to type check only.
def foo(x, y): # wrapper function typecheck(x, t1) typecheck(y, t2) r = foo__(x, y) typecheck(r, t3) return r
Here is a little proposal that would give implementors the choice. Build the (nice) type annotation syntax but don't hard-code the run-time semantics. Instead, put those semantics into a standard decorator. Allow module implementors to define a per-module decorator using __decorator__. Have __builtin__.__decorator__ default to the standard checking decorator. This would let adaption frameworks get the implicit adaption behavior that they want.
> I think the expression-based proposals are too limited: > they don't handle guards involving multiple arguments > very well, and the proposed overloading of type > expressions and boolean guards feels error-prone > (what if I make my guard 'True' while awaiting > inspiration for something better?). Also, there > are clear use cases for guards that (in Python) > can only be expressed using multiple statements.
Why can't you just farm complex guards out to full functions? It seems to me that a guard which requires multiple statements should be broken out and named for readability's sake. Simple example:
def nz(x): return (x != 0)
def gcd(a, b): -> a: int and b: int and nz(a) and nz(b) <- (output) output: int and nz(output) '''Euclid's Greatest Common Divisor.'''
while a: a, b = b%a, a return b
Doing so would probably be a show-stopper for pychecker's ability to benefit from _those portions_ of the guard expressions, but ISTM that this would be the case in any statement-based syntax. By using an expression-based syntax, we could at least enable partial pychecker support. In the many, many cases in which we do *not* need multiple statements, we could even have a sys.enable_guards flag, which (when turned off) would skip the guards at runtime, leaving the check to be performed by a (possibly built-in) pychecker.
As a side note, I prefer the explicit binding of the output value (as I've done above) when asserting the postconditions. But then, arg-checking is already orthogonal to getting "x: int" into the language, IMO. ;)
Having both written and read large Python programs I do think that it would be very helpful to be able to "bind a type restriction" to a variable. It certainly helps the reader of a function absolutely know that (say) a list is expected (or returned). The same can be said for local variables.
If this could be integrated with PyChecker that would be even better.
One question, I would expect the following to fail:
def foo(int: i): i = "junk"
because "i" can not be adapted to an int. In my view the type adaptations applies to all assignments in to that variable.
For the syntax, instead of: def foo(int: t1, str: t2) -> list: ...body...
could you not support: def foo(int x, str y) returns list: ...body...
you could also support typed variable creation/assignment:
int i = 1 for int ino in items: etc.
This seems to me this would be more in keeping with Guido's comment from his previous posting re syntax "Python's parser generator is so lame, but that in turn is intentional -- it is so lame to prevent me from inventing syntax that is either hard to write a parser for or hard to disambiguate by human readers, who always come first in Python's design".
Guido, I read all your posts about the static type thing and concluded that making it optional is the way to go. But i had learned a python principle that says "there's only one way to do it!" and liked because it makes my code more readable. Im not trying to stick in radical way with it, but i think we have some alternatives to evaluate.
As you said, interfaces is the only place where a new syntax is needed. So why do you put all that "new features" (types and contracts) in the interface? So you have a good backward compatible source code, and "only one way" to use these new features?
Interfaces could still have implementation body... this way you can add a lot of opportunities to new styles of programming, like pre/post condition:
interface I1(I2, I3): def foo(a: t1, b: t2) -> t3: "docstring" assert a > 0 assert b > a assert Do() > b # Where Do() calls the wrapped function!
class C(I1): # implements I1 def foo(a, b): return a+b
How about just putting constraints on names in the name space using an "as" keyword?
syntax: name [as 'type'] = object
The "as 'type'" statement would have the meaning of:
"From now on, this name can only point to objects of this type."
For single varibles:
a as 'int' = 1 b as 'float' = 3.5
To take a type constraint off:
a as 'none' = 'mars' # can be assigned to anything.
Or to change it to another type:
a as 'int' = 5 a as 'float' = 23.5
It wouldn't make sense to change a name to a different type without pointing it at an object of that type. To do so should generate an error.
a as 'string' = '123' a as 'int' = a.int()
For functions, we primerily need types to check for errors, we really only need to put constraints on varables in the function some place.
def gcd(a, b): a as 'int' = a b as 'int' = b while a: a, b = b%a, a return b
This function of course will only return an int, and will generate errors if it recieves other than ints for arguments.
To check the return value we only need to:
z as 'int' = acd(u,v)
Now, after saying this, I have no idea how difficult it would be to add types constraints to names in names space. But this seems "to me" from a average python programmer's point of view to be simple, easy to understand, easy to read, consistant with the rest of python, and doesn't crate any compatablility problems that I can think of.
Not sure if there would be a big performance hit.. or maybe gain, compaired with other methods. I've heard that doing things in name space is one of the things that makes python as fast as it is.
> How about just putting constraints on names in the name > space using an "as" keyword? > > syntax: > name [as 'type'] = object > > The "as 'type'" statement would have the meaning of: > > "From now on, this name can only point to objects of > this type."
Change all references to 'as' to 'of' in my previous post. I didn't remember 'as' was used else where.
It's the concept I'm sugesting anyways, not the specific word 'as'.
syntax: name [of 'type'] = object
x of 'int' = 123 y of 'float' = 2.5
The use of 'of' implys the name belongs to a set of 'type'. That might be a useful way to look at it.
It still keeps the readability that made 'as' attractive.
Flat View: This topic has 79 replies
on 6 pages