The Artima Developer Community
Sponsored Link

Weblogs Forum
Optional Static Typing -- Stop the Flames!

79 replies on 6 pages. Most recent reply: Feb 4, 2008 9:52 AM by Wolfgang Lipp

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 79 replies on 6 pages [ « | 1 2 3 4 5 6 | » ]
Charles Hixson

Posts: 1
Nickname: charlesh
Registered: Jan, 2005

Re: Optional Static Typing -- Stop the Flames! Posted: Jan 7, 2005 6:45 PM
Reply to this message Reply
Advertisement
+1 on optional static typing (as shown)

I do hope that eventually Python and Pyrex could combine into a single project...and that optimization could be done smoothly in various degrees as needed.

But +1 anyway, even if that's not possible.


+1 on argument declarations also, with the same considerations. I hope it will improve efficiency, I expect it will improve error checking. I see minimal downsides (as long as it's optional).


==0 on Interfaces. I really have no opinion, and can't see why I would use them.


Design by Contract is a really good idea, but I'm not sure what the implementation should be. And I'm not totally certain how the basic idea translates from Eiffel into Python.

Roeland Rengelink

Posts: 5
Nickname: rengelink
Registered: Jan, 2005

Re: Optional Static Typing -- Stop the Flames! Posted: Jan 8, 2005 12:13 AM
Reply to this message Reply
I certainly like the wording of current proposal better than the original one. Although I'm not sure that they are in fact different ;).

Maybe the problem is in the examples. Integers are really not very interesting as interfaces (and very interesting as optimization hints). Hence, your gcd example seemed to be about compile-time type-checks and optimization, rather than documentation and/or adaptation.



I have a couple of questions/suggestions.


1. I would expect the following invariant:

x.__conform__(T) is T.__adapt__(x.__conform__(T))

That is, if I (the function writer) say that x needs to be adaptable to T, can x lie about that? Is Python going to enforce the invariant?


2. I was wondering, if this would be more correct:

def f(x:T1) -> T2

is equivalent to:

def f(x):
x = adapt(x, T1)
r = f__(x)
assert adaptable(r, T2)
return r

I would be very surprised if I saw a function definition that claimed to return a T2, but in fact returned something that only became a T2 after adaptation to T2.


3 Actually, I would expect a function definition to be very concrete about its return type. For example: This would surprise me very much:

def weird(x: list) -> sequence:

This, on the other hand, nicely illustrates the difference between interface and type:

def not_weird(x: iterable) -> iterator

Should Python enforce that argument types are interfaces, while return types are concrete types? (in that case, replace 'adaptable' with 'isinstance' in my previous question/suggestion)


4. Type unions (T1|T2) seem to be out. However, I often return either a T or None from my function. How would I deal with that? Also, I often return tuples. Is (T1, T2) still in? How does adapt(x, (T1, T2)) work?


5. About pre and post conditions. How about using interfaces for pre-conditions? E.g:

interface posint(Int):
"""A positive integer"""
def __adapt__(self, x):
r = Int.__adapt__(x)
assert r>=0
return r

def f(x: posint):
# adaptation now asserts that x>=0

In your current proposal, where the return value is adapted to the return type, this would also work for post-conditions. However, as I noted previously, I think return types should be concrete types, and should be checked with isinstance(), rather than adapt(). In fact, I think there is a nice asymmetry between pre-and post conditions, that reflects the asymmetry between interfaces for arguments and concrete types for return types.

I.e, rather than:

interface Ipos_int_list(List):
def __adapt__(self, x):
r = List.__adapt__(x)
for i in r:
assert isinstance(i, int) and i>0

def f()->Ipos_int_list # adapt

I'd write

class pos_int_list(list):
def append(self, item):
assert isinstance(item, int) and item>0
list.append(self, item)
# other methods

def f()->pos_int_list # isinstance

and get the errors when they occur, rather than in a post-condition check. In other words, forget about post-conditions, and let interfaces deal with pre-conditions.

Guido van van Rossum

Posts: 359
Nickname: guido
Registered: Apr, 2003

Re: Roeland Rengelink Posted: Jan 8, 2005 10:49 AM
Reply to this message Reply
Roeland Rengelink: "x.__conform__(T) is T.__adapt__(x.__conform__(T))"

One would hope so, otherwise this should be classified as a bug in the __adapt__ or __conform__ implementations. I don't think Python can ensure this.

"I would be very surprised if I saw a function definition that claimed to return a T2, but in fact returned something that only became a T2 after adaptation to T2."

To the contrary. Remember that T2 may be specified by the interface. The implementation may return something suitable and can leave the adaptation to the desired type to the framework. In C or Java, would you be surprised to see "return 0;" in a function declared to return a float?

"I would expect a function definition to be very concrete about its return type."

Again, not at all. If I can write a function that takes a sequence (any sequence at all), why couldn't I write code that receives a sequence from something it calls? Why not allow conforming implementations of an interface to return either a list or a tuple?

"Type unions (T1|T2) seem to be out."

Good point. Perhaps adapt(None, T) should return None for all T? Or perhaps this should be up to T? I haven't thought much about this; perhaps someone who has used adaptation in Zope or Twisted can comment?

"How about using interfaces for pre-conditions?"

Sounds messy, but this can all be done by the interface implementer without my approval. I doubt we'll have standard types to represent lists of positive ints, though.

Roeland Rengelink

Posts: 5
Nickname: rengelink
Registered: Jan, 2005

Re: Roeland Rengelink Posted: Jan 8, 2005 2:05 PM
Reply to this message Reply
>
> "I would be very surprised if I saw a function
> definition that claimed to return a T2, but in fact
> returned something that only became a T2 after adaptation
> to T2."

>
> To the contrary. Remember that T2 may be specified by the
> interface. The implementation may return something
> suitable and can leave the adaptation to the desired type
> to the framework. In C or Java, would you be surprised to
> see "return 0;" in a function declared to return a float?
>

I wouldn't be surprised, but that's why we call C and Java weakly typed. I haven't been thinking of adaptation as casting yet. I'm not sure that I want to ;)

You're right if Python produced an error here:

def f()->float:
return 0

I would be pissed. Although that's also because Python already 'casts' ints where I think it should. (Note that this wouldn't be an error if int is a subclass of float)

And I even know I'm inconsistent, because what I'm saying is that I would be pissed if this didn't produce an error:

def f()->list:
return ()


> "I would expect a function definition to be very
> concrete about its return type."

>
> Again, not at all. If I can write a function that takes a
> sequence (any sequence at all), why couldn't I write code
> that receives a sequence from something it calls?

Sure, but f()->list allows you to use f() as a sequence. Saying that the return type of f should be concrete, doesn't restrict the user of f(). It does restrict the implementor of f. I think that's good.

Suppose that, as a user, I want to sort the result of f, and that f returns a sequence. I now have to
write:

x = f()
x = adapt(x, list) # or adapt(x, sortable)?
x.sort()

This wouldn't be necessary if f just said that it returned a list.

My point would be that the default 'user' of an interface is someone who uses a concrete implementation, not someone who writes a concrete implementation. And the interface should cater to the default user. I must admit that in the case of frameworks, this may actually be the other way around. Well, mabye that is the point that I've been missing.

> Why not
> allow conforming implementations of an interface to return
> either a list or a tuple?
>

Probably because I think of the interface definition first as a promise by the original implementor, not as a requirement on the writer of a conforming implementation.

Moreover, to parafrase what I said before, I think that you can ask the writer of conforming implementation, to be liberal about what he accepts and strict about what he produces too.

> "Type unions (T1|T2) seem to be out."
>
> Good point. Perhaps adapt(None, T) should return None for
> all T? Or perhaps this should be up to T? I haven't
> thought much about this; perhaps someone who has used
> adaptation in Zope or Twisted can comment?
>

Hmm. That would mean that the writer of a conforming implementation of f(x:T) should accept None as an input. Would you really want that in the case of f(x:int)?

Probably easiest to change youw wrapper to:

def f(x, y):
x = adapt(x, T1)
y = adapt(x, T2)
res = f__(x, y)
if res is not None:
res = adapt(res, T3)
return res


On the other hand it could actually be useful to write

def f(x:int=None):


Thanks for your reply. This is interesting.

Bruce Eckel

Posts: 875
Nickname: beckel
Registered: Jun, 2003

Re: Optional Static Typing -- Stop the Flames! Posted: Jan 8, 2005 3:16 PM
Reply to this message Reply
A small brainstorm on coverage: would it make sense for all objects (classes, functions) to have some standard method, say __tests__(), or perhaps a property __tests__[] that would be respectively overridden or populated with tests. To run the tests you find all the objects in your system and call the functions, and to discover coverage, you find out how many objects have overloaded/populated the __tests__ thing.

Each object could then be analyzed to see whether all of its code paths are exercised via its __tests__, so instead of trying to discover this using some complex tool that looks at all the code at once, it's done at a small level of granularity.

This or something like it might go a long way to alleviate the concern of unexercised code paths, but using dynamic tests rather than static ones.

nes

Posts: 137
Nickname: nn
Registered: Jul, 2004

Did I understand correctly? Posted: Jan 8, 2005 4:06 PM
Reply to this message Reply
Just seeing if I understood correctly.
We are talking about something like the following?

Argument and Return Type Declarations

.......considering......


def foo(x,y):
x=str(x)
y=str(y)
...body...
return str(r)

(x and y have a __str__ method)


........analogy to............


def foo(x,y):
x=t1(x)
y=t2(y)
...body...
return t3(r)


......and object x has...

def __t1__(self):
if cannot_convert: raise adaptation_error
return t1(...some stuff here...)


.....and object y has...

def __t2___(self):
if cannot_convert: raise adaptation_error
return t2(...something here...)

...etc..same with r....
I asume subclasses will automatically converted to their superclasses?


Attribute Declarations

class C:
def __init__(self,p):
x=t1(p)

..etc...

....We will have a lot of fun once we get overloaded methods:

def foo(x:t1):
...body...

def foo(y:t2):
...body...

r=foo(z:t3)


....where z can be converted either to t1 or t2.
....(The compiler will have fun choosing :-))

Interfaces

def bar(x:I1)->I1:
return I1()

r=bar(C())


....what kind of object will x and r be after conversion?

Design by Contract

We are forgetting invariants. Those are assertions that get checked after each method or attribute access to the object. They ensure that the object is always kept in a sane state.

e.g.

Class Person:
def __init__(self):
self.age=Int()
self.children=List()
Invariants:
if self.age<9:
assert len(self.children)=0


Related note:
I would really like to have pychecker in its current form integrated into python, maybe with a python -warn option. It is the first package I download after python itself. This stuff is lower priority for me.

Nestor

Phillip J. Eby

Posts: 28
Nickname: pje
Registered: Dec, 2004

Re: Roeland Rengelink Posted: Jan 9, 2005 12:14 AM
Reply to this message Reply
> Suppose that, as a user, I want to sort the result of
> f, and that f returns a sequence. I now have
> to
> write:
>

> x = f()
> x = adapt(x, list) # or adapt(x, sortable)?
> x.sort()
>

> This wouldn't be necessary if f just said that it returned
> a list.

Actually, 'x = sorted(f())' should work just fine, since 'sorted()' takes an iterable now.

Even if sorted didn't exist, it would suffice to say 'x=list(f()); x.sort()' anyway.


> My point would be that the default 'user' of an interface
> is someone who uses a concrete implementation, not someone
> who writes a concrete implementation. And the interface
> should cater to the default user. I must admit that in the
> case of frameworks, this may actually be the other way
> around.

Both kinds of user occur quite a bit, but for a framework one of the big reasons for having an interface is to document what the framework expects of an implementation created by the user. IOW, the user of an interface is quite frequently the author of a concrete implementation.



> Probably because I think of the interface definition first
> as a promise by the original implementor, not as a
> requirement on the writer of a conforming implementation.

It's both. You can't really have an agreement unless both sides agree; this is what "programming by contract" means. :)


> Moreover, to parafrase what I said before, I think that
> you can ask the writer of conforming implementation, to be
> liberal about what he accepts and strict about what he
> produces too.

In practice this isn't a good idea for the framework writer; an upgraded version of the framework may provide a different concrete class, or one of several concrete classes. It's better for the author to only guarantee as much as the client needs, and that's an interface.


> > "Type unions (T1|T2) seem to be out."
> >
> > Good point. Perhaps adapt(None, T) should return None
> for
> > all T? Or perhaps this should be up to T? I haven't
> > thought much about this; perhaps someone who has used
> > adaptation in Zope or Twisted can comment?
> >
>
> Hmm. That would mean that the writer of a conforming
> implementation of f(x:T) should accept None
> as an input. Would you really want that in the case of
> f(x:int)?

Returning None from __adapt__ and __conform__ means that there is *no* adaptation. The way the current PEP 246 adaptation scheme works, there's no way to interpret 'None' as being a valid output of *any* adaptation other than 'adapt(None,type(None))' and 'adapt(None,object)'. If you wanted for 'None' to be considered valid, it would be necessary to have a change in the design of the adaptation protocol.


> On the other hand it could actually be useful to write
> def f(x:int=None):

If the adaptation protocol could be made to support None as a valid adaptation target, then it would probably make sense to have an 'optional' type, used as e.g. 'optional[int]' to do the equivalent of 'int|None' in the earlier proposal.

However, I have no idea how to allow None to be an adaptation target in a PEP 246-style protocol, without having to use exceptions for control flow (very bad in this case) or introducing another special singleton value like 'CantAdapt' or something like that (which breaks existing PEP 246-based code). I suppose we could raise a special exception to mean "No, I really mean to return None", but that seems really kludgy.

Nick Coghlan

Posts: 13
Nickname: ncoghlan
Registered: Dec, 2004

Adaptation and None Posted: Jan 9, 2005 2:37 AM
Reply to this message Reply
I certainly like the ability to 'fall off the end' of __adapt__ or __conform__ in order to tell the adaptation machinery "I got nuthin'".

Then raising an exception can be used to say, "Not only do I not implement this conversion, but I want you to abort the whole attempt completely".

The main use for "this type or None" is in declaring return types.

Without syntactic support, this can be done manually (by only doing the adaptation if the result is not None).

Once a syntax goes in, I think the issue is better addressed by having the ability to flag "may be None" in the declaration of the return type.

e.g.:
def f() -> int or None:
pass

Roeland Rengelink

Posts: 5
Nickname: rengelink
Registered: Jan, 2005

Re: Roeland Rengelink Posted: Jan 9, 2005 7:35 AM
Reply to this message Reply
> > Suppose that, as a user, I want to sort the result of
> > f, and that f returns a sequence. I now
> > have to write:
> >

> > x = f()
> > x = adapt(x, list) # or adapt(x, sortable)?
> > x.sort()
> >

> > This wouldn't be necessary if f just said that it
> > returned a list.
>
> Actually, 'x = sorted(f())' should work just fine, since
> 'sorted()' takes an iterable now.
>

s/sort/append

> Even if sorted didn't exist, it would suffice to say
> 'x=list(f()); x.sort()' anyway.
>

Ahh, but that's my point. This would be a bad solution if f() did return a list, because then list(f()) is an unnecessary (potentially expensive) operation. So, you'd have to use adapt(f(), list), which just returns the result of f() if f() did indeed produce a list.

>
> > My point would be that the default 'user' of an
> > interface is someone who uses a concrete
> > implementation, not someone who writes a concrete
> > implementation. And the interface
> > should cater to the default user. I must admit that in
> > the case of frameworks, this may actually be the other
> > way around.
>
> Both kinds of user occur quite a bit, but for a framework
> one of the big reasons for having an interface is to
> document what the framework expects of an implementation
> created by the user. IOW, the user of an interface is
> quite frequently the author of a concrete implementation.
>

Let's make this distinction:

- The default user of a library is someone who would call f

- The default user of a framework is someone who would implement f

I.e., libraries and frameworks are distinguished by the way in which a user would use the interface.

I wrote my original comment as a library user. And I do think that a library interface shouldn't be vague about what it returns. Moreover, even if the library is not vague about what it returns, as a reader of library code, I wouldn't like to see something like:

def f()->list:
return ()

that return an empty list due to the adapatation magic. Hence, my only suggestion to Guido was to remove the adaptation step on the return value and replace it with an assertion

Note that when I write 'list' as the return type, I mean the list interface (List?), i.e.: an interface with 1 concrete implementation (possibly sub-classed).

I'm thinking of a new built-in conforms(obj, interface), where conforms(x, T) returns True iff adapt(x, T) would return x. And my suggestion would be to replace

def f(x:T1, y:T2)-> T2:

with

def f(x, t):
x=adapt(x, T1)
y=adapt(x, T2)
r = f__(x, y)
assert conforms(r, T3)
return r


This would still allow one to write

def f()->sequence:
return ()

but wouldn't allow:

def f()->List:
return ()

This doen't hurt the framework writers, but keeps the library writers honest.

>
> > Probably because I think of the interface definition
> > first as a promise by the original implementor, not as
> > a requirement on the writer of a conforming
> > implementation.
>
> It's both. You can't really have an agreement unless both
> sides agree; this is what "programming by contract" means.
> :)
>

But it's library writers and framework writers who have to agree with their respective users. They don't necessarily have to agree with each other (except about the implementation of Guido's wrapper)

Edward C. Jones

Posts: 3
Nickname: edcjones
Registered: Dec, 2004

Which user communities for Python 3/3000? Posted: Jan 9, 2005 9:03 AM
Reply to this message Reply
I think that the design of Python 3/3000 should start with user needs.

What types of applications should Python 3/3000 be designed for? Teaching, scripting, wrapping, algorithm development, ...?

What can be added to Python to make writing these applications easier?

But please keep Python easy to read and write.

Edward C. Jones

Posts: 3
Nickname: edcjones
Registered: Dec, 2004

Which user communities for Python 3/3000? Posted: Jan 9, 2005 9:04 AM
Reply to this message Reply
I think that the design of Python 3/3000 should start with user needs.

What types of applications should Python 3/3000 be designed for? Teaching, scripting, wrapping, algorithm development, ...?

What can be added to Python to make writing these applications easier?

But please keep Python easy to read and write.

Guido van van Rossum

Posts: 359
Nickname: guido
Registered: Apr, 2003

Re: Roeland Rengelink Posted: Jan 9, 2005 9:58 AM
Reply to this message Reply
"And I do think that a library interface shouldn't be vague about what it returns."

This isn't a discussion for here. The proposed syntax and semantics will let library and interface writers do what they want or need; if you need to be vague about a return type or you need to be concrete about an input type, Python won't stop you. What's best for a given situation depends; we shouldn't (in this stage, without much more experience) try to legislate this.

Guido van van Rossum

Posts: 359
Nickname: guido
Registered: Apr, 2003

Re: Adaptation and None Posted: Jan 9, 2005 10:05 AM
Reply to this message Reply
"Once a syntax goes in, I think the issue is better addressed by having the ability to flag "may be None" in the declaration of the return type."

I agree. Over the past year I've read and written a lot of Java code; one of the most pernicious problems we ran into was unexpected null pointers, mostly due to programmers being unclear (or changing their minds!) on which APIs accepted or could return null and which didn't.

As long as we're going to add interface declarations to Python, I think we should be explicit about whether null is okay, both for arguments and for return values.

Roeland Rengelink

Posts: 5
Nickname: rengelink
Registered: Jan, 2005

Re: Roeland Rengelink Posted: Jan 9, 2005 1:56 PM
Reply to this message Reply
> "And I do think that a library interface shouldn't be
> vague about what it returns."

>
> This isn't a discussion for here. The proposed syntax and
> semantics will let library and interface writers do what
> they want or need; if you need to be vague about a return
> type or you need to be concrete about an input type,
> Python won't stop you. What's best for a given situation
> depends; we shouldn't (in this stage, without much more
> experience) try to legislate this.

Don't worrq. I'll quit about this one. If only because you (and Philip Elby) have actually convinced me ;)

Now, the other thing you were discussing, was maybe adding attribute typing as in:

class C:
x:int = 3

as syntactic sugar for

class C:
x = typedAttribute(int, 3)

which can currently be implemented with properties. Since this is an idiom I use occasionally in the following form:

class C:
x = typesAttribute(int, 3, "a nice doc string")

I was wondering if you had considered attribute docstrings. Would this work?

class C:
x:int=3
"""a nice doc string"""

Jaroslaw Zabiello

Posts: 1
Nickname: zbiru
Registered: Jan, 2005

Re: Optional Static Typing -- Stop the Flames! Posted: Jan 9, 2005 4:35 PM
Reply to this message Reply
There is another static typed language which uses pythonic syntax: http://boo.codehaus.org/. Its syntax looks very cool and may be an inspiration for this debate.

Flat View: This topic has 79 replies on 6 pages [ « | 1  2  3  4  5  6 | » ]
Topic: Optional Static Typing -- Stop the Flames! Previous Topic   Next Topic Topic: Setting Up for Emulation

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use