The Artima Developer Community
Sponsored Link

Weblogs Forum
More Questions About Python 3000 Answered

22 replies. Most recent reply: Jun 24, 2008 9:10 AM by sumanta mukherjee

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a flat view of this topic  Flat View
Previous Topic   Next Topic
Threaded View: This topic has 22 replies on 1 page
Guido van van Rossum

Posts: 359
Nickname: guido
Registered: Apr, 2003

More Questions About Python 3000 Answered (View in Weblogs) Posted: Jul 31, 2007 10:44 AM
Reply to this message Reply
Summary
Last week I started a Python 3000 FAQ, and solicited additional questions. Here are some additional answers.

This is a sequal to last week's post. It's tempting to replace most Q/A pairs below with

Q. Will Python 3000 have feature X (which has not been proposed yet)?

A. No. The deadline for feature proposals (PEPs) was April 30, 2007.

but I figured it would be better to try to explain why various ideas (most of which aren't new) weren't proposed, or why they were rejected.


Q. Will implicit string concatenation be removed in Python 3000? (I.e., instead of ("a" "b") you'd have to write ("a" + "b").)

A. No. This was proposed in PEP 3126, but rejected.


Q. Will the binary API for strings be standardized in Python 3000? (Depending on a compile-time switch, Unicode strings use either a 2-byte wide or 4-byte wide representation.)

A. No, there are still compelling reasons to support 2 bytes in some cases and 4 bytes in others. Usually this is dealt with by compiling from source with the headers corresponding to the installed Python binary. If that doesn't work for you, and you really care about this, I recommend that you bring it up on the python-3000 mailing list, explaining your use case.


Q. Why isn't the GIL (Global Interpreter Lock) recursive?

A. Several reasons. Recursive locks are more expensive, and the GIL is acquired and released a lot. Python's thread package doesn't implement recursive locks in C (they are an add-on written in Python, see RLock in threading.py). Given the different thread APIs on different platforms it's important that the C code involved in threads is minimal. But perhaps the most important reason is that the GIL often gets released around I/O operations. Releasing only a single level of a recursive lock would not be correct here; one would have to release the underlying non-recursive lock and restore the recursion level after re-acquiring. This is all rather involved. A non-recursive lock is much easier to deal with.


Q. Will we be able to use statements in lambda in Python 3000?

A. No. The syntax (turning indentation back on inside an expression) would both awkward to implement and hard to read for humans. My recommendation is just to define a local (i.e., nested) function -- this has the same semantics as lambda without the syntactic restrictions. After all, this:

foo = lambda: whatever

is completely equivalent to this:

def foo(): return whatever

(except that the lambda doesn't remember its name).


Q. Will Python 3000 require tail call optimization?

A. No. The argument that this would be a "transparent" optimization is incorrect -- it leads to a coding style that essentially depends on tail call optimization, at which point the transparency is lost. (Otherwise, why bother asking to require it? :-) Also, tracebacks would become harder to read. Face reality -- Python is not a functional language. It works largely by side effects on mutable objects, and there is no opportunity for program transformation based on equivalent semantics.


Q. Will Python 3000 provide "real" private, protected and public?

A. No. Python will remain an "open kimono" language.


Q. Will Python 3000 support static typing?

A. Not as such. The language would turn into Java-without-braces. However, you can use "argument annotations" (PEP 3107) and write a decorator or metaclass to enforce argument types at run-time. I suppose it would also be possible to write an extension to pychecker or pylint that used annotations to check call signatures.


Q. Why doesn't str(c for c in X) return the string of concatenated c values?

A. Then, to be consistent, str(['a', 'b', 'c']) would have to return 'abc'. I don't think you want that. Also, what would str([1, 2, 3]) do? It's a grave problem if str() ever raises an exception because that means the argument cannot be printed: print calls str() on each of its arguments.


Re: More Questions About Python 3000 Answered Posted: Jul 31, 2007 11:24 AM
Reply to this message Reply
Posted by: Thomas    Posts: 4 / Nickname: tal197 / Registered: Sep, 2005
About the Unicode question, a little Googling reveals that it has been discussed on python-dev before, but without a conclusion:

"UCS2 vs. UCS4 matters because the two versions use and expose different C APIs and thus an extension written for UCS2 doesn't run with a Python built for UCS4 and vice-versa."

(from http://mail.python.org/pipermail/python-dev/2005-September/056837.html)

The question was about the interface, though. We don't care how it is stored internally, and converting between the two is rather trivial, I think. But it's very difficult to make a C binary that can link against different binary interfaces at runtime.

Our use case is simple: we want to combine Python and C code, because sometimes you need the extra speed of C, or its ability to integrate with other C libraries, or the ability to continue using existing C code while migrating code gradually to Python.

Since most of our users are not programmers, they don't typically have a C compiler or the Python headers, so only providing them with source code isn't very helpful.


Re: More Questions About Python 3000 Answered Posted: Jul 31, 2007 11:46 AM
Reply to this message Reply
Posted by: Guido van van Rossum    Posts: 359 / Nickname: guido / Registered: Apr, 2003
> The question was about the interface, though. We don't
> care how it is stored internally, and converting between
> the two is rather trivial, I think. But it's very
> difficult to make a C binary that can link against
> different binary interfaces at runtime.

This was done intentionally to prevent extension module compiled for 2-byte unicode to access the 4-byte unicode internals (or vice versa).

In Python 2.x, if you don't use any PyUnicode_* APIs, your C code should link with either version. Is that not your experience? In 3.0 this isn't much of an option because all strings are unicode. Perhaps we can do something different, but please do bring it up on the python-3000 list.

> Our use case is simple: we want to combine Python and C
> code, because sometimes you need the extra speed of C, or
> its ability to integrate with other C libraries, or the
> ability to continue using existing C code while migrating
> code gradually to Python.
>
> Since most of our users are not programmers, they don't
> typically have a C compiler or the Python headers, so only
> providing them with source code isn't very helpful.

Why don't you distribute a Python interpreter binary built with the right options? Depending on users having installed the correct Python version (especially if your users are not programmers) is asking for trouble.

Please do consider continuing this on python-3000@python.org.


Re: More Questions About Python 3000 Answered Posted: Jul 31, 2007 9:31 PM
Reply to this message Reply
Posted by: amit upadhyay    Posts: 2 / Nickname: n8d10021 / Registered: Jul, 2007
This is regarding function annotations. Given decorators so often rely on constructing a new function, and just pass *args, **kw to the original one, will it break annotation of the original function? Hope I am making myself clear.

def loggin_decorator(f):
print f, "called"
def _func(*args, **kw):
f(*args, **kw)
_func.__doc__ = f.__doc__
# etc
return _func

How will this work with annotations?

So often we do something like this in a constructor:

class Person(Model):
def __init__(self, *args, **kw):
super(Person, self).__init__(*args, **kw)
# do some person specific thing

What will happen to the annotation of Model.__init__. Does it mean in order to preserve the annotation of Model.__init__ I will have to duplicate them on my Person.__init__?

--
Amit Upadhyay
http://www.amitu.com/blog/


Re: More Questions About Python 3000 Answered Posted: Aug 1, 2007 5:01 PM
Reply to this message Reply
Posted by: Guido van van Rossum    Posts: 359 / Nickname: guido / Registered: Apr, 2003
> This is regarding function annotations. Given decorators
> so often rely on constructing a new function, and just
> pass *args, **kw to the original one, will it break
> annotation of the original function? Hope I am making
> myself clear.
>

> def loggin_decorator(f):
> print f, "called"
> def _func(*args, **kw):
> f(*args, **kw)
> _func.__doc__ = f.__doc__
> # etc
> return _func
>

> How will this work with annotations?

You can copy the annotations just like the docstring:

_func.__annotations__ = f.__annotations__.copy()

If _func's signature is a transformation of f's signature you can of course modify the copy appropriately.

> So often we do something like this in a constructor:
>

> class Person(Model):
> def __init__(self, *args, **kw):
> super(Person, self).__init__(*args, **kw)
> # do some person specific thing
>

> What will happen to the annotation of Model.__init__. Does
> it mean in order to preserve the annotation of
> Model.__init__ I will have to duplicate them on my
> Person.__init__?

No. In general the signature of a subclass __init__ doesn't have to match the signature of the base class __init__ so it would be a mistake to do something like this automatically; and deducing that the signatures are actually the same would require way too much understanding in the compiler.

For other methods than __init__ you could argue that the signature *should* be the same, but there are all sorts of situations where overriding methods actually have a different signature -- e.g. they could change the argument names, or add a new (optional) argument, or change the set of types they accept. So again, this is not done automatically.

Don't think of this as a weak form of static typing. It's more appropriate to think of it as per-argument docstrings.


Re: More Questions About Python 3000 Answered Posted: Aug 14, 2007 2:45 AM
Reply to this message Reply
Posted by: amit upadhyay    Posts: 2 / Nickname: n8d10021 / Registered: Jul, 2007
> You can copy the annotations just like the docstring:
>
> _func.__annotations__ = f.__annotations__.copy()

This takes care of my doubts. Thankyou.

> No. In general the signature of a subclass __init__
> doesn't have to match the signature of the base class
> __init__ so it would be a mistake to do something like
> this automatically; and deducing that the signatures are
> actually the same would require way too much understanding
> in the compiler.

That may be so, but what if there was a decorator may be to say that the signatures are same. You may be aware that, esp in GUI class hierarchies, we have a lot of override chains that just calles the super(SomeClass, self).__current_method__(*args, **kw). Having them kill annotations will defeat the purpose of annotation.

I guess there can be a few annotation related decorators:

* @preserves_signature. This will follow the super logic, and find the nearest super, and .copy() its annotation.

This might take as keyword arguments the new arguments added that were not there in the base method.


class BaseClass:
...

class NewClass(BaseClass):
@preserves_signature(new_var="annotation of new vars")
def __init__(self, new_var, *args, **kw):
self.new_var = new_var
super(NewClass, self).__init__(*args, **kw)


This will effectively copy the annotations from BaseClass.__init__ and add annotation for new_var, and apply to new __init__.

This leave the matter of changing annotation for return type. That too can be handled by similar means.

Is this what you think things will go? or is there still some gap in my understanding?


Blocks, or multi-line lambdas Posted: Aug 1, 2007 3:15 AM
Reply to this message Reply
Posted by: Bjørn Stabell    Posts: 29 / Nickname: bjoerns / Registered: Aug, 2003
I think Ruby's eating our lunch with respect to domain specific languages by offering blocks (and the ability to drop parentheses). Couldn't we do something like this:

..def mycommand(*args, **kw) cb:
......# this function requires a callback function called cb
......# do something with args and kw
......return cb() # call the callback

and to use it:

..mycommand("hello", name="test") do:
......# the body of this block will actually become a
......# method that is passed as an argument to mycommand

If you wanted to support arguments to callbacks you could add those after do, e.g.,

..mycommand("hello", name="test") do(arg): ...

It would be pretty similar to the way Ruby does it, no?


Re: Blocks, or multi-line lambdas Posted: Aug 1, 2007 5:03 PM
Reply to this message Reply
Posted by: Guido van van Rossum    Posts: 359 / Nickname: guido / Registered: Apr, 2003
> I think Ruby's eating our lunch with respect to domain
> specific languages by offering blocks (and the ability to
> drop parentheses).

I don't think Ruby-envy is the way to design a language. remember, Ruby doesn't have first-class functions! The languages are just different. Besides (as I mentioned at the top of the article) the deadline for new features (especially new syntax) is long past. Try again for Python 4000. ;-)


Re: Blocks, or multi-line lambdas Posted: Aug 2, 2007 7:58 AM
Reply to this message Reply
Posted by: Bjørn Stabell    Posts: 29 / Nickname: bjoerns / Registered: Aug, 2003
Sure, where's the PEP? :)


Re: Blocks, or multi-line lambdas Posted: Aug 2, 2007 8:45 AM
Reply to this message Reply
Posted by: Bjørn Stabell    Posts: 29 / Nickname: bjoerns / Registered: Aug, 2003
Btw, fun pun intended. I know it's too early. :)


List comprehensions -> generator expressions Posted: Aug 1, 2007 3:30 AM
Reply to this message Reply
Posted by: Bjørn Stabell    Posts: 29 / Nickname: bjoerns / Registered: Aug, 2003
I really love list comprehensions, but the (later introduced) generator expressions are much more useful and should be encouraged.

Unfortunately, list comprehensions got the best syntax using [], while generator expressions are using () which are already very overloaded as they are used in function calls etc.

Since you can easily "fake" list comprehensions by adding a list() around generator expressions, wouldn't it be better (since we can be incompatible) to give this (better) syntax to generator expressions and drop list comprehensions completely?

..list([ e for e in l ])

if you absolutely need list comprehensions.

My guess is that most people actually want generator expressions but they end up using list comprehensions since that's the most obvious syntax.


Re: More Questions About Python 3000 Answered Posted: Aug 1, 2007 6:47 AM
Reply to this message Reply
Posted by: Jeffery Reavis    Posts: 1 / Nickname: jjreavis / Registered: Aug, 2007
Bjørn, I agree that generator expressions are often more useful than list comprehensions. The problem is there is already a defined list literal syntax (eg L = [1,2,3]) in the language and list comprehensions have been used for awhile in Python (since 2.0). Changing the behavior might break a lot of code. I also like the ability to create a generator in a function call. For example:

s = set(word for line in page for word in line.split())

I believe that list comprehensions will actually be syntactic sugar for list(genexp) in Python 3000.

-jjr


Re: More Questions About Python 3000 Answered Posted: Aug 1, 2007 5:06 PM
Reply to this message Reply
Posted by: Guido van van Rossum    Posts: 359 / Nickname: guido / Registered: Apr, 2003
> Bjørn, I agree that generator expressions are often more
> useful than list comprehensions. The problem is there is
> already a defined list literal syntax (eg L = [1,2,3]) in
> the language and list comprehensions have been used for
> awhile in Python (since 2.0). Changing the behavior might
> break a lot of code. I also like the ability to create a
> generator in a function call. For example:
>
> s = set(word for line in page for word in
> line.split())

>
> I believe that list comprehensions will actually be
> syntactic sugar for list(genexp) in Python 3000.

Correct.

Also note that you don't need to parenthesize a generator expression when it's the sole argument to a function, so Bjørn's example list([ e for e in l ]) can actually be written list(e for e in l) already.


Re: More Questions About Python 3000 Answered Posted: Aug 2, 2007 8:23 AM
Reply to this message Reply
Posted by: Bjørn Stabell    Posts: 29 / Nickname: bjoerns / Registered: Aug, 2003
The point was that the common case should be for people to create genexps (in most cases it won't make a difference to the outcome of the algorithm, but will use less memory).

Not including [] or () around generator expressions or list comprehensions seems a bit hackish syntactically; at the very least it's another special case people have to learn.


Re: More Questions About Python 3000 Answered Posted: Aug 13, 2007 3:43 AM
Reply to this message Reply
Posted by: Petr Prikryl    Posts: 2 / Nickname: pepr / Registered: Aug, 2007
> > I believe that list comprehensions will actually be
> > syntactic sugar for list(genexp) in Python 3000.
>
> Correct.
>
> Also note that you don't need to parenthesize a generator
> expression when it's the sole argument to a function, so
> Bjørn's example list([ e for e in l ]) can
> actually be written list(e for e in l)
> already.

I like to hear that. I would like also to see the alternative name for the list comprehension in Python 3000. It looks strange to beginners and it is probably difficult to translate the term into other human languages (at least into the Czech one). I understand that it may be the part of Python folklore. On the other hand...

If double (or single) quote char is considered a string constructor in many languages, then [...] in Python (with the things inside) is a list constructor. If I want to have some special name for generator expression, then probably the alternative name could be the list generator.


Re: More Questions About Python 3000 Answered Posted: Aug 13, 2007 3:48 AM
Reply to this message Reply
Posted by: Petr Prikryl    Posts: 2 / Nickname: pepr / Registered: Aug, 2007
Correction of myself:

> [...] If I want to have some special name for generator
> expression
, then probably the alternative name could
> be the list generator.

...should be...

If I want to have some special name for [ generator expression ], then probably the alternative name could be the list generator.


Re: More Questions About Python 3000 Answered Posted: Aug 13, 2007 7:25 AM
Reply to this message Reply
Posted by: Guido van van Rossum    Posts: 359 / Nickname: guido / Registered: Apr, 2003
> (proposal to rename list comprehensions and generator expressions)


Actually, list comprehension is known in other languages too (e.g. Haskell) and was taken from mathematics. So I'm sure there is a translation into Polish, unless there is no math literature translated into Polish. :-)

The name "generator expression" was discussed endlessly when it was first introduced; I doubt that we'll be able to come up with a better name.


Re: More Questions About Python 3000 Answered Posted: Aug 1, 2007 10:45 AM
Reply to this message Reply
Posted by: Laurent Pointal    Posts: 1 / Nickname: pointal / Registered: Aug, 2007
For Stephane which have problem creating an artima account... (traduction from his french post in pythonfr mailing list - with my own english ;-):

Am-I the only one to deplore the reduce() function ?
I think its nice, it goes well with map()
(cf. http://labs.google.com/papers/mapreduce.html) and its a programming style to promote.

from Guido blog on http://www.artima.com/weblogs/viewpost.jsp?thread=211200

> Q. If you're killing reduce(), why are you keeping map() and filter()?
> A. I'm not killing reduce() because I hate functional programming; I'm
> killing it because almost all code using reduce() is less readable
> than the same thing written out using a for loop and an accumulator
> variable. On the other hand, map() and filter() are often useful and
> when used with a pre-existing function (e.g. a built-in) they are
> clearer than a list comprehension or generator expression. (Don't use
> these with a lambda though; then a list comprehension is clearer and
> faster.)


Re: More Questions About Python 3000 Answered Posted: Aug 1, 2007 5:13 PM
Reply to this message Reply
Posted by: Guido van van Rossum    Posts: 359 / Nickname: guido / Registered: Apr, 2003
> For Stephane which have problem creating an artima
> account... (traduction from his french post in pythonfr
> mailing list - with my own english ;-):

>
> Am-I the only one to deplore the reduce() function ?
> I think its nice, it goes well with map()
> (cf. http://labs.google.com/papers/mapreduce.html) and its
> a programming style to promote.

Not in Python. In function languages you don't have a way to update a variable on each pass though a loop, e.g.

total = 0
for x in lst:
total += x

So this must be written as the equivalent of

total = reduce(lambda x, y: x+y, lst)

However I have done an extensive survey of the use of reduce() in actual Python code, and found that most cases could easily be rewritten using the built-in sum() function or "".join(...), and those that couldn't were mostly completely unreadable. (For a particularly bad example, see http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/148061 ("one-liner word-wrap function"). It doesn't fit on a line and the algorithm used is quadratic. Apart from that, it's unreadable (and the fact that it was quadratic escaped most readers, including the Django developers who incorporated it into Django until I pointed it out).

So I have to disagree that reduce() is a style to promote, at least in Python, where a much more straightforward style is available.


A general question about Py3k motivation Posted: Aug 8, 2007 2:12 AM
Reply to this message Reply
Posted by: Grzegorz Staniak    Posts: 1 / Nickname: gstaniak / Registered: Aug, 2007
Hi,

Is Python 3000 actually worth the effort? I mean, voices from developers of high-profile projects (Zope, Twisted) can be heard saying they have no intention to port to py3k. This, together with the inconveniences of including two Pythons at the same time in a distribution, could mean a slow adoption rate and the status of, say, the Vista of Python releases. Was py3k necessary? I know, a stupid question to ask. But couldn't the goals of this release be achieved through the familiar cycle of "from __future__" imports, then going mainstream, then obsoleting old features? I suppose for syntax changes this could be tricky, if not impossible. But are the changes urgent enough to justify the risk taken?

Regards,
--
Grzegorz Staniak <gstaniak _at_ gmail *dot* com>


Re: A general question about Py3k motivation Posted: Aug 9, 2007 4:10 PM
Reply to this message Reply
Posted by: Larry Riedel    Posts: 2 / Nickname: larryr / Registered: Jan, 2003
Presuming the intra-process concurrency support in the most mainstream Python runtime(s) will be comparable to Java, and the use of function annotations will be ubiquitous, I think a /lot/ more Python code will be written in 3.x and beyond than was written before that, and any difficulty with the 2.x to 3.x transition will be little more than a historical footnote.


Larry


Re: A general question about Py3k motivation Posted: Aug 14, 2007 10:03 PM
Reply to this message Reply
Posted by: ian o    Posts: 8 / Nickname: iano / Registered: Aug, 2007
As i understand it, there is no strong motivation to move to python 3.0, that is not the point!
Whilst most releases focus on what can ADD- having a release without guaranteeing backward compatibility puts the focus on what to REMOVE.
So 3.0 is all about what should be removed to provide a better base to move forward. Which means the incentive will come in future releases when the improved base is actually built on.
Don't worry if people don't rush to move to 3.0, by 3.1 or 3.2 the reasons for moving forward should be more apparent!

Why move to 3.0? Just to get the deleted features deleted?

As i said, the job of 3.0 being to remove the deadwood that block moving forward means it is all about REMOVING things and not about adding. Expect 3.1 to have backlog of additions, with the only additions on 3.0 being cases where a new substitute is immediately required for the old. At 3.0 the 'emergency' substitutes will be in their first iteration, and will not doubt evolve.

It will take a while for the full wisdom of 3.0 to emerge, but the logic is sound and the logic says that 3.0 will not itself be the compelling version- but it will build a better base for future versions.

(where hopefully one day a 'where' clause will become part of the language :) )


Re: More Questions About Python 3000 Answered Posted: Jun 24, 2008 9:10 AM
Reply to this message Reply
Posted by: sumanta mukherjee    Posts: 1 / Nickname: sumanta / Registered: Jun, 2008
There is one query which I have tried and have not got any answers from anywhere and so I thought of asking here. I am not sure if this is the right forum.

Anyway, this is regarding of embedding of Python code in C++.

Is it possible to embed Python GUI code inside C++.

I tried out the following
At Python prompt.
import Tkinter
Tkinter._test().

This works correctly.

When I try to import the same module Tkinter from C++ code and then try to call the test function it crashes. Not sure why this could be the case.

Regards
SM.


Topic: More Questions About Python 3000 Answered Previous Topic   Next Topic Topic: Open Standards vs Open Source?

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use