The Artima Developer Community
Sponsored Link

Weblogs Forum
The fate of reduce() in Python 3000

117 replies on 8 pages. Most recent reply: Jul 15, 2010 8:30 AM by Jeffry Mcdowell

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 117 replies on 8 pages [ « | 1 2 3 4 5 6 7 8 | » ]
Michael Chermside

Posts: 17
Nickname: mcherm
Registered: Jan, 2004

Re: Syntax for "filter only" list comprehensions Posted: Mar 14, 2005 11:23 AM
Reply to this message Reply
Advertisement
I wrote:
Of course, I'm still expecting that a closer look will reveal that the parsing is ambiguous, or that it won't work for generator expressions, or some other impediment like that... but I haven't noticed what it is yet. Does anyone else see the lurking problem?

Thanks to the person (I forget who first showed me this) who found the fatal flaw:

If "x in some_list if condition" is a generator expression equivalent to "x for x in some_list if condition", then clearly "[x in some_list if condition]" needs to be an equivalent list comprehension.

But "[x in some_list]" already has a meaning (a one-element list containing either True or False, depending on whether the current value of the variable x is an element of some_list). Distinguishing these would require more than one token look-ahead, which is currently NOT ALLOWED in the Python grammer.

Too bad... it would have been cool if it had worked.

Christos Georgiou

Posts: 7
Nickname: tzotzioy
Registered: Mar, 2005

Re: Syntax for "filter only" list comprehensions Posted: Mar 15, 2005 4:01 AM
Reply to this message Reply
> If "x in some_list if condition" is a
> generator expression equivalent to "x for x in
> some_list if condition
", then clearly "[x in
> some_list if condition]
" needs to be an equivalent
> list comprehension.

> But "[x in some_list]" already has a meaning
> (a one-element list containing either True or
> False, depending on whether the current value
> of the variable x is an element of
> some_list). Distinguishing these would
> require more than one token look-ahead, which is currently
> NOT ALLOWED in the Python grammer.

Then let's not allow "<x> in <list>" since it's not needed anyway; the only valid syntax should be "<x> in <list> if <predicate>". Only one look-ahead token needed.

> Too bad... it would have been cool if it had worked.

It still could.

Nick Coghlan

Posts: 13
Nickname: ncoghlan
Registered: Dec, 2004

Re: Syntax for "filter only" list comprehensions Posted: Mar 15, 2005 6:05 AM
Reply to this message Reply
> Then let's not allow "<x> in <list>"
> since it's not needed anyway; the only valid syntax should
> be "<x> in <list> if <predicate>". Only one
> look-ahead token needed.

Err. . .
  found = x in container
Kind of hard to justify eliminating working code for this.

Christos Georgiou

Posts: 7
Nickname: tzotzioy
Registered: Mar, 2005

Re: Syntax for "filter only" list comprehensions Posted: Mar 15, 2005 6:43 AM
Reply to this message Reply
> > Then let's not allow "<x> in
> <list>
"
> > since it's not needed anyway; the only valid syntax
> should
> > be "<x> in <list> if <predicate>". Only
> one
> > look-ahead token needed.
>
> Err. . .
  found = x in container
Kind of hard to
> justify eliminating working code for this.

You are absolutely right, of course; I wasn't clear enough. Here's the whole story.

Michael had supported Ka Ping-Yee's syntax of "x in lst if pred" as a shortcut for "x for x in lst if pred", stating he felt something was wrong.

I replied that "x in lst" has a meaning already, so it depends on the parser if it can distinguish using one look-ahead symbol.

Michael summarized that since "x in lst" has a meaning, this invalidates the "x in lst if pred" syntax.

I replied that, with one look-ahead token, "x in lst if pred" can be valid, while "x in lst" as a generator comprehension is useless.

Felix Wiemann

Posts: 1
Nickname: felixw
Registered: Mar, 2005

Re: The fate of reduce() in Python 3000 Posted: Mar 15, 2005 2:43 PM
Reply to this message Reply
any() and all() are great ideas. +1

I'm not entirely sure about map(), because map is sometimes shorter, so I'm -0 on it.

Lambdas going away is fine, but they really, really should be replaced by something like inline functions. When I use lambdas in my code, it would usually be awkward to replace them with nested functions. As a side note, Ruby is quite cool in this regard, since it allows statements (not only expressions) in its "blocks" (= inline functions) -- even though Ruby's concepts are admittedly somewhat different from Python's. I'm sure many possibilities for inline function syntax have been discussed before, so I won't make any suggestions now. Just lambda going away without a substitute would be a pity.

Michael Chermside

Posts: 17
Nickname: mcherm
Registered: Jan, 2004

Re: Syntax for "filter only" list comprehensions Posted: Mar 16, 2005 9:45 AM
Reply to this message Reply
Christos Georgiou: Unfortunately, single token lookahead isn't sufficient to implement your proposal. For example, is the following snippet of code a containment test or a generator expression:

y = x in [1, 2, 3, .... and so on

Obviously I could complete it with either

... 4]

or with

... 4] if x % 2

And without looking ahead past the end of the list you don't know which one it is.

Joost de Vries

Posts: 19
Nickname: yoozd
Registered: May, 2003

Re: The fate of reduce() in Python 3000 Posted: Mar 16, 2005 11:54 PM
Reply to this message Reply
I like David Mertz' description of higher-order programming as described in "Text Processing in Python" http://gnosis.cx/TPiP/ and implemented in the gnosis.util.combinators package in his library http://gnosis.cx/download/. Definitely worth consideration.

Ka-Ping Yee

Posts: 24
Nickname: ping
Registered: Dec, 2004

Re: Syntax for "filter only" list comprehensions Posted: Mar 17, 2005 4:43 AM
Reply to this message Reply
> Christos Georgiou: Unfortunately, single token lookahead
> isn't sufficient to implement your proposal.

Yes, but then again, keep in mind we are talking about Python 3000 after all. It's not inconceivable that we might have a more flexible parser generator.

D H

Posts: 13
Nickname: dug
Registered: Jul, 2003

Re: The fate of reduce() in Python 3000 Posted: Mar 17, 2005 11:22 AM
Reply to this message Reply
Didn't you already state that "any" would have a different meaning in python 3? It would mean the equivalent of "Variant" in VB, or "var" in groovy, or "duck" in boo.

Also, yes, there is not much need for map and the other functions when lambda is so constrained as it is in python. Thus eliminating them as well as lambda is a good idea, since list comprehensions can serve the same purpose.

If python was to ever have full closures though like ruby or groovy or boo, then map (or "each") is more useful. You can do multi-line procedures for example that list comprehensions cannot handle. The only difference is boo had to switch the order of the parameters for "map" in boo, putting the function/closure last instead of first, to allow for multi-line closures.

Guido van van Rossum

Posts: 359
Nickname: guido
Registered: Apr, 2003

Re: The fate of reduce() in Python 3000 Posted: Mar 17, 2005 11:49 AM
Reply to this message Reply
> Didn't you already state that "any" would have a different
> meaning in python 3? It would mean the equivalent of
> "Variant" in VB, or "var" in groovy, or "duck" in boo.

We can always rename that to 'anything' for example.

> If python was to ever have full closures though like ruby
> or groovy or boo,

Um, Python doesn't have full closures? And what are nested functions other than closures? Or are we talking about a different type of closure? (The word seems to have a different meaning for different people.)

Stefan Plantikow

Posts: 1
Nickname: splant
Registered: Mar, 2005

Re: The fate of reduce() in Python 3000 Posted: Mar 17, 2005 8:22 PM
Reply to this message Reply
>
> You can try to implement category theory in Python and
> specify everything up to isomorphism. This could lead to a
> full flegded DSL or even better a complete computer
> algebra system ( possible without leaving the language? )
> I like this way of thinking but i suspect Your boss is
> merciless with those guys who differ to much from the
> average and though not being a masochist i have some
> comprehension for this point of view.

While surely there is truth in this, there are also a lot of people using python outside of such an context.
If I remember correctly, the language is said to come with batteries included. Please leave folks that like to think in terms of Y-operators and higher-order functions their toys, will you? Fixing lambda, adding all the missing higher-order functions and currying instead of dropping some of the better features of the language would be nice.

Having to do complex higher-order functional programming with inner functions and generator expressions only would be so terribly cumbersome.

Pasko Robert

Posts: 5
Nickname: pkr
Registered: Mar, 2005

Re: The fate of reduce() in Python 3000 Posted: Mar 18, 2005 5:00 AM
Reply to this message Reply
I think that these (reduce/lambda) should be viewed as two separate issues:

1. map/reduce/filter primitives, and
2. lambda.


If we take as a maxim that a feature should be provided once in the core language then there isn't any issue about the group 1 - list comprehensions do provide the feature, and if someone will miss the syntax, he can always define the corresponding functions himself.

Lambda, however, is different beast IMHO - because it allows you to define a closure without having to name it - that is something a nested function does not provide though they will both end up in a semantically equivalent closure object. And since python treats functions as first class objects (or perhaps 0.99 class objects? :-), it is a feature which has its justification. If you are allowed to write directly foo(X,Y) instead having to write

a, b = X, Y
foo(a,b)

for any object in python (OK, almost any), why should there be an exception for functions [well, there already is, as lambda does not provide everything a nested function does - but then removing lambda is not solving the problem either :-)]

Sure it can be used to write ugly code, but there isn't a single feature in any language, which could not be banned on that premise :-), and it is also possible to devise examples where anonymous functions will improve code clarity/readability.

Bram Cohen

Posts: 2
Nickname: bram
Registered: Mar, 2005

the proposed extensions and changes Posted: Mar 18, 2005 12:08 PM
Reply to this message Reply
I like the extension of sum() to work on non-integers, but think that breaking it for lists of length 0 would be very bad. I'd like to propose an alternate implementation:

def sum(mylist, total = 0):
for item in mylist:
total += item
return total

Then you can do sum(['these ', 'are ', 'words'], '')

I'm in favor of any() and all() being added as builtins.

I love the proposed [x in mylist if x > 0] style of syntax, and hope that it can be added. I already use [x for x in mylist if x > 0] because I can never remember the syntax of filter().

In my code there are only a few uses of lambda, mostly having to do with passing functions to graphical libraries. Defining the functions inline does improve clarity in these cases, but doing it with lambda feels like a hack, because usually they usually don't supply a meaningful return value. An inline def, using a reserved 'inline' keyword, which someone suggested in an earlier comment, would be much nicer.

I'm generally in favor of completely removing filter, map, reduce, and lambda, but have the caveats outlined above.

Nick Coghlan

Posts: 13
Nickname: ncoghlan
Registered: Dec, 2004

Re: the proposed extensions and changes Posted: Mar 18, 2005 6:20 PM
Reply to this message Reply
> def sum(mylist, total = 0):
> for item in mylist:
> total += item
> return total
> Then you can do sum(['these ', 'are ', 'words'], '')

That's basically the way sum works now (except that it uses total = total + item, and explicitly disallows strings as the initial value, pointing users towards ''.join() instead))

Something that might be interesting to add is operator.op_ident which would simply return the other operand for addition, multiplication and right division. Then a sequence of any type of object could be summed using:
  sum(seq, operator.op_ident)

Marcin Kowalczyk

Posts: 40
Nickname: qrczak
Registered: Oct, 2004

Re: the proposed extensions and changes Posted: Mar 19, 2005 7:56 AM
Reply to this message Reply
> explicitly disallows strings as the initial value, pointing users towards ''.join() instead)

Why does it point towards ''.join() instead of doing ''.join()?

Flat View: This topic has 117 replies on 8 pages [ « | 1  2  3  4  5  6  7  8 | » ]
Topic: Adding Optional Static Typing to Python Previous Topic   Next Topic Topic: Reviewable, Runnable Specifications Ensure Software Quality


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us