Weblogs Forum - The fate of reduce() in Python 3000

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
The fate of reduce() in Python 3000

119 replies on 8 pages. Most recent reply: Dec 18, 2018 1:39 AM by Claudemir Moliane

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 119 replies on 8 pages [ « | 1 2 3 4 5 6 7 8 | » ]

Michael Chermside

Posts: 17
Nickname: mcherm
Registered: Jan, 2004

Re: Syntax for "filter only" list comprehensions

Posted: Mar 14, 2005 8:23 AM

I wrote:
Of course, I'm still expecting that a closer look will reveal that the parsing is ambiguous, or that it won't work for generator expressions, or some other impediment like that... but I haven't noticed what it is yet. Does anyone else see the lurking problem?

Thanks to the person (I forget who first showed me this) who found the fatal flaw:

If "x in some_list if condition" is a generator expression equivalent to "x for x in some_list if condition", then clearly "[x in some_list if condition]" needs to be an equivalent list comprehension.

But "[x in some_list]" already has a meaning (a one-element list containing either True or False, depending on whether the current value of the variable x is an element of some_list). Distinguishing these would require more than one token look-ahead, which is currently NOT ALLOWED in the Python grammer.

Too bad... it would have been cool if it had worked.

Christos Georgiou

Posts: 7
Nickname: tzotzioy
Registered: Mar, 2005

Re: Syntax for "filter only" list comprehensions

Posted: Mar 15, 2005 1:01 AM

> If "x in some_list if condition" is a
> generator expression equivalent to "

x for x in
> some_list if condition

", then clearly "

[x in
> some_list if condition]

" needs to be an equivalent
> list comprehension.

> But "[x in some_list]" already has a meaning
> (a one-element list containing either True or
> False, depending on whether the current value
> of the variable x is an element of
> some_list). Distinguishing these would
> require more than one token look-ahead, which is currently
> NOT ALLOWED in the Python grammer.

Then let's not allow "<x> in <list>" since it's not needed anyway; the only valid syntax should be "<x> in <list> if <predicate>". Only one look-ahead token needed.

> Too bad... it would have been cool if it had worked.

It still could.

Nick Coghlan

Posts: 13
Nickname: ncoghlan
Registered: Dec, 2004

Re: Syntax for "filter only" list comprehensions

Posted: Mar 15, 2005 3:05 AM

> Then let's not allow "<x> in <list>"
> since it's not needed anyway; the only valid syntax should
> be "<x> in <list> if <predicate>". Only one
> look-ahead token needed.

Err. . .

  found = x in container

Kind of hard to justify eliminating working code for this.

Christos Georgiou

Posts: 7
Nickname: tzotzioy
Registered: Mar, 2005

Re: Syntax for "filter only" list comprehensions

Posted: Mar 15, 2005 3:43 AM

> > Then let's not allow "

<x> in
> <list>

"
> > since it's not needed anyway; the only valid syntax
> should
> > be "<x> in <list> if <predicate>". Only
> one
> > look-ahead token needed.
>
> Err. . .

  found = x in container

Kind of hard to
> justify eliminating working code for this.

You are absolutely right, of course; I wasn't clear enough. Here's the whole story.

Michael had supported Ka Ping-Yee's syntax of "x in lst if pred" as a shortcut for "x for x in lst if pred", stating he felt something was wrong.

I replied that "x in lst" has a meaning already, so it depends on the parser if it can distinguish using one look-ahead symbol.

Michael summarized that since "x in lst" has a meaning, this invalidates the "x in lst if pred" syntax.

I replied that, with one look-ahead token, "x in lst if pred" can be valid, while "x in lst" as a generator comprehension is useless.

Felix Wiemann

Posts: 1
Nickname: felixw
Registered: Mar, 2005

Re: The fate of reduce() in Python 3000

Posted: Mar 15, 2005 11:43 AM

any() and all() are great ideas. +1

I'm not entirely sure about map(), because map is sometimes shorter, so I'm -0 on it.

Lambdas going away is fine, but they really, really should be replaced by something like inline functions. When I use lambdas in my code, it would usually be awkward to replace them with nested functions. As a side note, Ruby is quite cool in this regard, since it allows statements (not only expressions) in its "blocks" (= inline functions) -- even though Ruby's concepts are admittedly somewhat different from Python's. I'm sure many possibilities for inline function syntax have been discussed before, so I won't make any suggestions now. Just lambda going away without a substitute would be a pity.

Michael Chermside

Posts: 17
Nickname: mcherm
Registered: Jan, 2004

Re: Syntax for "filter only" list comprehensions

Posted: Mar 16, 2005 6:45 AM

Christos Georgiou: Unfortunately, single token lookahead isn't sufficient to implement your proposal. For example, is the following snippet of code a containment test or a generator expression:


    y = x in [1, 2, 3,        .... and so on

Obviously I could complete it with either


                 ...   4]

or with


                 ...   4] if x % 2

And without looking ahead past the end of the list you don't know which one it is.

Joost de Vries

Posts: 19
Nickname: yoozd
Registered: May, 2003

Re: The fate of reduce() in Python 3000

Posted: Mar 16, 2005 8:54 PM

I like David Mertz' description of higher-order programming as described in "Text Processing in Python" http://gnosis.cx/TPiP/ and implemented in the gnosis.util.combinators package in his library http://gnosis.cx/download/. Definitely worth consideration.

Ka-Ping Yee

Posts: 24
Nickname: ping
Registered: Dec, 2004

Re: Syntax for "filter only" list comprehensions

Posted: Mar 17, 2005 1:43 AM

> Christos Georgiou: Unfortunately, single token lookahead
> isn't sufficient to implement your proposal.

Yes, but then again, keep in mind we are talking about Python 3000 after all. It's not inconceivable that we might have a more flexible parser generator.

D H

Posts: 13
Nickname: dug
Registered: Jul, 2003

Re: The fate of reduce() in Python 3000

Posted: Mar 17, 2005 8:22 AM

Didn't you already state that "any" would have a different meaning in python 3? It would mean the equivalent of "Variant" in VB, or "var" in groovy, or "duck" in boo.

Also, yes, there is not much need for map and the other functions when lambda is so constrained as it is in python. Thus eliminating them as well as lambda is a good idea, since list comprehensions can serve the same purpose.

If python was to ever have full closures though like ruby or groovy or boo, then map (or "each") is more useful. You can do multi-line procedures for example that list comprehensions cannot handle. The only difference is boo had to switch the order of the parameters for "map" in boo, putting the function/closure last instead of first, to allow for multi-line closures.

Guido van van Rossum

Posts: 359
Nickname: guido
Registered: Apr, 2003

Re: The fate of reduce() in Python 3000

Posted: Mar 17, 2005 8:49 AM

> Didn't you already state that "any" would have a different
> meaning in python 3? It would mean the equivalent of
> "Variant" in VB, or "var" in groovy, or "duck" in boo.

We can always rename that to 'anything' for example.

> If python was to ever have full closures though like ruby
> or groovy or boo,

Um, Python doesn't have full closures? And what are nested functions other than closures? Or are we talking about a different type of closure? (The word seems to have a different meaning for different people.)

Stefan Plantikow

Posts: 1
Nickname: splant
Registered: Mar, 2005

Re: The fate of reduce() in Python 3000

Posted: Mar 17, 2005 5:22 PM

>
> You can try to implement category theory in Python and
> specify everything up to isomorphism. This could lead to a
> full flegded DSL or even better a complete computer
> algebra system ( possible without leaving the language? )
> I like this way of thinking but i suspect Your boss is
> merciless with those guys who differ to much from the
> average and though not being a masochist i have some
> comprehension for this point of view.

While surely there is truth in this, there are also a lot of people using python outside of such an context.
If I remember correctly, the language is said to come with batteries included. Please leave folks that like to think in terms of Y-operators and higher-order functions their toys, will you? Fixing lambda, adding all the missing higher-order functions and currying instead of dropping some of the better features of the language would be nice.

Having to do complex higher-order functional programming with inner functions and generator expressions only would be so terribly cumbersome.

Pasko Robert

Posts: 5
Nickname: pkr
Registered: Mar, 2005

Re: The fate of reduce() in Python 3000

Posted: Mar 18, 2005 2:00 AM

I think that these (reduce/lambda) should be viewed as two separate issues:


1. map/reduce/filter primitives, and 
2. lambda.

If we take as a maxim that a feature should be provided once in the core language then there isn't any issue about the group 1 - list comprehensions do provide the feature, and if someone will miss the syntax, he can always define the corresponding functions himself.

Lambda, however, is different beast IMHO - because it allows you to define a closure without having to name it - that is something a nested function does not provide though they will both end up in a semantically equivalent closure object. And since python treats functions as first class objects (or perhaps 0.99 class objects? :-), it is a feature which has its justification. If you are allowed to write directly foo(X,Y) instead having to write


a, b = X, Y
foo(a,b)

for any object in python (OK, almost any), why should there be an exception for functions [well, there already is, as lambda does not provide everything a nested function does - but then removing lambda is not solving the problem either :-)]

Sure it can be used to write ugly code, but there isn't a single feature in any language, which could not be banned on that premise :-), and it is also possible to devise examples where anonymous functions will improve code clarity/readability.

Bram Cohen

Posts: 2
Nickname: bram
Registered: Mar, 2005

the proposed extensions and changes

Posted: Mar 18, 2005 9:08 AM

I like the extension of sum() to work on non-integers, but think that breaking it for lists of length 0 would be very bad. I'd like to propose an alternate implementation:


def sum(mylist, total = 0):
    for item in mylist:
        total += item
    return total

Then you can do sum(['these ', 'are ', 'words'], '')

I'm in favor of any() and all() being added as builtins.

I love the proposed [x in mylist if x > 0] style of syntax, and hope that it can be added. I already use [x for x in mylist if x > 0] because I can never remember the syntax of filter().

In my code there are only a few uses of lambda, mostly having to do with passing functions to graphical libraries. Defining the functions inline does improve clarity in these cases, but doing it with lambda feels like a hack, because usually they usually don't supply a meaningful return value. An inline def, using a reserved 'inline' keyword, which someone suggested in an earlier comment, would be much nicer.

I'm generally in favor of completely removing filter, map, reduce, and lambda, but have the caveats outlined above.

Nick Coghlan

Posts: 13
Nickname: ncoghlan
Registered: Dec, 2004

Re: the proposed extensions and changes

Posted: Mar 18, 2005 3:20 PM

> def sum(mylist, total = 0):
>     for item in mylist:
>         total += item
>     return total

> Then you can do sum(['these ', 'are ', 'words'], '')

That's basically the way sum works now (except that it uses total = total + item, and explicitly disallows strings as the initial value, pointing users towards ''.join() instead))

Something that might be interesting to add is operator.op_ident which would simply return the other operand for addition, multiplication and right division. Then a sequence of any type of object could be summed using:

  sum(seq, operator.op_ident)

Marcin Kowalczyk

Posts: 40
Nickname: qrczak
Registered: Oct, 2004

Re: the proposed extensions and changes

Posted: Mar 19, 2005 4:56 AM

> explicitly disallows strings as the initial value, pointing users towards ''.join() instead)

Why does it point towards ''.join() instead of doing ''.join()?

Flat View: This topic has 119 replies on 8 pages [ « | 1 2 3 4 5 6 7 8 | » ]

Previous Topic

Next Topic