The Artima Developer Community
Sponsored Link

Weblogs Forum
PyCon Update

13 replies on 1 page. Most recent reply: Jan 4, 2007 3:16 PM by James Watson

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 13 replies on 1 page
David Goodger

Posts: 48
Nickname: goodger
Registered: Apr, 2004

PyCon Update (View in Weblogs)
Posted: Jan 2, 2007 11:48 PM
Reply to this message Reply
Summary
What I'll be doing at PyCon 2007 ("Idiomatic Python" tutorial, Docutils sprint), and a reminder of the early-bird registration deadline.
Advertisement

In my last entry, I asked for talk ideas. I received a few ideas which helped me to narrow down my proposals, but not a deluge to pass on to others. Thanks to those who replied! I ended up submitting proposals for one tutorial and one talk.

The talk proposal, “Generators & Iterators 101”, was rejected. There was another proposal covering similar ground that was accepted. The names of proposal authors were kept hidden during the review process and this result demonstrates that the system worked (no favoritism!). There were so many good proposals that the reviewers (including me) had a hard time narrowing them down.

I'm happy to say that my tutorial proposal was accepted:

“Code Like a Pythonista: Idiomatic Python”

In the tutorial I presented at PyCon 2006 (“Text & Data Processing”), I was surprised at the reaction to a couple of incidental techniques I used (advanced “%” string formatting, and decorate-sort-undecorate). Many of the attendees were unaware of these tools that experienced Python programmers use every day without thinking. Feedback from some of the attendees told me that they got a lot out of these sidebars. So this year's tutorial will present many idioms & techniques that beginning programmers can benefit from immediately.

Here's the promotional copy:

Are you comfortable with Python's syntax, but have yet to master its idioms? Does Python still feel a bit like a foreign language? Are you looking for more "elegance" for your programs?

In this interactive tutorial, we'll cover many essential Python idioms and techniques in depth, adding immediately useful tools to your belt. Rationale will be provided for all idioms -- the "why" in addition to the "what & how". We'll run through lots of small, practical, hands-on examples.

Attendees should bring a laptop if possible, to try out the techniques they learn.

http://us.pycon.org/TX2007/TutorialsPM#PM5

The tutorial outline is here.

Recently I was fortunate to have been asked to present a similar tutorial to my colleagues at work. It went quite well. Inevitably there were some mistakes and omissions, which I will be addressing. I was asked some good questions whose answers will be incorporated. So 90% of the work is already done, and the material has been tested. We're good to go.

My tutorial is scheduled for the afternoon of Tutorial Day, Thursday, February 22. There are a bunch of morning tutorials, and alternate afternoon tutorials too. You can check out the selection here.

Sprints

I'll be staying for all four days of development sprints. I'd like to get some work done on Docutils, which I've been neglecting lately. There are lots of bugs to squash and features to add. So I'll coach a Docutils Sprint. Ideas and participants are welcome!

Early-Bird Registration

The deadline for early-bird registration is January 15. After that, the registration fee goes up. If you haven't registered yet, go and do it, now. Doug Napoleone has deployed a nice event schedule web app where you can see and select the talks, tutorials, and other events that make up PyCon.

And don't forget to reserve a hotel room at the conference hotel. Please do it through the conference pages so we get proper credit.

See you at PyCon!

PyCon is a very rewarding event. I'm looking forward to it. Lots of people to meet. Lots to learn. I hope to see you there!


James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: PyCon Update Posted: Jan 3, 2007 9:51 AM
Reply to this message Reply
I hope this isn't too off-subject but I came across something with Python (Jython) that I think is hugely powerful and insanely simple. The thing I am puzzled about is that I searched for a long while for an example of how to solve thins problem in Python and didn't see it anywhere. Since you are talking about Python idioms, perhaps you will indulge me.

I'm not sure if it's just so obvious that no one feels the need to point it out or if I was searching for the wrong things. I can't imagine that I am the first to come up with this. Maybe there's something really wrong with doing this.

Anyway, I wanted to create a simple wrapper around a Java Object that allowed me to pass in a method argument instead of a class instance. I couldn't extend the Java class and the instances were coming from a factory. I really needed a wrapper.

I finally realized I could do is something like this:

def __getattr__(self, name):
return eval("self.wrap." + name)

This is crude, but it works for my purposes. But it seems to me that this could easily be generalized to create a single class that could create mixin objects on the fly. Is there something like this already or is there something wrong with this approach? I'm sure a better implementation is possible.

David Goodger

Posts: 48
Nickname: goodger
Registered: Apr, 2004

Re: PyCon Update Posted: Jan 3, 2007 10:20 AM
Reply to this message Reply
James,

I don't use Java or Jython so don't really understand the need for the wrapper, but I'd recommend implementing the wrapper itself like this:


def __getattr__(self, name):
return getattr(self.wrap, name)


It's still dynamic, but direct.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: PyCon Update Posted: Jan 3, 2007 10:46 AM
Reply to this message Reply
> James,
>
> I don't use Java or Jython so don't really understand the
> need for the wrapper, but I'd recommend implementing the
> wrapper itself like this:

I don't 'need' the wrapper per se. The Java class I am using requires implementing a simple interface with one method. This is as close as Java gets to having method pointers. When I use this class in Jython, I am creating this wrapper to add the ability to pass in a method instead of having to extend the Java class.

>

> def __getattr__(self, name):
> return getattr(self.wrap, name)
>

>
> It's still dynamic, but direct.

Is that equivalent to the following?

self.wrap.getattr(name)

Because I tried that initially but I kept getting an exception on the call telling me [name] didn't exist on that call.

So are you saying this isn't a common idiom in Python? That kind of surprises me because runtime mixins are something I've really wanted for a while.

David Goodger

Posts: 48
Nickname: goodger
Registered: Apr, 2004

Re: PyCon Update Posted: Jan 3, 2007 11:06 AM
Reply to this message Reply
> >     def __getattr__(self, name):
> > return getattr(self.wrap, name)

> > It's still dynamic, but direct.
>
> Is that equivalent to the following?
>
> self.wrap.getattr(name)

Only if "wrap"'s class implements a "getattr" method. Classes in Python don't implement such a method by default.

> Because I tried that initially but I kept getting an
> exception on the call telling me [name] didn't exist on
> that call.

Are you sure the exception wasn't saying that "getattr" didn't exist? Because unless it was explicitly implemented, it didn't. I can't say more without seeing some code.

> So are you saying this isn't a common idiom in Python?
> That kind of surprises me because runtime mixins are
> e something I've really wanted for a while.

Dynamic attribute access is a relatively common technique in Python.

Python has multiple inheritance, reducing the need for mixins. We have mixins, but implemented as base classes.

I suspect you're trying to view Python through Java-colored glasses. But I don't know Java, so I can't interpret what you're saying. Try posting to <python-list at python dot org> (== comp.lang.python newsgroup).

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: PyCon Update Posted: Jan 3, 2007 11:19 AM
Reply to this message Reply
> Python has multiple inheritance, reducing the need for
> mixins. We have mixins, but implemented as base classes.

I guess I just don't see the point of using extension if I can do this. This kind of composition seems to supercede extension. What value does extension provide that I don't get from this? Doesn't declaring a new class require knowing what I want to mix together before the code executes? This would not, right?

David Goodger

Posts: 48
Nickname: goodger
Registered: Apr, 2004

Re: PyCon Update Posted: Jan 3, 2007 12:53 PM
Reply to this message Reply
> > Python has multiple inheritance, reducing the need for
> > mixins. We have mixins, but implemented as base
> > classes.
>
> I guess I just don't see the point of using extension if I
> can do this. This kind of composition seems to supercede
> extension. What value does extension provide that I don't
> get from this?

I don't know what you mean by "extension" here.

> Doesn't declaring a new class require
> knowing what I want to mix together before the code
> executes? This would not, right?

If you need runtime dynamism, you have a technique already. Again, I suspect that your Java origins are coloring your perception of Python here. The two languages do things in different ways, and some techniques just don't translate from one to the other.

This is getting far too abstract for my comfort. I no longer know for sure that I know what we're talking about, if you know what I mean ;-). Please show some real code, and not just snippets; show enough to define the issue.

Erik Engbrecht

Posts: 210
Nickname: eengbrec
Registered: Apr, 2006

Re: PyCon Update Posted: Jan 3, 2007 6:21 PM
Reply to this message Reply
> > Python has multiple inheritance, reducing the need for
> > mixins. We have mixins, but implemented as base
> classes.
>
> I guess I just don't see the point of using extension if I
> can do this. This kind of composition seems to supercede
> extension. What value does extension provide that I don't
> get from this? Doesn't declaring a new class require
> knowing what I want to mix together before the code
> executes? This would not, right?

You can use this technique to create a proxy object, potentially with some sort of behavior like lazy initialization.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: PyCon Update Posted: Jan 3, 2007 11:41 PM
Reply to this message Reply
> > > Python has multiple inheritance, reducing the need
> for
> > > mixins. We have mixins, but implemented as base
> > > classes.
> >
> > I guess I just don't see the point of using extension if
> I
> > can do this. This kind of composition seems to
> supercede
> > extension. What value does extension provide that I
> don't
> > get from this?
>
> I don't know what you mean by "extension" here.

A subclass extends a superclass. This is 'extension'.

> > Doesn't declaring a new class require
> > knowing what I want to mix together before the code
> > executes? This would not, right?
>
> If you need runtime dynamism, you have a technique
> already. Again, I suspect that your Java origins are
> coloring your perception of Python here. The two
> languages do things in different ways, and some techniques
> just don't translate from one to the other.

I'm sure my perceptions are colored by the languages I have used in the past but I want to be clear that this is something that cannot be done in Java.

> This is getting far too abstract for my comfort. I no
> longer know for sure that I know what we're talking about,
> if you know what I mean ;-). Please show some real code,
> and not just snippets; show enough to define the issue.

Here's a simple example of what I mean.


class Mixin:
def __init__(self, *wrap):
self.wrap = wrap
def __getattr__(self, name):
for w in self.wrap:
if hasattr(w, name):
return getattr(w, name)

class Foo:
def foo(self):
print "foo!"
def foobar(self):
print "foo!"

class Bar:
def bar(self):
print "bar!"
def foobar(self):
print "foo!"


m = Mixin(Foo(), Bar())

m.foo()
m.bar()
m.foobar()


I can also do extend Mixin directly and make Objects that decorate existing instance as if they extend that Object's base class.

The only problem I have no solution for is getting the dir method to show the methods of the wrapped Objects as being part of the mixin. But this still seems like a really useful way to write code. I'm not sure why I had such a hard time finding how to do it.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: PyCon Update Posted: Jan 3, 2007 11:43 PM
Reply to this message Reply
errata: the foobar method of Bar should print "bar", not "foo".

David Goodger

Posts: 48
Nickname: goodger
Registered: Apr, 2004

Re: PyCon Update Posted: Jan 4, 2007 12:08 PM
Reply to this message Reply
>>>> Python has multiple inheritance, reducing the
>>>> need for mixins. We have mixins, but
>>>> implemented as base classes.
>>>
>>> I guess I just don't see the point of using
>>> extension if I can do this. This kind of
>>> composition seems to supercede extension.
>>> What value does extension provide that I don't
>>> get from this?
>>
>> I don't know what you mean by "extension" here.
>
> A subclass extends a superclass. This is
> 'extension'.

I call that "subclassing".

We seem to be talking at cross-purposes here.
Python has no discrete "mixin" or "interface"
type. Python implements mixins via the
subclassing of multiple superclasses
simultaneously (multiple inheritance). So the
question, "What value does extension provide that
I don't get from this?", seems to be meaningless:
"this" and "extension" seem to be the same thing
(subclassing).

Also, you don't seem to be using the term "mixin"
in the standard way; see http://en.wikipedia.org/wiki/Mixin

> Here's a simple example of what I mean.
...

You're doing dynamic method dispatching on wrapped
objects, the "composite" design pattern. So?

> I can also do extend Mixin directly and make
> Objects that decorate existing instance as if
> they extend that Object's base class.
>
> The only problem I have no solution for is
> getting the dir method to show the methods of
> the wrapped Objects as being part of the mixin.

From the docs for dir() (which is a function, not
a method):

"This information is gleaned from the object's
__dict__ attribute, if defined, and from the class
or type object."

So to get the wrapped objects' methods to show up
in a dir() call, you'd need to reimplement their
__dict__ namespaces to chain lookups the way
inheritance does. That's a sign that you may be
better off just using inheritance (and
polymorphism, etc.).

Also there's this:

"Note: Because dir() is supplied primarily as a
convenience for use at an interactive prompt, it
tries to supply an interesting set of names more
than it tries to supply a rigorously or
consistently defined set of names, and its
detailed behavior may change across releases."

In other words, you shouldn't be using dir() for
anything serious in your code, just for casual
introspection.

> But this still seems like a really useful way to
> write code.

No offense, but it seems to me like a really
twisted and indirect way to write code. Perhaps
your application requires this level of dynamism,
but I suspect it can be done in an easier way. I
can't say without more concrete info about what
you are trying to accoplish.

> I'm not sure why I had such a hard
> time finding how to do it.

Perhaps because it's not typically necessary,
therefore not done very often. It's not an idiom.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: PyCon Update Posted: Jan 4, 2007 1:10 PM
Reply to this message Reply
> >>>> Python has multiple inheritance, reducing the
> >>>> need for mixins. We have mixins, but
> >>>> implemented as base classes.
> >>>
> >>> I guess I just don't see the point of using
> >>> extension if I can do this. This kind of
> >>> composition seems to supercede extension.
> >>> What value does extension provide that I don't
> >>> get from this?
> >>
> >> I don't know what you mean by "extension" here.
> >
> > A subclass extends a superclass. This is
> > 'extension'.
>
> I call that "subclassing".

It's a pretty standard term but I suppose it's used differently in the Python universe.

> We seem to be talking at cross-purposes here.
> Python has no discrete "mixin" or "interface"
> type. Python implements mixins via the
> subclassing of multiple superclasses
> simultaneously (multiple inheritance). So the
> question, "What value does extension provide that
> I don't get from this?", seems to be meaningless:
> "this" and "extension" seem to be the same thing
> (subclassing).

From the section you quoted "This kind of composition..."

'This' is composition. The question is what does subclassing provide that using composition as demonstrated in my example does not? You are not saying composition is subclassing are you?

> Also, you don't seem to be using the term "mixin"
> in the standard way; see
> http://en.wikipedia.org/wiki/Mixin

From that entry: "With mixins the class definition defines only the attributes and parameters associated with that class; methods are left to be defined elsewhere"

I don't see how that's fundamentally different from what this does but whatever, call it a Composite. It's irrelevant to my point.

> > Here's a simple example of what I mean.
> ...
>
> You're doing dynamic method dispatching on wrapped
> objects, the "composite" design pattern. So?

What does subclassing buy me over this? This does things that subclassing cannot.

> > I can also do extend Mixin directly and make
> > Objects that decorate existing instance as if
> > they extend that Object's base class.
> >
> > The only problem I have no solution for is
> > getting the dir method to show the methods of
> > the wrapped Objects as being part of the mixin.
>
> From the docs for dir() (which is a function, not
> a method):
>
> "This information is gleaned from the object's
> __dict__ attribute, if defined, and from the class
> or type object."
>
> So to get the wrapped objects' methods to show up
> in a dir() call, you'd need to reimplement their
> __dict__ namespaces to chain lookups the way
> inheritance does. That's a sign that you may be
> better off just using inheritance (and
> polymorphism, etc.).

But inheritance it not nearly as flexible. That's what I am getting at.

> Also there's this:
>
> "Note: Because dir() is supplied primarily as a
> convenience for use at an interactive prompt, it
> tries to supply an interesting set of names more
> than it tries to supply a rigorously or
> consistently defined set of names, and its
> detailed behavior may change across releases."
>
> In other words, you shouldn't be using dir() for
> anything serious in your code, just for casual
> introspection.

That basically contradicts what you just wrote above. I understand that dir() is not crucial but it can't hurt.

> > But this still seems like a really useful way to
> > write code.
>
> No offense, but it seems to me like a really
> twisted and indirect way to write code. Perhaps
> your application requires this level of dynamism,
> but I suspect it can be done in an easier way.

I don't see what's hard about it. I don't find "it's just not done that way" to be very convincing. Given a class that cannot be extended, or instances that come from a factory that you cannot change, what's a better way to wrap those instances in an Object that has all the methods and attributes of the original instance?

In any event, some very smart people have argued that composition is almost always preferable to extension (subclassing). I've come to agree with them. Composition is much more reusable. It allows you to not only add the attributes and methods of another class, it allows you to add the attributes, methods and state of an instance of that class. Some people have even argued that subclassing should be relegated to the trash-heap.

subclassing is really very similar to composition in most ways and composition can effectively replace sublclassing. On the other hand, there are lots of things that you can do with composition that you cannot with subclassing.

> I can't say without more concrete info about what
> you are trying to accoplish.

I don't see what's hard about it. The syntax for creating an instance of the class I show above isn't really different from declaring a class to subclass another.

> > I'm not sure why I had such a hard
> > time finding how to do it.
>
> Perhaps because it's not typically necessary,
> therefore not done very often. It's not an idiom.

I never suggested it was necessary. An automobile is not necessary but I own one anyway. Anyway, that's the answer to my question: it's not a standard idiom.

David Goodger

Posts: 48
Nickname: goodger
Registered: Apr, 2004

Re: PyCon Update Posted: Jan 4, 2007 1:52 PM
Reply to this message Reply
>>> A subclass extends a superclass. This is
>>> 'extension'.
>>
>> I call that "subclassing".
>
> It's a pretty standard term but I suppose it's used
> differently in the Python universe.

In Python, we extend or override methods.

> 'This' is composition. The question is what does
> subclassing provide that using composition as demonstrated
> in my example does not?

Subclassing provides automatic chained lookups of
methods and attributes from superclasses; it's
built-in, with syntax support. You had to roll
your own lookups in your composition example.
Subclassing also allows cooperative classes
(http://www.python.org/download/releases/2.2/descrintro/#cooperation).

> You are not saying composition is
> subclassing are you?

Of course not. I was just trying to clarify a
terminology misunderstanding.

>> Also, you don't seem to be using the term "mixin"
>> in the standard way; see
>> http://en.wikipedia.org/wiki/Mixin
>
> From that entry: "With mixins the class definition defines
> only the attributes and parameters associated with that
> class; methods are left to be defined elsewhere"

Taken in isolation, that sentence is confusing at
best. It applies to Flavors and CLOS. Try the
first sentence instead: "In object-oriented
programming languages, a mixin is a class that
provides a certain functionality to be inherited
by a subclass, but is not meant to stand alone."
In other words, it's *mixed in* with other
functionality.

>>> Here's a simple example of what I mean.
>> ...
>>
>> You're doing dynamic method dispatching on wrapped
>> objects, the "composite" design pattern. So?
>
> What does subclassing buy me over this? This
> does things that subclassing cannot.

Different techniques for different applications.
If you need composition, use it.

>> In other words, you shouldn't be using dir() for
>> anything serious in your code, just for casual
>> introspection.
>
> That basically contradicts what you just wrote above.

No it doesn't. Read it again.

>>> But this still seems like a really useful way to
>>> write code.
>>
>> No offense, but it seems to me like a really
>> twisted and indirect way to write code. Perhaps
>> your application requires this level of dynamism,
>> but I suspect it can be done in an easier way.
>
> I don't see what's hard about it.

Not hard. Just not done on a daily basis.

> I don't find "it's just not done that way" to be
> very convincing. Given a class that cannot be
> extended, or instances that come from a factory
> that you cannot change, what's a better way to
> wrap those instances in an Object that has all
> the methods and attributes of the original
> instance?

That's a perfectly valid application of composition
or adaptation.

>> I can't say without more concrete info about what
>> you are trying to accoplish.
>
> I don't see what's hard about it. The syntax
> for creating an instance of the class I show
> above isn't really different from declaring a
> class to subclass another.

Look, you show some abstract code in a vacuum, misuse
some terminology, and ask if it's an idiom. I was just
trying to pull the necessary details out of you to
answer that. Like pulling teeth ;-)
Now that I have some details, I can
confirm that what you're doing looks just fine.

Time to end this thread.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: PyCon Update Posted: Jan 4, 2007 3:16 PM
Reply to this message Reply
> >>> A subclass extends a superclass. This is
> >>> 'extension'.
> >>
> >> I call that "subclassing".
> >
> > It's a pretty standard term but I suppose it's used
> > differently in the Python universe.
>
> In Python, we extend or override methods.
>
> > 'This' is composition. The question is what does
> > subclassing provide that using composition as
> demonstrated
> > in my example does not?
>
> Subclassing provides automatic chained lookups of
> methods and attributes from superclasses; it's
> built-in, with syntax support. You had to roll
> your own lookups in your composition example.

But I would never have to write that class again. I can just reuse it. That's a pretty weak argument.

> Subclassing also allows cooperative classes
> http://www.python.org/download/releases/2.2/descrintro/#co
> operation).

That looks to me like a solution to one of the problems with inheritance. There's nothing about it that cannot be done with composition. A solution to a problem caused by inheritance is not an argument for inheritance.

> > You are not saying composition is
> > subclassing are you?
>
> Of course not. I was just trying to clarify a
> terminology misunderstanding.
>
> >> Also, you don't seem to be using the term "mixin"
> >> in the standard way; see
> >> http://en.wikipedia.org/wiki/Mixin
> >
> > From that entry: "With mixins the class definition
> defines
> > only the attributes and parameters associated with that
> > class; methods are left to be defined elsewhere"
>
> Taken in isolation, that sentence is confusing at
> best. It applies to Flavors and CLOS. Try the
> first sentence instead: "In object-oriented
> programming languages, a mixin is a class that
> provides a certain functionality to be inherited
> by a subclass, but is not meant to stand alone."
> In other words, it's *mixed in* with other
> functionality.

And anyone can go in there and change that. Wikipedia is great and all but it's not an authority. That sounds like the definition of a trait. I think of a mixin as the final class built from traits. 'Dynamic Mixin' seems like a good way to describe an Object with attributes defined by it's parts to me. But I didn't really start the question to quibble over terms. I guess I'm more about ideas than idioms.

> >>> Here's a simple example of what I mean.
> >> ...
> >>
> >> You're doing dynamic method dispatching on wrapped
> >> objects, the "composite" design pattern. So?
> >
> > What does subclassing buy me over this? This
> > does things that subclassing cannot.
>
> Different techniques for different applications.
> If you need composition, use it.
>
> >> In other words, you shouldn't be using dir() for
> >> anything serious in your code, just for casual
> >> introspection.
> >
> > That basically contradicts what you just wrote above.
>
> No it doesn't. Read it again.

You said that supporting dir() meant I should be using inheritance then said dir() isn't important. If it's not important, it wouldn't suggest that I need to use inheritance to me.

> Look, you show some abstract code in a vacuum, misuse
> some terminology, and ask if it's an idiom.

You forgot to mention the part where you were very dismissive of something because it wasn't familiar to you.

Flat View: This topic has 13 replies on 1 page
Topic: PyCon Update Previous Topic   Next Topic Topic: Podcast Interview


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us