The Artima Developer Community
Sponsored Link

Weblogs Forum
Self in the Argument List: Redundant is not Explicit

43 replies on 3 pages. Most recent reply: Nov 30, 2011 5:26 AM by ilian zapryanov

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 43 replies on 3 pages [ « | 1 2 3 | » ]
John Roth

Posts: 8
Nickname: johnroth
Registered: Jan, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 24, 2008 6:54 PM
Reply to this message Reply
Advertisement
I thoroughly agree, Bruce, and I've been saying it for quite a while. In fact, I'd go further than you're suggesting and make both self and cls keywords.

The reason is that this makes the distinction between instance methods, class methods and static methods inferable by inspecting the body, and completely eliminates the need for wrappers. It also eliminates the need for the silly @ declaration or the earlier classmethod() or staticmethod() assignment.

John Roth

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 3:14 AM
Reply to this message Reply
>

> class C(object):
> def m1(arg1, arg2): pass
> def m2(arg1, arg2, self=self): pass
>


???

The self parameter assigned to the keyword argument must exist once the class C is created. I have a hard time to understand the object identity of the self parameter.

George Sakkis

Posts: 14
Nickname: gsakkis
Registered: Jun, 2007

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 8:32 AM
Reply to this message Reply
> > On the other hand, life is much more difficult in
> Python,
> > and the hooks for syntax beautifying (metaclass
> > programming) makes Bruce's suggestion difficult to
> > implement without breaking other features of the
> > language:
> >
> > - decorators
> > - metaclasses
> > - lambda functions
> > - instance-only callables: obj.blah = lambda
> self,
> > foo, bar: None

> > - virtual members implemented by
> > obj.__getattr__
> > - .im_self and .im_func
> members
> > of the method object builtin class.
>
> + nested class definitions where the inner class can refer
> to the self of the outer class.
>
> One has to do a lot more work to turn this proposition
> into a PEP.

Indeed, changing the way things work now is too difficult (if not impossible); however providing a shortcut for the common case is a different issue and it's worth considering. The crux of the problem here is that plain vanilla instance methods are far more common than all the aforementioned features combined. Therefore it's easy to claim "explicit self is redundant" if one never needs any of the less common features. One of Python's appeals though is that it "makes easy things easy and hard things possible". 'self' is explicit in order to "make hard things possible", with the downside that it makes easy things (plain instance methods) slightly harder.

It's worth noting that it is possible to define implicitly-self'ed methods in current python, at least with bytecode hacking [1]. Whether the 'selfless' decorator is an improvement over explicit self - let alone including it as a Python builtin - is debatable. Self-hating people (no pun intended) may like it better though.

[1] http://www.voidspace.org.uk/python/weblog/arch_d7_2006_12_16.shtml

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 10:31 AM
Reply to this message Reply
> > One has to do a lot more work to turn this proposition
> > into a PEP.
>
> Indeed, changing the way things work now is too difficult
> (if not impossible); however providing a shortcut for the
> common case is a different issue and it's worth
> considering.

Yes, the reduction of combinations of syntactical elements shall always be considered if it doesn't lower expressiveness or leads to inconsistencies. However I do think that those combinations are already deliberately chosen in Python and neither arbitrary nor redundant. If someone claims the opposite this is a pretty strong claim that needs to be defended by good arguments not just mourning and polemics.

> The crux of the problem here is that plain
> vanilla instance methods are far more common than all the
> aforementioned features combined.

My counterexample just addresses scoping rules. Another one would have to deal with reflection inside of decorators that are applied to methods.

def print_class(f):
def method(self, *args, **kwd):
print self.__class__
return f(self, *args, **kwd)
method.__name__ = f.__name__
return method


If you sum this up lots of broken stuff ( scope, reflection ... ) has to fixed individually and special purpose semantics has to be developed. I wonder if the most simple solution isn't already available to everyone?


> Therefore it's easy to
> claim "explicit self is redundant" if one never needs any
> of the less common features. One of Python's appeals
> though is that it "makes easy things easy and hard things
> possible". 'self' is explicit in order to "make hard
> things possible", with the downside that it makes easy
> things (plain instance methods) slightly harder.

This famous Alan Kay quote usually doesn't lead to a sensible language design discussion. Performance optimization is the hard part in Python and the easy one in C while the perspective is entirely reversed when it comes to memory management and usage of container data structures. I tend to think in tradeoffs rather than in a fixed cardinal ordering of simple and complex.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 2:00 PM
Reply to this message Reply
> ... However I do
> think that those combinations are already deliberately
> chosen in Python and neither arbitrary nor redundant. If
> someone claims the opposite this is a pretty strong claim
> that needs to be defended by good arguments not just
> mourning and polemics.

I agree strongly.

> > The crux of the problem here is that plain
> > vanilla instance methods are far more common than all
> the
> > aforementioned features combined.
>
> My counterexample just addresses scoping rules. Another
> one would have to deal with reflection inside of
> decorators that are applied to methods.
>

> def print_class(f):
> def method(self, *args, **kwd):
> print self.__class__
> return f(self, *args, **kwd)
> method.__name__ = f.__name__
> return method
>


I'm not totally clear how this would break. Wouldn't the call work exactly the same way? The only difference would be that self would be an implicit argument, right?

> If you sum this up lots of broken stuff ( scope,
> reflection ... ) has to fixed individually and special
> purpose semantics has to be developed. I wonder if the
> most simple solution isn't already available to everyone?

The pragmatist in me tends to agree and he's driving. But, just for shits and giggles and because I've gained a lot from your input in the past, let's assume breaking things was not a concern i.e. if you were to recreate Python from a clean slate, would you chose to require the parent object be passed into member methods in all cases? Do you feel that this is really the best design or is it (as is my opinion) a wart but one that we can live with.

I think it's obvious that a alternate solution that doesn't require the explicit self parameter to be defined is possible. But if you are challenging that assertion then it's a much different conversation.

Wolfgang Lipp

Posts: 17
Nickname: wolf2005
Registered: Sep, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 2:24 PM
Reply to this message Reply
@Kay """The self parameter assigned to the keyword argument must exist once the class C is created"""---i’d believe the deeper trouble with the formulation you gave, self = self in the method declaration, as you pointed out, is that there is a necessity to ‘beam’ a self into position at the appropriate point in time. we already do exactly that: mything.f() ‘beams’ mything right into the first position of the ensuing call, f(mything). this is without doubt a much better choice than breaking expectations about at what point in time default values get assigned, as people find that point already a little hard to guess. so the very point where self is landed---when it will be available to whom?---is well chosen, as it is easy to mentally trace and understand.

i tried to get friends with any of the proposed solutions, but none can convince me. right here and now, my feeling is that we should make self less of a hard rule and lift it a bit into keyword space (but not quite), by simply stopping to insist on self having to appear in the signature.

i suggest: to make the appearance of a first parameter self in object method declarations an option. thus, code as already exists can go on, code that is to be written will see that all lines def f(x,y): will be read exactly as def f(self,x,y):.

that is, in our incantation, we just leave out a word that we know is always there, be it explicit or implicit. this could happen at a fairly early stage in parsing. the resulting (modified) method object irons over the deficient signature, but also keeps the information that something particular happened within the vm tree.

this step would satisfy all code that does not use something but self for the intended purpose---people who put a self later in the signature can receive warnings, and people who use another name for self (i heard some people use s) can expect their code to break in mysterious ways, which suits them well ;-)

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 4:08 PM
Reply to this message Reply
> @Kay """The self parameter assigned to the keyword
> argument must exist once the class C is created"""---i’d
> believe the deeper trouble with the formulation you gave,
> self = self in the method declaration, as you
> pointed out, is that there is a necessity to ‘beam’ a
> self into position at the appropriate point
> in time. we already do exactly that:
> mything.f() ‘beams’ mything
> right into the first position of the ensuing call,
> f(mything).

There is nothing paradox about it. The existence of mything and f precedes the method call. Just because one can de-sugar the method call into a function call and flip names doesn't mean there are serious problems with the lifetime of the objects.

We expect the self on the RHS of self = self to be created when the object is instantiated, not before even the class exists.

> i suggest: to make the appearance of a first parameter
> self in object method declarations an option.
> thus, code as already exists can go on, code that is to be
> written will see that all lines def f(x,y):
> will be read exactly as def f(self,x,y):.

This could work. But I'd rather mark the first argument explicitly when it is intended to be the object instance to be passed into the method. The way I'd define it is def f(!self, x, y):. This makes special semantics explicit and also avoids major trouble with scoping in case of nested classes. I have no opinion about making self a keyword.

Another idea could be using annotations instead of special characters:

def f(cls:0, x, y):

For methods the 0th parameter can be explicitely denotated or omitted. Without 0 annotation the first parameter is indexed by 1 and self has the obvious meaning.

> that is, in our incantation, we just leave out a word that
> we know is always there, be it explicit or implicit. this
> could happen at a fairly early stage in parsing.
> the
> resulting (modified) method object irons over the
> deficient signature, but also keeps the information that
> something particular happened within the vm tree.

I don't expect any trouble with the bytecode compiler.

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 5:18 PM
Reply to this message Reply
> >

> > def print_class(f):
> > def method(self, *args, **kwd):
> > print self.__class__
> > return f(self, *args, **kwd)
> > method.__name__ = f.__name__
> > return method
> >

>
> I'm not totally clear how this would break. Wouldn't the
> call work exactly the same way? The only difference would
> be that self would be an implicit argument, right?

It's actually the same problem as with lambda expressions or any other standalone functions that are going to replace methods. So it might add little to what was discussed already.

> I think it's obvious that a alternate solution that
> doesn't require the explicit self parameter to be defined
> is possible. But if you are challenging that assertion
> then it's a much different conversation.

See my solution presented to Wolfgang. It generates syntactical overhead in the general case but omits self from the parameter list in the convenient one. I think it is compliant with the "convention over configuration" mantra. I still reject the assertion made in the blog entry title which assumes self is just there for some awkward "philosophical" reason and could be omitted without serious implications.

Wolfgang Lipp

Posts: 17
Nickname: wolf2005
Registered: Sep, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 5:21 PM
Reply to this message Reply
@Kay """There is nothing paradox about it."""---exactly, but it must be done right. i don’t think i could formulate that behavior entirely in python itself. seen from that point, the behavior of first parameters is indeed ‘magic’ (i.e. out of the reach of python the language, only there for python the vm to do).

your choice to use a sigil like an exclamation mark is surely well founded; however, it effects the exact opposite of what i want: i want to bring declaring object methods into a line with declaring module-level methods. i do not want to freight the signature with more stuff. i’d rather sacrify the implicitness of the co-existing local and global namespaces---in a method body, i must always use self explicitly to access an instance attribute, but any global variable can easily sneak in, as they have no special mark---they could just as well be method-local names. that is alright for well-known staples of the language, but it is less of an obvious privilege for names that may have only sprung into existence in an if __name__ == '__main__' block. it just isn’t very obvious. that being, said, ruby’s implicit-in-the-method-body self and @ is very unconvincing, so self should remain explicit in the method body. if there was a requirement to explicitly mark global names, the special treatment of the method body namespace could go, and this could open this namespace for injection of names---in other words, without implicit globals, the formulation of an optional self becomes, among other things, doable in python-the-language itself, a clear advantage in clarity.---i already had deleted another similar paragraph, but i’m afraid such considerations are inevitable when mucking around with self.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 5:34 PM
Reply to this message Reply
>> I'm not totally clear how this would break. Wouldn't the
>> call work exactly the same way? The only difference
>> would be that self would be an implicit argument, right?
>
> It's actually the same problem as with lambda expressions
> or any other standalone functions that are going to
> replace methods. So it might add little to what was
> discussed already.

I didn't really understand how those would be broken either. Perhaps my understanding of how Python works in this regard is wrong. I was under the impression that when you made a call like:

obj.foo()

It actually translated (at least roughly) to calling a class method with the obj as the first parameter. I think someone else pointed at this as a 'small lie' so I'm clearly missing something important.

If it doesn't work that way, it actually kind of bugs me more that self is required in the method declaration because it's then not just tedious and repetitive but actually confusing and misleading.

Michael Chermside

Posts: 17
Nickname: mcherm
Registered: Jan, 2004

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 5:35 PM
Reply to this message Reply
Bruce:

Yes. I agree completely.

I know... I'm not adding much with this comment, but I still wanted to say it.

-- Michael Chermside

Attila Vágvölgyi

Posts: 3
Nickname: wigy
Registered: Sep, 2008

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 25, 2008 6:58 PM
Reply to this message Reply
Sorry, James, that I was a little bit too strong with the expression "small lie".

I just referred to codes like this one:

class C:
def x(i, j):
j + 1
class D:
y = C.x
z = lambda i, j: j + 2
d = D()
print d.z
print d.y # <<< Surprise!
print d.z(4)
print d.y(4) # <<< Surprise again!

If you run this code, you get something like:

<bound method D.<lambda> of <__main__.D instance at 0x00FBC328>>
<unbound method C.x>
6
Traceback (most recent call last):
File "C:\example.py", line 11, in ?
print d.y(4)
TypeError: unbound method x() must be called with C instance as first argument (got int instance instead)

I think this is very surprising for the uninitiated. I have not read the relevant part of the CPython source code (yet), but in a duck-typed language I did not expect that an unbound method object carries type information with itself and at the commented line the interpreter refuses to bind the first parameter with the D instance.

It does not fit into the D.y(d, 4) pattern.

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 26, 2008 3:35 AM
Reply to this message Reply
> I didn't really understand how those would be broken
> either. Perhaps my understanding of how Python works in
> this regard is wrong. I was under the impression that
> when you made a call like:
>
> obj.foo()
>
> It actually translated (at least roughly) to calling a
> class method with the obj as the first parameter.
> I think
> someone else pointed at this as a 'small lie' so I'm
> clearly missing something important.

Fact is, the following equation holds: assert obj.foo() == obj.__class__.foo(obj)
whatever object you choose ( unless obj.foo() already erases the harddisc of course ).

In case we have a fixed class Bar = obj.__class__ the method Bar.foo is called unbound whereas obj.foo is bound.

The language has to deal with both. When we apply a decorator on a method it is the unbound method that is passed to the decorator. It cannot be otherwise. But how do we call an unbound method? It is clear from the equation statement that we pass the object instance and I wonder what
happens if it isn't represented in the argument signature?

Maybe one can work around normal argument passing and define a function like bind_instance returning a bound method. Or one uses some sort of currying syntax:

class Bar:
def foo(self)(*args, **kwd):...

the application Bar.foo(obj) returns a specialized function which is exactly a bound method. We can omit the (self) expression in the code for convenience.

Wolfgang Lipp

Posts: 17
Nickname: wolf2005
Registered: Sep, 2005

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 26, 2008 5:33 AM
Reply to this message Reply
@Kay """When we apply a decorator on a method it is the unbound method that is passed to the decorator."""---which seems to imply that within python the language we stand no chance to inject self, because it is simply not around at that time, right? but what we can do is to retrieve, in the decorator, the source lines of the original method without self, prepend self to the signature, exec() the method declaration, and return the method that results. i know it sounds horrible to fiddle around with lines of code but apart from bytecodehacks i cannot see another way to inject names into a method namespace---the method code must be re-interpreted to make self a local variable. this way, nothing is changed about python; the decorator can be applied where desired; and a metaclass, a class decorator, or a special import statement could make the thing available by default.

Attila Vágvölgyi

Posts: 3
Nickname: wigy
Registered: Sep, 2008

Re: Self in the Argument List: Redundant is not Explicit Posted: Sep 26, 2008 6:18 AM
Reply to this message Reply
Hi Kay,

Thanks for clarifying the important concepts 'bound method object' and 'unbound method object' in your post, which I missed from my previous posts.

Again without going into details, I show you an example that is valid python, and breaks your invariant:

class Class:
def give1(self):
return 1
def give2(self):
return 2
obj = Class()
obj.give1 = obj.give2
print obj.give1()
print obj.__class__.give1(obj)
print obj.give1() == obj.__class__.give1(obj)

This code prints:

2
1
False

There are so many aspects of the language being involved in calling a simple method on an object, that changing this behavior needs a language lawyer and reading the source code of the interpreter. Here you forgot about instance-only members, but I listed some more in my first post.

I liked your idea that separates binding the first argument from calling the already bound method. And that is why I would prefer a syntax like:

class C:
def self.meth(i):
print i
c = C()
c.meth(2)

This would separate the special argument from the rest, and would better resemble the actual call. For that to work, we would need to add self as a reserved word, and maybe also something like def class.meth(i): for class methods, and then the simple def func(i): would be a plain function in the class dictionary, i not being bound to the object nor the class when being called with a dotted syntax.

Also we would need to extend the lambda syntax with self.lambda and class.lambda to make fewer exceptions in the interpreter behavior.

I believe that these changes are too much for 3.1 and need to go into 4.0, because the current heuristics in the interpreter cannot coexist with these syntax changes without causing troubles. Probably we could have a from future import explicit_methods in the meantime.

Flat View: This topic has 43 replies on 3 pages [ « | 1  2  3 | » ]
Topic: Some Insights From Europe Previous Topic   Next Topic Topic:


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us