The Artima Developer Community
Sponsored Link

Weblogs Forum
Mixins considered harmful/4

4 replies on 1 page. Most recent reply: Apr 8, 2009 7:56 PM by Kevin Teague

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 4 replies on 1 page
Michele Simionato

Posts: 222
Nickname: micheles
Registered: Jun, 2008

Mixins considered harmful/4 (View in Weblogs)
Posted: Apr 8, 2009 1:13 AM
Reply to this message Reply
Summary
In this final installment I show a solution of the design problem posed last week, i.e. I show a possible mechanism for adding multiple interfaces to an object without polluting its namespace with mixin methods.
Advertisement

A possible solution

The design problem described in the last article is clearly a problem of interfaces. We have an object and we want to interact with it through multiple interfaces (GUI, HTTP, FTP, etc.). The mixin solution just adds a bunch of methods for each interface, with the result of creating a monster object with hundreds of methods.

This design is a maintenance nightmare. The human brain can manage a limited amount of information. An object with ten methods can be kept in mind easily enough, but an object with a hundred methods is outside the reach of the average programmer.

The solution is to split the hundred methods into ten categories with ten methods each: at this point you can keep the ten categories in your mind. This solution scales well: if I need a thousand methods, I can just define ten macro-categories, each macro-category including ten micro-categories, and I can keep in mind the macro-categories.

Hierarchical categories are the natural way to memorize information for the human mind, at least from Aristotle's times: this is the right solution, not to have a hundred methods at the same level in the same namespace.

We need therefore a mixin-like solution which keeps the methods in separate namespaces explicitly (usual mixins keeps the methods in separate namespaces but implicitly, without visibility to the use of the class).

Technically this idea can be implemented by defining an interface wrapper object which is also an attribute descriptor (if you are not familiar with the concept of descriptors in Python, you must read the beautiful essay by R. Hettinger on the subject):

class BaseWrapper(object):
    "Base class for interface wrappers built from mixin classes"
    def __init__(self, obj):
        self.__obj = obj
    def __get__(self, obj, objcls=None):
        if obj is None:
            return self
        return self.__class__(obj)
    def __getattr__(self, name):
        obj = self.__obj
        if obj is None:
            raise TypeError('Unbound wrapper %r' % self)
        return getattr(obj, name)
    def __repr__(self):
        names = ', '.join(n for n in dir(self) if not n.startswith('_'))
        msg = 'bound to %r' % self.__obj if self.__obj is not None else ''
        return '<%s {%s} %s>' % (self.__class__.__name__, names, msg)

def iwrapper(mixin):
    "Convert a mixin class into an interface wrapper object"
    # could be implemented with single-inheritance only, but why bother?
    cls = type(mixin.__name__ + 'Wrapper', (BaseWrapper, mixin), {})
    return cls(None)

Interface wrapper objects are instances of the mixin (interface) class but also proxies: they are intended to wrap the inner object, by dispatching first on the mixin methods and then to the inner methods. In practice, if you want to add an interface to an instance c of a class C, and the methods of the interface are stored into a mixin class M, you can just add an interface wrapper to C:

class C(object):
    m = iwrapper(M)

Now c.m is an instance of M which can also be used as a fake C object thanks to the __getattr__ trick. For simplicity I assuming that there are no name clashes between the names of the interface methods and the names of the inner methods.

Here is how you would use interface wrappers in the PictureContainer example:

class PictureContainer(DictMixin, object):
  # interface wrappers are instances of the corresponding mixin class;
  # moreover they dispatch on PictureContainer objects
  gui = iwrapper(GUI)
  http = iwrapper(HTTP)
  webdav = iwrapper(WEBDAV)
  ftp = iwrapper(FTP)
  auth = iwrapper(AUTH)

  @property
  def log(self):
    return logging.getLogger(self.__class__.__name__)

  def __init__(self, id, pictures_or_containers):
      self.id = id
      self.data = {}
      for poc in pictures_or_containers:
        # both pictures and containers must have an .id
        self.data[poc.id] = poc

  def __getitem__(self, id):
    return self.data[id]

  def __setitem__(self, id, value):
    self.log.info('Adding or replacing %s into %s', id, self.id)
    self.data[id] = value

  def __delitem__(self, id):
    self.log.warn('Deleting %s', id)
    del self.data[id]

  def keys(self):
    return self.data.keys()

Notice that I have refactored PictureContainer a bit. I have defined the property log explicitly (before it was imported, from utility import log). Since it takes only three lines, it makes sense to write them out and to avoid forcing the reader to look in another module. There is always a compromise between code reuse and spaghetti code. When in doubt, I always remind myself that readability counts.

As you see, I have removed all mixin classes except DictMixin. After all, I have decided that a PictureContainer is a dictionary, but it is not really also an object of type GUI, HTTP, WEBDAV, FTP, AUTH. Logically those are different interfaces or wrappers over the basic object.

There is still multiple inheritance from object, because DictMixin is an old-style class (for backward compatibility reasons) whereas interface wrappers, being attribute descriptors, are intended to be used with new-style classes. Inheriting from object makes PictureContainer a new style class. This is one of the rare cases where multiple inheritance is convenient, but this use case has already disappeared in Python 3.0, where all classes are new-style.

Discussion

Let me check if this solution to our design problem is consistent with the Zen di Python. First of all, the implementation of the interface wrapper concept is simple - 20 lines of code - and this is already a step in the right direction (if the implementation is hard to explain, it's a bad idea). All the methods of the mixin are localized in the mixin namespace and they do not pollute the namespace of the original class, and that's good (namespaces are one honking great idea -- let's do more of those!). Moreover, we are following the explicit is better than implicit principle. For instance, to access the POST method of the HTTP mixin we need to write self.http.POST, which is good, since the readers of our code will not need to guess the origin of the method (in the face of ambiguity, refuse the temptation to guess).

The solution is also usable (practicality beats purity): you can instantiate a PictureContainer object and perform some experiment from the Python prompt:

>>> pc = PictureContainer('root', [])

Autocompletion works pretty well:

>>> pc.ftp. # press TAB
pc.ftp.RECV
pc.ftp.SEND
...

the help function does not provide excessive information

>>> help(pc)
Help on PictureContainer in module mixins2 object:
class PictureContainer(UserDict.DictMixin, __builtin__.object)
|  Method resolution order:
|      PictureContainer
|      UserDict.DictMixin
|      __builtin__.object
|
|  Methods defined here:
|  ...
|
|  auth = <AUTHWrapper {is_admin ...} >
|  ftp = <FTPWrapper {RECV, SEND ...} >
|  gui = <GUIWrapper {draw_button, draw_buttons ...} >
|  http = <HTTPWrapper {GET, POST ...} >
|  webdav = <WEBDAVWrapper {GET, POST, LOCK, UNLOCK ...} >
...

and it is possible to introspect just the features you are interested in, without having everything mixed in (sparse is better than dense):

>>> print dir(pc.http)
['GET, 'POST', ...]

Conclusion

Is the solution presented here the only solution or even the best solution to the problem of adding multiple interfaces to an object? Certainly not. I have written it down in half an hour an I am not even using it, since (fortunately) I am not a framework writer.

The solution is intended as a suggestion for people which are refactoring a framework based on mixins and which have their methods organized in mixin classes. Then, the iwrapper function is able to convert such pre-existing classes into objects which can be used as class attributes, replacing multiple inheritance with composition.

If you do not already have the mixin classes, you may be better off with a different solution. Moreover, if you are using Python 2.6 or later, it is natural to tackle this problem in terms of Abstract Base Classes (ABC), which I have completely ignored.

The solution I have presented lists all the interfaces supported by an object directly (statically); however you could use a different design based on adapters, where the object is dynamically adapted with the correct interface before being passed to its consumer object. A solution based on adapters is fine if the list of supported interfaces is not know a priori.

My point here was to show here that Python (at least from Python 2.2) makes it easy to implement solutions based on composition rather than on inheritance.

Actually, I would say that the general trend of modern Python frameworks is to favor component programming over inheritance. You should take in account this fact. Instead of my home made solution you may want to try out an enterprise-ready solution, like the component framework of Zope 3 (I personally prefer home made solutions to over-engineered frameworks, but YMMV).

Nowadays I tend to consider multiple inheritance and mixins more of a hack than a legitimate design technique: they may be useful when you need to integrate with pre-existing code with a minimal offert, or as a debugging tool, when you want to instrument a third party hierarchy, but you if are designing an application from scratch you are often better off if you do not rely on mixins. Actually I usually recommend to use as little as possible even single inheritance.


Chris McDonough

Posts: 4
Nickname: mcdonc
Registered: Jan, 2006

Re: Mixins considered harmful/4 Posted: Apr 8, 2009 10:26 AM
Reply to this message Reply
"Instead of my home made solution you may want to try out an enterprise-ready solution, like the component framework of Zope 3 (I personally prefer home made solutions to over-engineered frameworks, but YMMV)"

The "home-made" vs. "overengineered" comparison is incorrect here, IMO. I think it might actually be reversed in this specific case.

Were to use the Zope Component Architecture, you'd probably use a set of adapters registered against the PictureContainer class. The PictureContainer class would not have any mixins or magic at all, nor any declarations at class scope. It'd be completely ignorant about HTTP, DAV, FTP etc. Instead, all the magic is done by registering adapters against the PictureContainer class externally, then using those adapters to get APIs in code that manipulates a PictureContainer.

p = PictureContainer()
http = getAdapter(p, IHTTP)
return http.POST(...)

... where some adapter like this is registered against the PictureContainer class:

class PictureContainerHTTP:
def POST(self, data):
....

The registrations can be done in Python or via a configuration language (ZCML).

Maybe the best treatment of this pattern in comparison to inheritance, etc is at http://rhodesmill.org/brandon/adapters/ . The talk that accompanies these slides from PyCon 2008 is also online (poor quality audio) at http://www.youtube.com/watch?v=UF77e2TeeQo

Michele Simionato

Posts: 222
Nickname: micheles
Registered: Jun, 2008

Re: Mixins considered harmful/4 Posted: Apr 8, 2009 10:42 AM
Reply to this message Reply
Yes, I have idea of how the ZCA solves this and I thank
you for the pointer. But explaining the component architecture would have taken me too far.
Moreover, I must confess that I am not a fan of
the Zope 3 component architecture, that I find more
complex than needed. This is a matter of personal taste,
of course, and in general I am an anti-framework kind of guy.

Chris McDonough

Posts: 4
Nickname: mcdonc
Registered: Jan, 2006

Re: Mixins considered harmful/4 Posted: Apr 8, 2009 11:34 AM
Reply to this message Reply
Fair enough, I also pick-and-choose when it comes to overly-frameworky bits so I can sympathize. Personally I don't think the ZCA is overly frameworky (although other bits of Zope are). I recognize you've flipped the bozo bit on the ZCA for whatever reason, but I'll go into some more detail here for folks that are less discerning. ;-)

What might be making folks feel that the ZCA is overly frameworky for this problem might be a misunderstanding: while adapters need to be looked up by interface, adapter registrations don't need to be declared *against* interfaces. They can adapt classes too. So one could do:


from zope.interface import implements
from zope.interface import Interface

from zope.component import provideAdapter

class PostContainer(object):
pass

class IHTTP(Interface):
def POST(data):
""" Do whatever the POST task implies """

class PostContainerHTTPAdapter:
implements(IHTTP)
def __init__(self, context):
self.context = context

def POST(self, data):
# self.context will be the PostContainer, do stuff to it here

provideAdapter(PostContainerHTTPAdapter, adapts=(PostContainer,), provides=IHTTP)


That's the setup and registration code. Then in code that actually does stuff, say, in some function, you would use the adapter to do the work:


from zope.component import getAdapter
from mypackage.mymodule import IHTTP

def dopost(postcontainer, data):
adapter = getAdapter(postcontainer, IHTTP)
adapter.POST(data)


This separates out the 'http' behavior entirely from the PostContainer class itself. The PostContainer knows *nothing* about HTTP.

Using this pattern also makes unit testing the "dopost" function above very easy as you can swap out the adapter registration at testing time with one that is a dummy implementation. To do that with Michele's solution, you'd need to monkeypatch or do something similarly clever.

Kevin Teague

Posts: 15
Nickname: wheat
Registered: Mar, 2006

Re: Mixins considered harmful/4 Posted: Apr 8, 2009 7:56 PM
Reply to this message Reply
"the component framework of Zope 3" is not quite accurate. Zope 3 is a full framework and exhibits many "frameworky" tendencies -- overengineering and inversion of control. This means that someone using Zope 3 as a framework would write components that "fit" into the framework, and Zope 3 would do the work of making sure the right objects were comoposed into the right adapters, etc.

But the "Zope Component Architecture" which just does the object composition is not a framework - nor would I describe that part of Zope as overengineered. It's simply a couple of Python packages (zope.component and zope.interface) and can be used entirely standalone as a library (and doesn't need any of the other kajillion packages that form the rest of the Zope framework), e.g. it's up to the calling code to determine how the object composition is configured (be that with registrations in Python, XML files or using convention-over-configuration). As Chris's example shows, the benefit of this approach is that the code which is being composed against can be 100% unaware of the object compositions that it's taking part in.

I'd also say that using Interfaces (from zope.interface) is a better solution than using Python's ABCs. I haven't seen anyone attempt object composition which takes advantage of ABCs yet, but there are a number of limitations that ABCs have which Interfaces do not - such as ABCs need to be added to the inheritance hierarchy (usually as a mixin), and they apply globally to all object instances of a class whereas interfaces can be applied per object.

Flat View: This topic has 4 replies on 1 page
Topic: OSCON 2009 Registration Open Previous Topic   Next Topic Topic: AntiSocial Networking


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us