Proxy object in Python

Question

I'm looking for way to pass method calls through from an object (wrapper) to a member variable of an object (wrappee). There are potentially many methods that need to be externalised, so a way to do this without changing the interface of the wrapper when adding a method to the wrappee would be helpful.

class Wrapper(object)
  def __init__(self, wrappee):
    self.wrappee = wrappee

  def foo(self):
    return 42

class Wrappee(object):
  def bar(self):
    return 12

o2 = Wrappee()
o1 = Wrapper(o2)

o1.foo() # -> 42
o1.bar() # -> 12
o1.<any new function in Wrappee>() # call directed to this new function 

It would be great if this call redirection is "fast" (relative to a direct call, i.e. not adding too much overhead).

Answer 1

A somewhat elegant solution is by creating an "attribute proxy" on the wrapper class:

class Wrapper(object):
    def __init__(self, wrappee):
        self.wrappee = wrappee

    def foo(self):
        print 'foo'

    def __getattr__(self, attr):
        return getattr(self.wrappee, attr)


class Wrappee(object):
    def bar(self):
        print 'bar'

o2 = Wrappee()
o1 = Wrapper(o2)

o1.foo()
o1.bar()

all the magic happens on the __getattr__ method of the Wrapper class, which will try to access the method or attribute on the Wrapper instance, and if it doesn't exist, it will try on the wrapped one.

if you try to access an attribute that doesn't exist on either classes, you will get this:

o2.not_valid
Traceback (most recent call last):
  File "so.py", line 26, in <module>
    o2.not_valid
  File "so.py", line 15, in __getattr__
    raise e
AttributeError: 'Wrappee' object has no attribute 'not_valid'

Answer 2

If you really need this to be fast, the fastest option is to monkeypatch yourself at initialization:

def __init__(self, wrappee):
    for name, value in inspect.getmembers(wrappee, callable):
        if not hasattr(self, name):
            setattr(self, name, value)

This will give your Wrapper instances normal data attributes whose values are bound methods of the Wrappee. That should be blazingly fast. Is it?

class WrapperA(object):
    def __init__(self, wrappee):
        self.wrappee = wrappee
        for name, value in inspect.getmembers(wrappee, callable):
            if not hasattr(self, name):
                setattr(self, name, value)

class WrapperB(object):
    def __init__(self, wrappee):
        self.wrappee = wrappee
    def __getattr__(self, name):
        return getattr(self.wrappee, name)

In [1]: %run wrapper
In [2]: o2 = Wrappee()
In [3]: o1a = WrapperA(o2)
In [4]: o1b = WrapperB(o2)
In [5]: %timeit o2.bar()
10000000 loops, best of 3: 154 ns per loop
In [6]: %timeit o1a.bar()
10000000 loops, best of 3: 159 ns per loop
In [7]: %timeit o1b.bar()
1000000 loops, best of 3: 879 ns per loop
In [8]: %timeit o1b.wrapper.bar()
1000000 loops, best of 3: 220 ns per loop

So, copying bound methods has a 3% cost (not sure why it even has that much…). Anything more dynamic than this would have to pull attributes from self.wrapper, which has a minimum 66% overhead. The usual __getattr__ solution has 471% overhead (and adding unnecessary extra stuff to it can only make it slower).

So, that sounds like an open and shut win for the bound-methods hack, right?

Not necessarily. That 471% overhead is still only 700 nanoseconds. Is that really going to make a difference in your code? Probably not unless it's being used inside a tight loop—in which case you're almost certainly going to want to copy the method to a local variable anyway.

And there are a lot of downsides of this hack. It's not the "one obvious way to do it". It won't work for special methods that aren't looked up on the instance dict. It's statically pulling the attributes off o2, so if you create any new ones later, o1 won't be proxying to them (try building a dynamic chain of proxies this way…). It wastes a lot of memory if you have a lot of proxies. It's slightly different between Python 2.x and 3.x (and even within the 2.x and 3.x series, if you rely on inspect), while __getattr__ has very carefully been kept the same from 2.3 up to the present (and in alternate Python implementations, too). And so on.

If you really need the speed, you may want to consider a hybrid: a __getattr__ method that caches proxied methods. You can even do it in two stages: something that's called once, you cache the unbound method in a class attribute and bind it on the fly; if it's then called repeatedly, you cache the bound method in an instance attribute.

Answer 3

Here's another monkey-patch method. This one copies methods into the Wrapper class directly rather than the created wrapper object. The key advantage to this one is that all special methods such as __add__ will work.

class Wrapper(object):
    def __init__(self, wrappee):
        self.wrappee = wrappee

    def foo(self):
        print('foo')


def proxy_wrap(attr):
    "This method creates a proxy method that calls the wrappee's method."
    def f(self, *args):
        return getattr(self.wrappee, attr)(*args)
    return f

# Don't overwrite any attributes already present
EXCLUDE = set(dir(Wrapper))

# Watch out for this one...
EXCLUDE.add('__class__')

for (attr, value) in inspect.getmembers(Wrappee, callable):
    if attr not in EXCLUDE:
        setattr(Wrapper, attr, proxy_wrap(attr))

I used this to wrap numpy arrays. With Wrappee set to np.ndarray:

import numpy as np

Wrappee = np.ndarray

# [The block I wrote above]

wrapped = Wrapper(np.arange(10))

Operations such as wrapped + 1 still work.

Answer 4

For anyone coming across this old question, looking for an out-of-the-box solution that takes care of all the nitty gritty details such as type comparison and special methods. I ran into a utility package that claims to do all of the heavy lifting: wrapt.

Especially the Object Proxy part is relevant for the original question:

class CustomProxy(wrapt.ObjectProxy):
    
    def foo(self):
        return 42

wrapper = CustomProxy(wrappee)
wrapper.foo()  # 42
wrapper.bar()  # 12, proxied to wrappee

One downside I encountered: when you want to override a property of the wrapped object, you have to jump through some hoops with @property decorators, which is why in the end I settled for a very simple __getattr__ override instead of using wrapt. But if proxying is a bigger part of your application this is probably worth looking into.

Wrapt also has a lot of utilities for creating decorators.

Disclaimer: I haven't actually used wrapt in any code, just came across it looking for a solution and it seems to fit the bill of the original question, so I thought I'd share.