不能腌制 <type 'instancemethod'>使用多处理 Pool.map() 时

问题描述:

我正在尝试使用 multiprocessingPool.map() 函数来同时划分工作.当我使用以下代码时,它工作正常:

I'm trying to use multiprocessing's Pool.map() function to divide out work simultaneously. When I use the following code, it works fine:

import multiprocessing

def f(x):
    return x*x

def go():
    pool = multiprocessing.Pool(processes=4)        
    print pool.map(f, range(10))


if __name__== '__main__' :
    go()

但是,当我以更面向对象的方法使用它时,它不起作用.它给出的错误信息是:

However, when I use it in a more object-oriented approach, it doesn't work. The error message it gives is:

PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup
__builtin__.instancemethod failed

当以下是我的主程序时会发生这种情况:

This occurs when the following is my main program:

import someClass

if __name__== '__main__' :
    sc = someClass.someClass()
    sc.go()

以下是我的 someClass 类:

import multiprocessing

class someClass(object):
    def __init__(self):
        pass

    def f(self, x):
        return x*x

    def go(self):
        pool = multiprocessing.Pool(processes=4)       
        print pool.map(self.f, range(10))

有人知道可能是什么问题,或者有什么简单的方法可以解决吗?

Anyone know what the problem could be, or an easy way around it?

问题是多处理必须pickle 东西才能在进程间sling 它们,而绑定的方法是不能picklable 的.解决方法(无论您认为它容易"与否;-)是将基础结构添加到您的程序中以允许腌制此类方法,并使用 copy_reg 标准库方法.

The problem is that multiprocessing must pickle things to sling them among processes, and bound methods are not picklable. The workaround (whether you consider it "easy" or not;-) is to add the infrastructure to your program to allow such methods to be pickled, registering it with the copy_reg standard library method.

例如,Steven Bethard 对此线程的贡献(在线程的末尾)展示了一种非常可行的方法来允许通过 copy_reg 进行方法pickling/unpickling.

For example, Steven Bethard's contribution to this thread (towards the end of the thread) shows one perfectly workable approach to allow method pickling/unpickling via copy_reg.