用Python卸载模块

TL/DR:

import gc, sys

print len(gc.get_objects()) # 4073 objects in memory

# Attempt to unload the module

import httplib
del sys.modules["httplib"]
httplib = None

gc.collect()
print len(gc.get_objects()) # 6745 objects in memory

更新 我已就此问题联系Python开发人员，确实是无法完全卸载模块未来五年". (请参阅链接)

UPDATE I've contacted Python developers about this problem and indeed it's not going to be possible to unload a module completely "in next five years". (see the link)

请接受Python在2.x中确实不支持针对严重，根本，不可克服的技术问题的卸载模块.

Please accept that Python indeed does not support unloading modules for severe, fundamental, insurmountable, technical problems, in 2.x.

在我最近一次寻找应用中的内存泄漏的过程中，我将其范围缩小到模块，即我无法垃圾收集一个已卸载的模块.使用下面列出的 any 方法卸载模块会在内存中保留数千个对象.换句话说-我无法在Python中卸载模块...

During my recent hunt for a memleak in my app, I've narrowed it down to modules, namely my inability to garbage collect an unloaded module. Using any method listed below to unload a module leaves thousands of objects in memory. In other words - I can't unload a module in Python...

剩下的问题是尝试以某种方式垃圾收集模块.

The rest of the question is attempt to garbage collect a module somehow.

让我们尝试一下:

import gc
import sys

sm = sys.modules.copy()  # httplib, which we'll try to unload isn't yet 
                         # in sys.modules, so, this isn't the source of problem

print len(gc.get_objects()) # 4074 objects in memory

让我们保存一个sys.modules的副本，以便以后尝试将其还原. 因此，这是一个基线4074对象.理想情况下，我们应该以某种方式返回到这一点.

Let's save a copy of sys.modules to attempt to restore it later. So, this is a baseline 4074 objects. We should ideally return to this somehow.

让我们导入一个模块:

import httplib
print len(gc.get_objects()) # 7063 objects in memory

我们最多有7K个非垃圾对象. 让我们尝试从sys.modules中删除httplib.

We're up to 7K non-garbage objects. Let's try removing httplib from sys.modules.

sys.modules.pop('httplib')
gc.collect()
print len(gc.get_objects()) # 7063 objects in memory

嗯，那没用.嗯，但是__main__中没有参考吗?哦，是的:

Well, that didn't work. Hmm, but isn't there a reference in __main__? Oh, yeah:

del httplib
gc.collect()
print len(gc.get_objects()) # 6746 objects in memory

万岁，击落了300个物体.不过，没有雪茄，那就是4000多个原始对象. 让我们尝试从副本中恢复sys.modules.

Hooray, down 300 objects. Still, no cigar, that's way more than 4000 original objects. Let's try restoring sys.modules from copy.

sys.modules = sm
gc.collect()
print len(gc.get_objects()) # 6746 objects in memory

嗯，那是没有意义的，没有变化. 也许如果我们消灭全局变量...

Hmmm, well that was pointless, no change.. Maybe if we wipe out globals...

globals().clear()
import gc # we need this since gc was in globals() too
gc.collect()
print len(gc.get_objects()) # 6746 objects in memory

本地人?

locals().clear()
import gc # we need this since gc was in globals() too
gc.collect()
print len(gc.get_objects()) # 6746 objects in memory

那..如果我们imported在exec内部的一个模块怎么办?

What the.. what if we imported a module inside of exec?

local_dict = {}
exec 'import httplib' in local_dict
del local_dict
gc.collect()
print len(gc.get_objects())  # back to 7063 objects in memory

现在，这不公平，将其导入到__main__中，为什么?它应该永远不会离开local_dict ...啊！我们回到完全导入的httplib. 也许我们将其替换为虚拟对象?

Now, that's not fair, it imported it into __main__, why? It should have never left the local_dict... Argh! We back to fully imported httplib. Maybe if we replaced it with a dummy object?

from types import ModuleType
import sys
print len(gc.get_objects())  # 7064 objects in memory

血腥..... !!

Bloody.....!!

sys.modules['httplib'] = ModuleType('httplib')
print len(gc.get_objects())  # 7066 objects in memory

死模块，死！！

import httplib
for attr in dir(httplib):
    setattr(httplib, attr, None)
gc.collect()
print len(gc.get_objects())  # 6749 objects in memory

好吧，经过所有尝试，最好的方法是从起点开始+2675(几乎+50％)...这仅仅是一个模块...甚至什么都没有...

Okay, after all attempts, the best is +2675 (nearly +50%) from starting point... That's just from one module... That doesn't even have anything big inside...

好吧，现在认真点，我的错误在哪里? 如何卸载模块并擦除所有内容? 还是Python的模块是一次巨大的内存泄漏?

Ok, now seriously, where's my error? How do I unload a module and wipe out all of it's contents? Or is Python's modules one giant memory leak?

以更简单的形式复制的完整源代码: http://gist.github.com/450606

Full source in simpler to copy form: http://gist.github.com/450606