带有生成器的WSGI文件流
我有以下代码:
def application(env, start_response):
path = process(env)
fh = open(path,'r')
start_response('200 OK', [('Content-Type','application/octet-stream')])
return fbuffer(fh,10000)
def fbuffer(f, chunk_size):
'''Generator to buffer file chunks'''
while True:
chunk = f.read(chunk_size)
if not chunk: break
yield chunk
我不确定这是正确的,但是我在互联网上发现的信息残片使我认为它应该起作用.基本上,我想分块地流式传输文件,并为此从我的应用程序函数中传回一个生成器.但是,这只会打印出标题,而实际上并没有发送回任何数据,任何人都可以告诉我为什么吗?
I'm not sure that it's right but the scraps of information I've found on the internet have led me to think it ought to work. Basically I want to stream a file out in chunks, and to do that I'm passing a generator back from my application function. However this only prints out the headers and doesn't actually send back any data, can anyone tell me why this is?
或者,如果这是完全错误的,那么执行此操作的最佳方法是什么?我无法将整个文件缓冲在内存中,因为我将要使用的文件可能是千兆字节.
Alternatively, if this is completely wrong, what's the best way to do this? I can't buffer the whole file in memory as the files I'll be working with are potentially gigabytes large.
第三级问题:完成输出后,关闭文件的最佳方法是什么?在我发布的代码中,我仍然看不到要关闭文件.
Tertiary question: What's the best way to close the file once I'm done outputting it? In the code I posted I can't see anyway to actually close the file.
(我正在使用uWSGI 1.2.4运行python 3.2.3)
(I'm running python 3.2.3 with uWSGI 1.2.4)
不小心,uwsgi注意不要让错误泄漏,但是如果您在更严格的实现中运行应用程序,如果将python提供的代码称为wsgiref.simple_server
,则可以更轻松地看到问题所在.
Without some care, uwsgi is careful not to allow errors to leak, but a if you run your application in a stricter implementation, say the one provided with python as wsgiref.simple_server
, you can more easily see the problem.
Serving <function application at 0xb65848> http://0.0.0.0:8000
Traceback (most recent call last):
File "/usr/lib64/python3.2/wsgiref/handlers.py", line 138, in run
self.finish_response()
File "/usr/lib64/python3.2/wsgiref/handlers.py", line 179, in finish_response
self.write(data)
File "/usr/lib64/python3.2/wsgiref/handlers.py", line 264, in write
"write() argument must be a bytes instance"
AssertionError: write() argument must be a bytes instance
localhost.localdomain - - [04/Aug/2012 16:27:08] "GET / HTTP/1.1" 500 59
问题在于,wsgi要求必须将往返于HTTP网关的数据用作bytes
,但是当您使用open(path, 'r')
时,python 3方便地将读取的数据转换为unicode,在python 3中是str
,使用默认编码.
The problem is that wsgi requires that data transmitted to and from the HTTP gateway must be served as bytes
, but when you use open(path, 'r')
, python 3 conveniently converts the data read to unicode, what in python 3 is str
, using the default encoding.
更改
fh = open(path, 'r')
到
fh = open(path, 'rb')
# ^
修复它.