从AWS Lambda/tmp目录导入python包
我正在尝试将一个大小为300MB的zip文件(主要包含python软件包)上传到AWS Lambda.我清楚地了解,如果我们直接使用AWS开发工具包上传,则超出了可以上传到Lambda的zip的限制.因此,这将不起作用.
I am trying to upload a zip file, which contains mostly python packages, of size 300MB to AWS Lambda. I clearly understand that this exceeds the limit for the zip that can be uploaded to Lambda if we uploaded directly using AWS SDK. Therefore, this will not work.
为了克服这个问题,我决定将软件包下载到/tmp
目录中,并将其导入到主文件中(参考
In order to overcome this, i decided to download the packages in the /tmp
directory and import them to the main file (reference here). I compressed the required packages as pkgs.zip
and upload it to AWS S3. Then I download them using requests
extract them to /tmp/
.
def get_pkgs(url):
import requests
import io
import zipfile
print("Getting Packages...")
re = requests.get(url)
z = zipfile.ZipFile(io.BytesIO(re.content))
print("Extracting Packages...")
z.extractall("/tmp/")
print("Packages are downloaded and extracted.")
def attempt_import():
print("="*50)
print("ATTEMPT TO IMPORT DEPENDENCIES...")
print("="*50)
import numpy
import scipy
import six
print("IMPORTING DONE.")
def main():
URL = "https://s3-ap-southeast-1.amazonaws.com/BUCKET_NAME/pkgs.zip"
get_pkgs(URL)
attempt_import()
def lambda_handler(event, context):
main()
return "Hello Lambda"
但是,当我测试lambda函数时,它返回一条错误消息,提示找不到 numpy
However, when i test the lambda function, it returns an error saying that numpy
cannot be found
导入错误:没有名为numpy的模块
我的问题是,如何从/tmp/
目录导入所需的软件包?
My question is, How do I import the required packages from the /tmp/
diretory?
谢谢.
在将已下载的任何软件包导入到/tmp
文件夹(例如,导入到/tmp/要求
文件夹),则必须告诉系统在那儿查找依赖项.在代码的开头,仅在以下几行:
Before you can import any package that you have downloaded to the /tmp
folder (for example to the /tmp/requirements
folder) you have to tell the system to look for the dependencies over there. In the beginning of the code, just at these lines:
import sys
sys.path.insert(0, '/tmp/requirements/') # Or any path you desire