从Python 3.8中的AWS lambda temp目录导入软件包
我遇到的问题类似于 SO问题但答案对我不起作用.我正在尝试从AWS Lambda的/tmp
文件夹导入Python库(假设为 xgboost
).
The problem I have is similar to this SO question but the answer doesn't work for me.
I am trying to import Python library (let's say xgboost
) from /tmp
folder in AWS Lambda.
库 requests
已添加到Lambda层,我所做的是:
Library requests
is added to Lambda layer and what I did is:
import json
import io
import os
import zipfile
import requests
import sys
sys.path.insert(0, '/tmp/')
sys.path.append('/tmp/')
os.environ["PYTHONPATH"] = "/var/task"
def get_pkgs(url):
print("Getting Packages...")
re = requests.get(url)
z = zipfile.ZipFile(io.BytesIO(re.content))
print("Extracting Packages...")
z.extractall("/tmp/")
print("Packages are downloaded and extracted.")
def attempt_import():
print("="*50)
print("ATTEMPT TO IMPORT DEPENDENCIES...")
print("="*50)
import xgboost
print("IMPORTING DONE.")
def main():
URL = "https://MY_BUCKET.s3.MY_REGION.amazonaws.com/MY_FOLDER/xgboost/xgboost.zip"
get_pkgs(URL)
attempt_import()
def lambda_handler(event, context):
main()
return "Hello Lambda"
我得到的错误是 [ERROR] ModuleNotFoundError:没有名为"xgboost"的模块
.我授予了S3存储桶所有必要的权限,并且我肯定Lambda可以访问 .zip
文件,因为 requests.get
有效并且变量 z
返回:
The error I get is [ERROR] ModuleNotFoundError: No module named 'xgboost'
. I gave my S3 bucket all necessary permissions, and I am positive that Lambda can access the .zip
file since the requests.get
works and variable z
returns:
< zipfile.ZipFile file =< _io.BytesIO对象,位于0x7fddaf31c400>mode ='r'>
实际上,我上面的代码有效,并且我犯了一个非常愚蠢的错误.我实际上没有压缩 xgboost
软件包文件夹( xgboost
, xgboost.libs
和 xgboost.dist-info
)压缩了他们的父母文件夹,我将其命名为 package-xgoboost
,但该文件夹在AWS lambda中不起作用.确保您实际上直接将这三个文件夹压缩了.
Actually, my code above works and I had a rather silly error. Instead of zipping the xgboost
package folders (xgboost
, xgboost.libs
and xgboost.dist-info
) I actually zipped their parental folder which I named package-xgoboost
, and that didn't worked in AWS lambda. Be sure that you actually zip those 3 folders directly.
还要确保您的 xgboost
库是最新的.以前,我使用的是1.2.1版本,该版本也不起作用.最终,升级库并压缩最新的 xgboost
版本(在我的情况下为1.3.3)即可.
Also, make sure your xgboost
library is up-to-date. Previously I used version 1.2.1 which didn't work either. Upgrading the library and zipping the newest xgboost
version (in my case 1.3.3) finally worked.