如何在某些后缀s3中从一个存储桶复制到另一个存储桶

问题描述：

我有3个水桶1.commonfolder 2.jsonfolder 3.csvfolder.

公用文件夹将同时包含json和csv文件

Common folder will be having both json and csv files

需要将所有csv文件复制到csvfolder

need to copy all csv files to csvfolder

需要将所有json文件复制到json文件夹

need to copy all json files to json folder

下面的代码可用于获取commonfolder之后的所有文件

Code is below to get all the files from commonfolder How to copy after that

import boto3

s3 = boto3.client('s3')
def lambda_handler(event, context):
    #List all the bucket names
    response = s3.list_buckets()
    for bucket in response['Buckets']:
        print (bucket)
        print(f'{bucket["Name"]}')
        #Get the files of particular bucket
        if bucket["Name"] == 'tests3json':
         
            resp = s3.list_objects_v2(Bucket='commonfolder')
            for obj in resp['Contents']:
                files = obj['Key']
                print(files)

            if(filename.split('.')[1].lower()=='json'):
                copyjson(bucket,filename)
                #copyjson(jsonfolder,filename)
            elif(filename.split('.')[1].lower()=='csv'):
                copycsv(bucket, filename)
                #copycsv(csvfolder,filename)

需要创建一个新函数copyjson，copycsv来完成这项工作
- need to create a new function copyjson,copycsv to do this job
  
  需要根据文件扩展名从common-bucket复制到csv-bucket或json-bucket
  
  Need to copy from common-bucket to either csv-bucket or json-bucket depending on the file extension

答

您可以检查以下代码:

import boto3

s3 = boto3.resource('s3')

def lambda_handler(event, context):
    
    source_bucket = s3.Bucket('01-commonfolder-231')
    json_bucket = s3.Bucket('02-jsonfolder-3435')
    csv_bucket = s3.Bucket('03-csvfolder-4552')
    
    for object in source_bucket.objects.all():
        
        #print(object)
        
        if object.key.endswith('.json'):
            
            print(f"{object.key} to json bucket")
      
            copy_object = json_bucket.Object(object.key)
            copy_object.copy({'Bucket': object.bucket_name,
                              'Key': object.key})                             
            
        elif object.key.endswith('.csv'):
            
            print(f"{object.key} to csv bucket")            

            copy_object = csv_bucket.Object(object.key)
            copy_object.copy({'Bucket': object.bucket_name,
                              'Key': object.key})

我使用自己的带有测试文件的样本存储桶对此进行了测试:

I tested this using my own sample buckets with test files:

aaa.json to json bucket
bbbbb.csv to csv bucket
bbbbb.json to json bucket
hhhh.csv to csv bucket

如何在某些后缀s3中从一个存储桶复制到另一个存储桶

相关推荐