AMAZON S3(一)Introduction and File Upload API

AMAZON S3(1)Introduction and File Upload API
AMAZON S3(1)Introduction and File Upload API

S3 Amazon Simple Storage Service(Amazon S3) - Simple Storage Service

Price
5GB of Amazon S3, 20,000 Get requests, 2,000 Put Requests and 15GB of data transfer out each month for one year.
First 1 TB/month  $0.03 per G    or     $0.0125 per GB      or  $0.007 per GB

1. Using the High-Level Java API for Multipart Upload
http://docs.aws.amazon.com/AmazonS3/latest/dev/mpListPartsJavaAPI.html

2. Using the Low-Level Java API for Multipart Upload
http://docs.aws.amazon.com/AmazonS3/latest/dev/llJavaUploadFile.html

3. Upload My InputStream File to S3
Prepare the S3Storage Client
package com.sillycat.persist

import com.amazonaws.auth.BasicAWSCredentials
import com.amazonaws.regions.{Regions, Region}
import com.amazonaws.services.s3.AmazonS3Client
import com.sillycat.util.{IncludeLogger, IncludeConfig}

object S3Storage extends IncludeConfig with IncludeLogger{

  private def getCredential = {
    new BasicAWSCredentials(
      config.getString(envStr("s3.keyId")),
      config.getString(envStr("s3.accessKey"))
    )
  }

  val getClient = new AmazonS3Client(getCredential)
  val regionName = config.getString(envStr("s3.regionName"))
  getClient.setRegion(Region.getRegion(Regions.fromName(regionName)))

}

Util Class upload the InputStream
package com.sillycat.persist

import java.io.{File, InputStream}

import com.amazonaws.services.s3.model.{CannedAccessControlList, ObjectMetadata, PutObjectRequest}
import com.amazonaws.util.{Base64, IOUtils}
import com.sillycat.util.IncludeConfig
import org.apache.commons.codec.digest.DigestUtils

object S3StorageUtil extends IncludeConfig{

  val bucketName = config.getString(envStr("s3.bucketName"))

  val publicURLHost = config.getString(envStr("s3.publicURL"))

  def uploadObject(keyName:String, inputStream: InputStream): String ={
    val s3client = S3Storage.getClient

    val metadata = new ObjectMetadata()
    val request = new PutObjectRequest(bucketName, keyName, inputStream, metadata)
    request.setCannedAcl(CannedAccessControlList.PublicRead)

    val meta = new ObjectMetadata()
    meta.setContentLength(IOUtils.toByteArray(inputStream).length)

    val resultByte = DigestUtils.md5(inputStream)
    val streamMD5 = new String(Base64.encode(resultByte))
    meta.setContentMD5(streamMD5)

    s3client.putObject(request)
    val publicURL = publicURLHost + File.separator + bucketName + File.separator + keyName
    publicURL
  }

}

In this implementation, I solve the memory issue.
[warn] c.a.s.s.AmazonS3Client - No content length specified for stream data.  Stream contents will be buffered in memory and could result in out of memory errors.

References:
http://docs.aws.amazon.com/AmazonS3/latest/dev/llJavaUploadFile.html
http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadObjSingleOpJava.html
http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMPDotJavaAPI.html

https://aws.amazon.com/s3/
http://abc08010051.iteye.com/blog/2082956

http://stackoverflow.com/questions/8351886/amazons3-putobject-with-inputstream-length-example