A better, modern implementation of Skipper's S3 file adapter
This adapter uses ECMAScript 2015 (ES 6) syntax: you must run at least Node.js v4 in order to use this adapter.
Compared to the official S3 adapter, this one allows you to do some extra things currently not possible (or too difficult) with the official one:
- Customise access control to uploaded files (i.e. make a file publicly readable)
- Upload filesystem-originated readable streams (or any other type of stream), not just http file uploads
- Create signed S3 URLs
$ npm install skipper-better-s3 --save
Also make sure you have Skipper installed as your body parser.
Skipper is installed by default in Sails v0.10 and above.
You upload the files as usual with any other adapter. See some examples below.
|key||String||Your Amazon access key|
|secret||String||Your Amazon secret|
|bucket||String||The S3 bucket to upload the file to|
|region||String||Optional. If your S3 bucket resides in a different region than "us-standard", you can use this parameter to operate on that region.|
|s3params||Object||Optional parameters to be passed to the underlying Amazon SDK library when performing the file upload. This could be any parameter that is supported by the S3's
|s3options||Object||Optional configuration for the upload request to be passed to the underlying Amazon SDK library. This could be any parameter that is supported by the second argument of the
|s3config||Object||Optional configuration for the S3 service itself. This is passed directly to the
|onProgress||Function||Marked by Skipper core as experimental. If provided, will be called periodically as the data is being uploaded with current progress information.|
const options =// This is the usual stuffadapter:key: 'somekeyhere'secret: 'dontsharethis'bucket: 'my-s3-bucket'region: 'us-east-1' // Optional - default is 'us-standard'// Let's use the custom s3params to upload this file as publicly// readable by anyones3params:ACL: 'public-read'// And while we are at it, let's monitor the progress of this uploadsailslogreq
Server-initiated file uploads
If you need to upload a file which your application generates (i.e. the file does not come via http upload request), you need to create a receiver to which you write the readable stream.
// You can use all the supported options hereconst options =key: 'somekeyhere'secret: 'dontsharethis'bucket: 'my-s3-bucket'// Create and configure an adapterconst adapter = options// Now, create a receiver - receiver is a writable stream which accepts// other streams that should be uploaded to S3// All files will be saved to this S3 directory (this configuration object is optional)const receiver = adapter// Suppose we want to upload a file which we temporarily saved to diskconst file = fs// And now, just send the file to the receiver!receiver
If you need to upload any other type of stream than a file stream or HTTP upload body (i.e. a zlib or crypto stream), you should add a
fdproperty to the stream, which should be the name of the file in the S3 storage. If you forget to add this property, a random unique identifier will be generated for you.
This feature is currently experimental as it lacks the expected response structure that is provided by Skipper when you upload a standard http stream.
To download a previously uploaded file, you must first get an actual adapter object to use it and provide some basic configuration, like your S3 credentials and the bucket to read from.
You can either get a readable stream that you can consume or pipe to the client directly (efficient, does not require buffering the whole file to memory), or you can provide a callback which will then receive the full contents of the file being downloaded.
const options =key: 'somekeyhere'secret: 'dontsharethis'bucket: 'my-s3-bucket'// This will give you an adapter instance configured with the// credentials and bucket defined aboveadapter = options// Now the adapter is ready to interface with your S3 bucket.// Let's assume you have a file named 'avatar.jpg' stored// in the root of this bucket...// Option 1: get a readable stream of the fileconst readableStream = adapter// Option 2: get the full file contents in a callbackadapter
You can get a list of files at a given path in a bucket.
const options =key: 'somekeyhere'secret: 'dontsharethis'bucket: 'my-s3-bucket'// This will give you an adapter instance configured with the// credentials and bucket defined aboveadapter = options// Assuming there is a directory named 'avatars' in your bucket root...adapter
adapter.rm(fd, done) on a configured adapter instance (see previous examples on how to get such instance).
fd should be the path to the object to be deleted, relative to the bucket.
// Assuming you already have an adapter instance...adapter
This adapter comes with some extra functionality not defined in the Skipper adapter specifications.
Generating signed URLs for S3
Signed URLs are a great way of allowing others to interact with your S3 storage directly. For example, you can generate a file download link that will be valid only for 5 minutes, or a link which will allow someone to upload a file into a predetermined location in your S3 bucket.
This is great, because it allows your clients to interact with your S3 storage directly, instead of bothering your server with all the network traffic and computational power necessary to upload/download files.
// Assuming you already have an adapter instance...const url = adapter// Give the url to the client - they can read this file directly from there// Optionally do a redirect (303 - "see other") to this file yourself:// res.redirect(303, url)
Full API documentation with detailed description of all methods is available offline. Clone this repo and do the following within the cloned folder:
$ npm install$ make docs$ open docs/index.html # If open does not work, just double-click this file
This software is licensed under the BSD-3-Clause License. See the LICENSE file for more information.