Direct to S3 Uploads with AJAX Presigning

Previously, I covered uploading to S3 from a Rails app using a presigned-url. This works just fine, but means the data flows from the visitors computer to your server before heading to S3. I’ve previously showed how to upload the file directly to S3, which requires a world-writable bucket.

There’s also a hybrid solution that has the server generate a presigned-url which the browser then uses to go direct to S3, allowing the data to bypass the server. Let’s look at two ways to do this. First, as is always the case when we want the browser to send a AJAX request to a different server, we need to configure S3 to sent a CORS header, specifically Access-Control-Allow-Origin. On the bucket Properties tab select Permissions > Edit CORS Configuration and paste in:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>http://example.com</AllowedOrigin>
        <AllowedMethod>PUT</AllowedMethod>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

Replacing http://example.com with the domain you are serving the form from (or simply with *). If you need to allow other methods, can be found at https://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html#how-do-i-enable-cors

Now that CORS is taken care of, we can make the AJAX call. As before, we need the aws-sdk gem to sign requests. In your Gemfile:

gem 'aws-sdk', '~> 2'

A simple approach is to have the server render the presigned URL in a form. With this approach we needed to predefine the filename as we signed URL before we know anything about the file. A UUID makes a good filename, unique filename in this case.

s3_resource = Aws::S3::Resource::new(region: 'us-east-1')
filename = SecureRandom.uuid
object = s3_resource.bucket('my-bucket').object(filename)
@presigned_url = object.presigned_url(:put)

(Outside of rails you would need to require 'securerandom'. See the previous post for details on S3 credentials.)

Our form looks like:

<form class="upload-form">
   <input id="image" type="file" name="image" accept="image/x-png, image/gif, image/jpeg" />
   <button type="submit">Go!</button>
</form>

And our Javascript looks like:

$(function() {

    var upload_image = function(form) {
        var field = $(form).find('input[name=image]');
        var file = field[0].files[0];
        var url = '<%= raw @presigned_url %>';
        console.log(file);
        $.ajax({
            type : 'PUT',
            url : url,
            data : file,
            processData: false,  // tell jQuery not to convert to form data
            contentType: file.type,
            success: function(json) { console.log('Upload complete!') },
            error: function (XMLHttpRequest, textStatus, errorThrown) {
                console.log('Upload error: ' + XMLHttpRequest.responseText);
            }
        });
    }

    $( '#upload-form' ).submit(function( event ) {
        event.preventDefault();
        var form = this;
        upload_image(form);
    });
});

The downside of generating the URL before the file is chosen in the browser is that we can’t know the file extension (i.e. .jpg). However, the image will be served from S3 with the right mime type, displaying correctly in the browser, and most browsers will add the extension if the image is saved to disk.

A fancier approach solves this issue. Instead of generating the presigned URL when the form is displayed, we generate it via an API call after the form is submitted. First, we need a controller method that will take a file name and signed it:

def presigned_url
  filename = params[:filename]
  object = s3_resource.bucket('my-bucket').object(filename)
  @presigned_url = object.presigned_url(:put)
  render json: {url: presigned_url }
end

Of course that requires some sort of checking to make sure we’re not overwriting an existing file, which is left as an exercise for the reader. An alternative is to extract the file extension and combine it with a UUID as above:

extension = File.extname(params[:filename])
filename = "#{SecureRandom.uuid}.#{extension}"

Our Javascript becomes:

$(function() {
    var upload_image = function(form,url) {
        var field = $(form).find('input[name=image]');
        var file = field[0].files[0];
        $.ajax({
            type : 'PUT',
            url : url,
            data : file,
            processData: false,  // tell jQuery not to convert to form data
            contentType: file.type,
            success: function(json) { console.log('Upload complete!') },
            error: function (XMLHttpRequest, textStatus, errorThrown) {
                console.log('Upload error: ' + XMLHttpRequest.responseText);
            }
        });
    }

    $( '#upload-form' ).submit(function( event ) {
        event.preventDefault();
        var form = this;
        var field = $(this).find('input[name=image]');
        var file = field[0].files[0];

        $.getJSON('/sign/presign_url.json',
                  {filename: file.name},
                  function(data) {
                      upload_image(form,data['url']);
                  }
                 );
    });
});

One security note, while this is more secure than a world-writable bucket, it is possible, if somewhat unlikely, that someone could disassemble your Javascript code, find you API endpoint, and use it to sign requests. You’ll want to consider how your can secure that endpoint (require authentication, etc.)

There you have it, signed, direct to S3 uploads that avoid moving the upload through your server. Next time we’ll look at working with both uploads and data in S3.