Cheap backups with Amazon Glacier

Last week Amazon announced Amazon Glacier, where you can have files stored at $0.01 per GB / month, quite a good deal, considering that S3 goes for $0.093 GB/month with reduced redundancy, or Dropbox at its best is 0.825/GB committing to 100GB for a full year, although obviously they fill very different use cases.

To get that pricing there are some drawbacks that make it only useful for storing files that don’t need to be retrieved often, ie. backups for disaster recovery. Downloading or listing files in Glacier take more than 4 hours, so that gives you an idea. Behind the scenes it uses Amazon SQS (Simple Queue Service) and SNS (Simple Notification Service) to handle the download and inventory requests, so you can do extra things like getting emails when your requests are ready.

I have created glacier-cli using the Java API to upload, download, delete and list files stored in Glacier from the command line, as Amazon only provides the APIs for now and some examples. Make sure you save the output when uploading the files, as you will need the ids of the files later on when you need to download them.

Get the code from GitHub.

Glacier-CLI

Building

mvn clean package

Configuration

Create $HOME/AwsCredentials.properties with your AWS keys

secretKey=…
accessKey=…

Commands

  • upload vault_name file1 file2 …
  • download vault_name archiveId output_file
  • delete vault_name archiveId
  • inventory vault_name

Command line options

 -output <file_name>   File to save the inventory to. Defaults to 'glacier.json'
 -queue <queue_name>   SQS queue to use for inventory retrieval. Defaults to 'glacier'
 -region <region>      Specify URL as the web service URL to use. Defaults to 'us-east-1'
 -topic <topic_name>   SNS topic to use for inventory retrieval. Defaults to 'glacier'

Examples

Upload file1 and file2 to vault pictures

java -jar glacier-1.0-jar-with-dependencies.jar upload pictures file1 file2

Download archive with id xxx from vault pictures to file pic.tar (takes >4 hours)

java -jar glacier-1.0-jar-with-dependencies.jar download pictures xxx pic.tar

Delete archive with id xxx from vault pictures

java -jar glacier-1.0-jar-with-dependencies.jar delete pictures xxx

Get the inventory for vault pictures (takes >4 hours)

java -jar glacier-1.0-jar-with-dependencies.jar inventory pictures

Upload file1 and file2 to vault pictures in Europe region

java -jar glacier-1.0-jar-with-dependencies.jar -region eu-west-1 upload pictures file1 file2

12 thoughts on “Cheap backups with Amazon Glacier

  1. I can’t delete a vault, with a name that start with ‘-’

    Exception in thread “main” org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -iqBZYGe_UiABfQcOYeyyW5gz-poMIpJd-_Wr-…

    Thank you

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s