Git Backups with S3

Git Backups with S3

Introduction

Amazon S3 Backup is the most reliable, fast and simple to use solution for keeping the data backed up online. S3 achieves high availability by replicating data across multiple servers within Amazon’s data centers, and offers 99.999999999% durability. S3 typical use cases involve media sharing, media distribution, Server/PC backup, online storage, and application storage. This blog shows how we backup our code base to S3 twice a day using different interesting tools like S3cmd, and S3 browser.

Use Case

This use case shows how we backup our bare git repository to S3 twice a day using different tools like S3cmd, and S3 browser. This use case is kept intentionally simple to demonstrate S3.

What we want to do:

  • Ensure Python is set up
  • Install & Configure S3cmd tools
  • Download & Configure S3 browser
  • Create a sample bare git repository
  • Create a script to upload the repository to S3

Solution

Ensure Python is set up:

  • There are so many blogs and articles explaining about how to install Python, and the links are referenced below. For this use case, we use Ubuntu and Python 2.7.5

http://askubuntu.com/questions/101591/how-do-i-install-python-2-7-2-on-ubuntu

http://heliumhq.com/docs/installing_python_2.7.5_on_ubuntu

Install & Configure S3cmd tools:

  • Install S3cmd:
  • Configure S3Cmd:
  • Verify S3Cmd:

Download & Configure S3 browser:

  • Download S3browser:

Download and Install S3browser from http://s3browser.com/download/s3browser-4-1-1.exe

  • Configure S3browser:

Specify Amazon access and secret key along with any unique account name as shown in the below screenshot

Amazon access and secret key

  • Verify S3browser:

The below screenshot shows a sample S3 bucket with name “backupdemo” created for this blog purpose that doesn’t have any objects yet.

backupdemo

Create a sample bare git repository:

  • Create a bare blessed repository
  • Create a sample working directory and push the master to blessed repo

Create a script to upload the repository to S3 :

  • Backup Script:
  • Execute the script to upload the repo to S3:
  • Verify:

The blessed repo is uploaded in compressed format to the S3 bucket ‘backupdemo”

S3 bucket backupdemo

Conclusion

  • Amazon S3 is the popular cloud storage solution that can be used to backup any type of files and comes with various features like Access Control List to buckets, Lifecycle configuration, Server Side Encryption and so on.
  • Tools like S3cmd and S3browser come in handy to automate and verify many S3 activities similar to what was done above.

References

 

4172 Views 5 Views Today