Page 1 of 2

Tips for backing up to Amazon's S3 Service

Posted: Tue Jul 28, 2015 6:36 pm
by missionaryman
Hi all,

I love this forum! I find loads of answers here, and people are pretty-good at answering my (sometimes dumb) questions, so I thought I'd share back with something I've learned recently. How to backup your backups to Amazon's S3:

1) Obviously, get yourself an S3 account (free for the first year!) and get your Access Key Id and Access Key. Click here for more details: http://aws.amazon.com/
2) On S3, make a unique bucket (folder) name for your backups. For this example, I'll call mine vesta-backups.
3) Install s3cmd (website here: http://s3tools.org/s3cmd) on your server. I use Ubuntu, so the command is sudo apt-get install s3cmd.
REALLY IMPORTANT PART:
4) You'll need to configure s3cmd so that it works with the VestaCP admin account. Either make sure you've SSHed in to your server as admin, or su admin to ensure you're configuring the right user. VestaCP turns off SSH access for admin by default, so be sure to edit the admin user settings to allow SSH access, even if just for this operation. If you configure the wrong user, the backup process will not work. Be warned!
5) type s3cmd --configure and enter all the requested details. You'll need to enter your S3 credentials and these are long! use <shift><insert> to paste into a terminal window.
6) s3cmd will test the connection and confirm success.
7) In /home/admin, create the following file: cron-s3sync.txt and past the following in:

Code: Select all

# SYNC FILES
#!/bin/bash
s3cmd sync  --delete-removed -m "application/x-tar" /home/backup/ s3://vesta-backups/ >/dev/null 2>&1
8) Obviously, make sure the s3://vesta-backups is the name of your created S3 bucket. All the code from > onwards stops any reporting. Remove this and you'll receive emails regarding the process, either success or failure.
9) change the attributes of the cron-sync.txt to allow file execution: chmod 755 cron-sync.txt
10) Finally, create a CRON job in VestaCP that points to /home/admin/cron-s3sync.txt

When testing, remove the code from > to allow email reporting on success or failure, then change the CRON time to your current time +1 minute and check that everything backs up correctly. The actual code syncs the backup folder on VestaCP with S3 rather than just adding the latest backup. If you wanted different functionality, I'm sure you can figure it out.

Hope that helps!

Adam

Re: Tips for backing up to Amazon's S3 Service

Posted: Wed Jul 29, 2015 2:08 pm
by skurudo
Nice manual, good work! ;-)

I have indiscreet intimate question - how much is the fish? ;-) In reality how much s3 cost for user? xxx gb = price.

Re: Tips for backing up to Amazon's S3 Service

Posted: Thu Jul 30, 2015 7:29 am
by tjebbeke
skurudo wrote:Nice manual, good work! ;-)

I have indiscreet intimate question - how much is the fish? ;-) In reality how much s3 cost for user? xxx gb = price.
http://aws.amazon.com/s3/pricing/

0.01$ - 0.03$ /GB /month

Re: Tips for backing up to Amazon's S3 Service

Posted: Thu Jul 30, 2015 8:54 am
by missionaryman
From http://aws.amazon.com/s3/pricing/
AWS Free Usage Tier*

As part of the AWS Free Usage Tier, you can get started with Amazon S3 for free. Upon sign-up, new AWS customers receive 5 GB of Amazon S3 standard storage, 20,000 Get Requests, 2,000 Put Requests, and 15GB of data transfer out each month for one year.

Re: Tips for backing up to Amazon's S3 Service

Posted: Sat Aug 08, 2015 9:54 am
by pandabb
Hello Sir, Thanks for this tutorial.

What if i want to backup multiple different users? My Admin account has no webdomains i created seperate users for each domain.

Re: Tips for backing up to Amazon's S3 Service

Posted: Sat Aug 08, 2015 9:57 am
by missionaryman
This tutorial shows you how to copy the VestaCP backups generated by the control panel to Amazon's S3 service. It doesn't cover how to configure the VestaCP backup service.

Re: Tips for backing up to Amazon's S3 Service

Posted: Sat Aug 08, 2015 12:21 pm
by hafeezksa
Is there way to keep backup to Dropbox?

Re: Tips for backing up to Amazon's S3 Service

Posted: Tue Aug 11, 2015 9:04 am
by skurudo

Re: Tips for backing up to Amazon's S3 Service

Posted: Sun Feb 07, 2016 1:06 pm
by asixja
One question.
I have successfully installed all the s3.
I have to put in the SSH panel controls as follows:

Code: Select all

s3cmd sync --recursive/home/backup/ s3://vesta-backup
successfully migrate files on Amazon

I tried to add s3cmd Cronn direct command

Code: Select all

s3cmd sync --recursive/home/backup/ s3://vesta-backup
and I put refresh every minute (that I tested)

And then the email I keep getting this error:

Code: Select all

ERROR: /home/admin/.s3cfg: None
ERROR: Configuration file not available.
ERROR: Consider using --configure parameter to create one.
I tried to create a file according to your instructions "cron-s3sync.txt" and I put it in the /home/admin/ and in Cron put this

Code: Select all

/home/admin/cron-s3sync.txt
But then I get the same error:

Code: Select all

ERROR: /home/admin/.s3cfg: None
ERROR: Configuration file not available.
ERROR: Consider using --configure parameter to create one.
I tried that and I bet .s3cfg directly to "/home/admin/" but then does not work Cron

What do you suggest to me?

Re: Tips for backing up to Amazon's S3 Service

Posted: Thu Jan 12, 2017 5:04 pm
by Nacho
Hi guys, do you have info about backups to amazon glacier, supposed to be a lot cheaper and now you can get files quicker than before. Thanks in advance.