Upload Vesta user backups to Dropbox automatically daily
Upload Vesta user backups to Dropbox automatically daily
I have setup my Centos 7 server to send the Vesta backups of all my websites (users) to Dropbox automatically every day (after Vesta has run the backup job). In case it helps anyone, here is how.
Configure Dropbox uploader
Optional: Send a single file to Dropbox as a test (replace with your backup filename).
Schedule the backups to go to Dropbox daily
Make this new file:
Add this to the file, then save it (type Esc, :wq):
Note: The backup file that is uploaded to Dropbox has the date removed from its filename so that every day, the new backup will overwrite the previous backup in Dropbox. This way you don't get an accumulation of backups (e.g. admin.2017-03-17.tar, admin.2017-03-18.tar, etc...) and always only have one backup file for each user (e.g. admin.tar) in Dropbox. Using Dropbox's file version history, you can access older backups of that user. The backups on the server are unchanged (they are not renamed, and they are kept for two days as per Vesta's normal behaviour).
Optional: Run it once as a test (your backups folder must have today's backups in it).
Verify what time Vesta does its daily backups:
Edit crontab:
Add this line to crontab so your backups are sent every morning at 3:45am (or any other time you want, just make sure it is at least 30 minutes after Vesta does the backups). Change the email address at the end to whatever address you wish to get the emailed report.
Optional: You can view the log here (but this will also get emailed to as per the previous step).
That is all, your backups will now be sent to Dropbox daily at the time you specified in crontab, and you will get an email report with the results.
I have successfully used this for files up to 10 Gb in size, and I think it should work fine for even larger. You can read more about how the Dropbox Uploader script does this, and other options it offers, here: https://github.com/andreafabrizi/Dropbox-Uploader
Configure Dropbox uploader
- Go here: https://www.dropbox.com/developers/apps
- Create a new Dropbox app and give it access to a new folder in the Apps directory (e.g. server_backups)
- Generate and copy your access token
- On your server, enter these commands (make a directory, navigate to it, download the dropbox upload bash script dropbox_uploader.sh, give it execution permissions, execute it) and paste your token when prompted to.
Code: Select all
cd /
mkdir dropbox
cd dropbox
curl "https://raw.githubusercontent.com/andreafabrizi/Dropbox-Uploader/master/dropbox_uploader.sh" -o dropbox_uploader.sh
chmod 755 dropbox_uploader.sh
./dropbox_uploader.sh
Code: Select all
/dropbox/dropbox_uploader.sh upload "/home/backup/admin.2017-03-17.tar" /
Schedule the backups to go to Dropbox daily
Make this new file:
Code: Select all
vi /usr/local/bin/send_site_backup_files_to_dropbox.sh
Code: Select all
#!/bin/bash
#Save current date as YYYY-MM-DD to a variable
DATE=$(date +"%Y-%m-%d")
#Loop through each file in the backup folder whose name has the current date
for X in /home/backup/*$DATE*; do
#X is the filename with path. Remove path to get just the filename.
NAME_NO_PATH=${X##*/}
#Remove the date from the name (removes all text between the periods)
NEW_NAME="${NAME_NO_PATH%%.*}.${NAME_NO_PATH##*.}"
#Copy the file to tmp with the new non-dated name
cp $X /tmp/$NEW_NAME
#Send it to Dropbox
/dropbox/dropbox_uploader.sh -f /root/.dropbox_uploader upload "/tmp/$NEW_NAME" /
#Delete the file from tmp
rm -rf /tmp/$NEW_NAME
done
Optional: Run it once as a test (your backups folder must have today's backups in it).
Code: Select all
sh /usr/local/bin/send_site_backup_files_to_dropbox.sh
- Log into Vesta as admin
- Click Cron
- Find “sudo /usr/local/vesta/bin/v-backup-users” and click edit
- Verify what time Vesta does backups (change the time if desired)
Code: Select all
yum install moreutils
Code: Select all
crontab -e
Code: Select all
45 03 * * * sh /usr/local/bin/send_site_backup_files_to_dropbox.sh | ts "[\%Y-\%m-\%d \%H:\%M:\%S]" 2>&1 | tee /var/log/send_backups_to_dropbox.log | mailx -s "Report for Vesta backup files sent to Dropbox" [email protected]
Code: Select all
cat /var/log/send_backups_to_dropbox.log
I have successfully used this for files up to 10 Gb in size, and I think it should work fine for even larger. You can read more about how the Dropbox Uploader script does this, and other options it offers, here: https://github.com/andreafabrizi/Dropbox-Uploader
Last edited by vesta_mtl on Mon Jan 22, 2018 11:52 am, edited 6 times in total.
Re: Upload Vesta user backups to Dropbox automatically daily
Nice manual, vesta_mtl
Sticky post now.
Did you test this solution on big files like 1-2-3-4-5-10Gb or more?
Sticky post now.
Did you test this solution on big files like 1-2-3-4-5-10Gb or more?
Re: Upload Vesta user backups to Dropbox automatically daily
Thanks skurudo. Yes, I have uploaded files up to 10 Gb successfully. I have updated my post with this detail, and a link to the Dropbox Uploader script which explains how this is done, along with other options.
Re: Upload Vesta user backups to Dropbox automatically daily
Thanks a lot for this.
Re: Upload Vesta user backups to Dropbox automatically daily
thank you will try this right away.
Re: Upload Vesta user backups to Dropbox automatically daily
The backup worked perfectly. But I made some changes.vesta_mtl wrote:I have setup my Centos 7 server to send the Vesta backups of all my websites (users) to Dropbox automatically every day (after Vesta has run the backup job). In case it helps anyone, here is how.
Configure Dropbox uploader
- Go here: https://www.dropbox.com/developers/apps
- Create a new Dropbox app and give it access to a new folder in the Apps directory (e.g. server_backups)
- Generate and copy your access token
- On your server, enter these commands (make a directory, navigate to it, download the dropbox upload bash script dropbox_uploader.sh, give it execution permissions, execute it).
Optional: Send a single file to Dropbox as a test (replace with your backup filename).Code: Select all
cd / mkdir dropbox cd dropbox curl "https://raw.githubusercontent.com/andreafabrizi/Dropbox-Uploader/master/dropbox_uploader.sh" -o dropbox_uploader.sh chmod 755 dropbox_uploader.sh ./dropbox_uploader.sh
Code: Select all
/dropbox/dropbox_uploader.sh upload "/home/backup/admin.2017-03-17.tar" /
Schedule the backups to go to Dropbox daily
Make this new file:Add this to the file, then save it (type Esc, :wq):Code: Select all
vi /usr/local/bin/send_site_backup_files_to_dropbox.sh
Note: The backup file that is uploaded to Dropbox has the date removed from its filename so that every day, the new backup will overwrite the previous backup in Dropbox. This way you don't get an accumulation of backups (e.g. admin.2017-03-17.tar, admin.2017-03-18.tar, etc...) and always only have one backup file for each user (e.g. admin.tar) in Dropbox. Using Dropbox's file version history, you can access older backups of that user. The backups on the server are unchanged (they are not renamed, and they are kept for two days as per Vesta's normal behaviour).Code: Select all
#!/bin/bash #Save current date as YYYY-MM-DD to a variable DATE=$(date +"%Y-%m-%d") #Loop through each file in the backup folder whose name has the current date for X in /home/backup/*$DATE*; do #X is the filename with path. Remove path to get just the filename. NAME_NO_PATH=${X##*/} #Remove period and date from filename NEW_NAME=${NAME_NO_PATH/.$DATE/} #Copy the file to tmp with the new non-dated name cp $X /tmp/$NEW_NAME #Send it to Dropbox /dropbox/dropbox_uploader.sh -f /root/.dropbox_uploader upload "/tmp/$NEW_NAME" / #Delete the file from tmp rm -rf /tmp/$NEW_NAME done
Optional: Run it once as a test (your backups folder must have today's backups in it).Verify what time Vesta does its daily backups:Code: Select all
sh /usr/local/bin/send_site_backup_files_to_dropbox.sh
Install moreutils (needed for "ts" command used in crontab):
- Log into Vesta as admin
- Click Cron
- Find “sudo /usr/local/vesta/bin/v-backup-users” and click edit
- Verify what time Vesta does backups (change the time if desired)
Edit crontab:Code: Select all
yum install moreutils
Add this line to crontab so your backups are sent every morning at 3:45am (or any other time you want, just make sure it is at least 30 minutes after Vesta does the backups). Change the email address at the end to whatever address you wish to get the emailed report.Code: Select all
crontab -e
Optional: You can view the log here (but this will also get emailed to as per the previous step).Code: Select all
45 03 * * * sh /usr/local/bin/send_site_backup_files_to_dropbox.sh | ts "[\%Y-\%m-\%d \%H:\%M:\%S]" 2>&1 | tee /var/log/send_backups_to_dropbox.log | mailx -s "Report for Vesta backup files sent to Dropbox" [email protected]
That is all, your backups will now be sent to Dropbox daily at the time you specified in crontab, and you will get an email report with the results.Code: Select all
cat /var/log/send_backups_to_dropbox.log
I have successfully used this for files up to 10 Gb in size, and I think it should work fine for even larger. You can read more about how the Dropbox Uploader script does this, and other options it offers, here: https://github.com/andreafabrizi/Dropbox-Uploader
After installing, I created the file /usr/local/vesta/bin/backup-dropbox.sh (permission 0755):
Code: Select all
#!/bin/bash
for entry in "$search_dir"/backup/*
do
/dropbox/dropbox_uploader.sh upload "$entry" /backup/
done
rm -rf /backup/*
Code: Select all
sudo /usr/local/vesta/bin/backup-dropbox.sh
-
- Support team
- Posts: 1096
- Joined: Sat Sep 06, 2014 9:58 pm
- Contact:
- Os: Debian 8x
- Web: apache + nginx
Re: Upload Vesta user backups to Dropbox automatically daily
Interesting... I never knew Dropbox increased their limit of single file size to be 20GB. Previously it was only 2GB per file.
https://www.dropbox.com/help/space/upload-limitations
https://www.dropbox.com/help/space/upload-limitations
Re: Upload Vesta user backups to Dropbox automatically daily
Friend, thanks for the great work!
I noticed that my backups were added "admin.2018-01-15_05-10-12.tar" the time, so the file is not being renamed to admin.tar in the dropbox, could you add a fix in the script?
Many thanks and I'm sorry, I'm using google.
I noticed that my backups were added "admin.2018-01-15_05-10-12.tar" the time, so the file is not being renamed to admin.tar in the dropbox, could you add a fix in the script?
Many thanks and I'm sorry, I'm using google.
Re: Upload Vesta user backups to Dropbox automatically daily
Yes, the script needed to be updated because Vesta now includes the time in the backup filename. I have updated my original post and the script should now work with the old Vesta (only yyyy-mm-dd in filename) and the new Vesta (yyyy-mm-dd_hh-mm-ss in the filename).ruinzaO wrote:Friend, thanks for the great work!
I noticed that my backups were added "admin.2018-01-15_05-10-12.tar" the time, so the file is not being renamed to admin.tar in the dropbox, could you add a fix in the script?
Many thanks and I'm sorry, I'm using google.
Let me know if you have any trouble. Your Google translation is excellent! :-)
Re: Upload Vesta user backups to Dropbox automatically daily
Thanks for the answer!vesta_mtl wrote:Yes, the script needed to be updated because Vesta now includes the time in the backup filename. I have updated my original post and the script should now work with the old Vesta (only yyyy-mm-dd in filename) and the new Vesta (yyyy-mm-dd_hh-mm-ss in the filename).ruinzaO wrote:Friend, thanks for the great work!
I noticed that my backups were added "admin.2018-01-15_05-10-12.tar" the time, so the file is not being renamed to admin.tar in the dropbox, could you add a fix in the script?
Many thanks and I'm sorry, I'm using google.
Let me know if you have any trouble. Your Google translation is excellent! :-)
Friend in setup this error appears.
/dropbox/dropbox_uploader.sh: line 1559: : No such file or directory
Am I doing something wrong?