Upload Vesta user backups to Dropbox automatically daily Topic is solved

Section with additional software for Vesta
Forum rules
Before creating a new topic or reply on the forum you should fill out additional fields "Os" and "Web" in your profile section.
In case of violation, the topic can be closed or response from the support will not be received.
vesta_mtl
Posts: 46
Joined: Wed Dec 21, 2016 2:08 pm

Upload Vesta user backups to Dropbox automatically daily  Topic is solved

Postby vesta_mtl » Fri Mar 17, 2017 2:49 pm

I have setup my Centos 7 server to send the Vesta backups of all my websites (users) to Dropbox automatically every day (after Vesta has run the backup job). In case it helps anyone, here is how.

Configure Dropbox uploader

  1. Go here: https://www.dropbox.com/developers/apps
  2. Create a new Dropbox app and give it access to a new folder in the Apps directory (e.g. server_backups)
  3. Generate and copy your access token
  4. On your server, enter these commands (make a directory, navigate to it, download the dropbox upload bash script dropbox_uploader.sh, give it execution permissions, execute it).

Code: Select all

cd /
mkdir dropbox
cd dropbox
curl "https://raw.githubusercontent.com/andreafabrizi/Dropbox-Uploader/master/dropbox_uploader.sh" -o dropbox_uploader.sh
chmod 755 dropbox_uploader.sh
./dropbox_uploader.sh


Optional: Send a single file to Dropbox as a test (replace with your backup filename).

Code: Select all

/dropbox/dropbox_uploader.sh upload "/home/backup/admin.2017-03-17.tar" /



Schedule the backups to go to Dropbox daily

Make this new file:

Code: Select all

vi /usr/local/bin/send_site_backup_files_to_dropbox.sh


Add this to the file, then save it (type Esc, :wq):

Code: Select all

#!/bin/bash
#Save current date  as YYYY-MM-DD to a variable
DATE=$(date +"%Y-%m-%d")
#Loop through each file in the backup folder whose name has the current date
for X in /home/backup/*$DATE*; do
    #X is the filename with path. Remove path to get just the filename.
    NAME_NO_PATH=${X##*/}
    #Remove the date from the name (removes all text between the periods)
    NEW_NAME="${NAME_NO_PATH%%.*}.${NAME_NO_PATH##*.}"
    #Copy the file to tmp with the new non-dated name
    cp $X /tmp/$NEW_NAME
    #Send it to Dropbox
    /dropbox/dropbox_uploader.sh -f /root/.dropbox_uploader upload "/tmp/$NEW_NAME" /
    #Delete the file from tmp
    rm -rf /tmp/$NEW_NAME
done

Note: The backup file that is uploaded to Dropbox has the date removed from its filename so that every day, the new backup will overwrite the previous backup in Dropbox. This way you don't get an accumulation of backups (e.g. admin.2017-03-17.tar, admin.2017-03-18.tar, etc...) and always only have one backup file for each user (e.g. admin.tar) in Dropbox. Using Dropbox's file version history, you can access older backups of that user. The backups on the server are unchanged (they are not renamed, and they are kept for two days as per Vesta's normal behaviour).


Optional: Run it once as a test (your backups folder must have today's backups in it).

Code: Select all

sh /usr/local/bin/send_site_backup_files_to_dropbox.sh


Verify what time Vesta does its daily backups:
  1. Log into Vesta as admin
  2. Click Cron
  3. Find “sudo /usr/local/vesta/bin/v-backup-users” and click edit
  4. Verify what time Vesta does backups (change the time if desired)

Install moreutils (needed for "ts" command used in crontab):

Code: Select all

yum install moreutils


Edit crontab:

Code: Select all

crontab -e


Add this line to crontab so your backups are sent every morning at 3:45am (or any other time you want, just make sure it is at least 30 minutes after Vesta does the backups). Change the email address at the end to whatever address you wish to get the emailed report.

Code: Select all

45 03 * * * sh /usr/local/bin/send_site_backup_files_to_dropbox.sh | ts "[\%Y-\%m-\%d \%H:\%M:\%S]" 2>&1 | tee /var/log/send_backups_to_dropbox.log | mailx -s "Report for Vesta backup files sent to Dropbox" email@example.com


Optional: You can view the log here (but this will also get emailed to as per the previous step).

Code: Select all

cat /var/log/send_backups_to_dropbox.log


That is all, your backups will now be sent to Dropbox daily at the time you specified in crontab, and you will get an email report with the results.

I have successfully used this for files up to 10 Gb in size, and I think it should work fine for even larger. You can read more about how the Dropbox Uploader script does this, and other options it offers, here: https://github.com/andreafabrizi/Dropbox-Uploader
Last edited by vesta_mtl on Mon Jan 15, 2018 10:04 pm, edited 2 times in total.

skurudo
VestaCP Team
Posts: 7808
Joined: Fri Dec 26, 2014 2:23 pm
Contact:

Re: Upload Vesta user backups to Dropbox automatically daily

Postby skurudo » Mon Mar 20, 2017 6:55 am

Nice manual, vesta_mtl
Sticky post now.

Did you test this solution on big files like 1-2-3-4-5-10Gb or more?
-> DigitalOcean competition - please, support us
-> fix for phpmyadmin - nice and sweet now

vesta_mtl
Posts: 46
Joined: Wed Dec 21, 2016 2:08 pm

Re: Upload Vesta user backups to Dropbox automatically daily

Postby vesta_mtl » Mon Mar 20, 2017 12:34 pm

Thanks skurudo. Yes, I have uploaded files up to 10 Gb successfully. I have updated my post with this detail, and a link to the Dropbox Uploader script which explains how this is done, along with other options.

vikhyat
Posts: 70
Joined: Wed Sep 14, 2016 5:39 pm

Re: Upload Vesta user backups to Dropbox automatically daily

Postby vikhyat » Sun Apr 09, 2017 6:16 pm

Thanks a lot for this.

pandabb
Posts: 192
Joined: Sat Aug 08, 2015 3:03 am

Re: Upload Vesta user backups to Dropbox automatically daily

Postby pandabb » Tue May 02, 2017 1:35 pm

thank you will try this right away.

lpcs007
Posts: 5
Joined: Fri Dec 16, 2016 3:17 am

Re: Upload Vesta user backups to Dropbox automatically daily

Postby lpcs007 » Tue Oct 10, 2017 1:28 pm

vesta_mtl wrote:I have setup my Centos 7 server to send the Vesta backups of all my websites (users) to Dropbox automatically every day (after Vesta has run the backup job). In case it helps anyone, here is how.

Configure Dropbox uploader

  1. Go here: https://www.dropbox.com/developers/apps
  2. Create a new Dropbox app and give it access to a new folder in the Apps directory (e.g. server_backups)
  3. Generate and copy your access token
  4. On your server, enter these commands (make a directory, navigate to it, download the dropbox upload bash script dropbox_uploader.sh, give it execution permissions, execute it).

Code: Select all

cd /
mkdir dropbox
cd dropbox
curl "https://raw.githubusercontent.com/andreafabrizi/Dropbox-Uploader/master/dropbox_uploader.sh" -o dropbox_uploader.sh
chmod 755 dropbox_uploader.sh
./dropbox_uploader.sh


Optional: Send a single file to Dropbox as a test (replace with your backup filename).

Code: Select all

/dropbox/dropbox_uploader.sh upload "/home/backup/admin.2017-03-17.tar" /



Schedule the backups to go to Dropbox daily

Make this new file:

Code: Select all

vi /usr/local/bin/send_site_backup_files_to_dropbox.sh


Add this to the file, then save it (type Esc, :wq):

Code: Select all

#!/bin/bash

#Save current date  as YYYY-MM-DD to a variable
DATE=$(date +"%Y-%m-%d")

#Loop through each file in the backup folder whose name has the current date
for X in /home/backup/*$DATE*; do
    #X is the filename with path. Remove path to get just the filename.
    NAME_NO_PATH=${X##*/}
    #Remove period and date from filename
    NEW_NAME=${NAME_NO_PATH/.$DATE/}
    #Copy the file to tmp with the new non-dated name
    cp $X /tmp/$NEW_NAME
    #Send it to Dropbox
    /dropbox/dropbox_uploader.sh -f /root/.dropbox_uploader upload "/tmp/$NEW_NAME" /
    #Delete the file from tmp
    rm -rf /tmp/$NEW_NAME
done

Note: The backup file that is uploaded to Dropbox has the date removed from its filename so that every day, the new backup will overwrite the previous backup in Dropbox. This way you don't get an accumulation of backups (e.g. admin.2017-03-17.tar, admin.2017-03-18.tar, etc...) and always only have one backup file for each user (e.g. admin.tar) in Dropbox. Using Dropbox's file version history, you can access older backups of that user. The backups on the server are unchanged (they are not renamed, and they are kept for two days as per Vesta's normal behaviour).


Optional: Run it once as a test (your backups folder must have today's backups in it).

Code: Select all

sh /usr/local/bin/send_site_backup_files_to_dropbox.sh


Verify what time Vesta does its daily backups:
  1. Log into Vesta as admin
  2. Click Cron
  3. Find “sudo /usr/local/vesta/bin/v-backup-users” and click edit
  4. Verify what time Vesta does backups (change the time if desired)

Install moreutils (needed for "ts" command used in crontab):

Code: Select all

yum install moreutils


Edit crontab:

Code: Select all

crontab -e


Add this line to crontab so your backups are sent every morning at 3:45am (or any other time you want, just make sure it is at least 30 minutes after Vesta does the backups). Change the email address at the end to whatever address you wish to get the emailed report.

Code: Select all

45 03 * * * sh /usr/local/bin/send_site_backup_files_to_dropbox.sh | ts "[\%Y-\%m-\%d \%H:\%M:\%S]" 2>&1 | tee /var/log/send_backups_to_dropbox.log | mailx -s "Report for Vesta backup files sent to Dropbox" email@example.com


Optional: You can view the log here (but this will also get emailed to as per the previous step).

Code: Select all

cat /var/log/send_backups_to_dropbox.log


That is all, your backups will now be sent to Dropbox daily at the time you specified in crontab, and you will get an email report with the results.

I have successfully used this for files up to 10 Gb in size, and I think it should work fine for even larger. You can read more about how the Dropbox Uploader script does this, and other options it offers, here: https://github.com/andreafabrizi/Dropbox-Uploader


The backup worked perfectly. But I made some changes.

After installing, I created the file /usr/local/vesta/bin/backup-dropbox.sh (permission 0755):

Code: Select all

#!/bin/bash

for entry in "$search_dir"/backup/*
do
  /dropbox/dropbox_uploader.sh upload "$entry" /backup/
done

rm -rf /backup/*


So I created a cron on the vesta panel (user root/admin) at 7 o'clock.

Code: Select all

sudo /usr/local/vesta/bin/backup-dropbox.sh

mehargags
Moderators
Posts: 903
Joined: Sat Sep 06, 2014 9:58 pm
Contact:

Re: Upload Vesta user backups to Dropbox automatically daily

Postby mehargags » Tue Oct 10, 2017 3:54 pm

Interesting... I never knew Dropbox increased their limit of single file size to be 20GB. Previously it was only 2GB per file.
https://www.dropbox.com/help/space/upload-limitations

ruinzaO
Posts: 3
Joined: Fri Sep 22, 2017 11:05 am

Os: Ubuntu 16x
Web: apache + nginx

Re: Upload Vesta user backups to Dropbox automatically daily

Postby ruinzaO » Mon Jan 15, 2018 4:29 pm

Friend, thanks for the great work!
I noticed that my backups were added "admin.2018-01-15_05-10-12.tar" the time, so the file is not being renamed to admin.tar in the dropbox, could you add a fix in the script?

Many thanks and I'm sorry, I'm using google.

vesta_mtl
Posts: 46
Joined: Wed Dec 21, 2016 2:08 pm

Re: Upload Vesta user backups to Dropbox automatically daily

Postby vesta_mtl » Mon Jan 15, 2018 10:05 pm

ruinzaO wrote:Friend, thanks for the great work!
I noticed that my backups were added "admin.2018-01-15_05-10-12.tar" the time, so the file is not being renamed to admin.tar in the dropbox, could you add a fix in the script?

Many thanks and I'm sorry, I'm using google.


Yes, the script needed to be updated because Vesta now includes the time in the backup filename. I have updated my original post and the script should now work with the old Vesta (only yyyy-mm-dd in filename) and the new Vesta (yyyy-mm-dd_hh-mm-ss in the filename).

Let me know if you have any trouble. Your Google translation is excellent! :-)

ruinzaO
Posts: 3
Joined: Fri Sep 22, 2017 11:05 am

Os: Ubuntu 16x
Web: apache + nginx

Re: Upload Vesta user backups to Dropbox automatically daily

Postby ruinzaO » Tue Jan 16, 2018 11:59 am

vesta_mtl wrote:
ruinzaO wrote:Friend, thanks for the great work!
I noticed that my backups were added "admin.2018-01-15_05-10-12.tar" the time, so the file is not being renamed to admin.tar in the dropbox, could you add a fix in the script?

Many thanks and I'm sorry, I'm using google.


Yes, the script needed to be updated because Vesta now includes the time in the backup filename. I have updated my original post and the script should now work with the old Vesta (only yyyy-mm-dd in filename) and the new Vesta (yyyy-mm-dd_hh-mm-ss in the filename).

Let me know if you have any trouble. Your Google translation is excellent! :-)


Thanks for the answer!
Friend in setup this error appears.
/dropbox/dropbox_uploader.sh: line 1559: : No such file or directory

Am I doing something wrong?


Return to “3rd Party Software”



Who is online

Users browsing this forum: No registered users and 2 guests

cron