Page 1 of 1

Automatically upload users (and admin) backup files to Google Drive

Posted: Tue Apr 04, 2017 2:14 am
by batmanu
LE I edited the script adding

Code: Select all

exit 0
as it would otherwise have been left temporary files on your partition.

I managed to write a script to automatically upload my backups to my Google Drive. Obviously, with Google Drive app installed, that means that automatically synced files will end up on my home machine also, which I find very convenient ;)

So, here it is (any feedback would be great, as I am kind of a newbie when it comes to bash scripting):
  1. First of all, you have to get the gdrive client, specifically the binary for your OS (gdrive-linux-x64 for me, as I'm on CentOS 7):

    Code: Select all

    wget https://docs.google.com/uc?id=0B3X9GlR6EmbnQ0FtZmJJUXEyRTA&export=download
    make it executable:

    Code: Select all

    chmod +x gdrive
    and place it somewhere from where you can call it easily:

    Code: Select all

    mv gdrive /usr/sbin/
    Then, set it up:

    Code: Select all

    gdrive about
    Follow the link and get the verification code by authenticating with the google account for the drive you want access to.
    --
  2. Now, let's rock!

    Code: Select all

    #!/bin/bash
    IFS=$'\n'
    
    # If you're short on Drive space, you may want to delete previous backups
    upTime=$(date --date="23 hours ago" '+%Y-%m-%dT%T')
    gdrive list -q "name contains '.enc' and modifiedTime < '$upTime'" >> drive.txt
    filename='drive.txt'
    # Read the list file
    while read -r line
    do
       theId=$(cut -c-2 <<< "$line")
       #basically skip the first line...
       if [ "$theId" != "Id" ]; then
            #send the delete command
            gdrive delete $(cut -c-28 <<< "$line")
       fi
    done < $filename
    # Remove temp list file
    rm -rf drive.txt
    
    # For every user you have:
    for USERINFO in `grep ":/home" /etc/passwd`
    do
     USERNAME=$(echo $USERINFO | cut -d: -f1)
     
     # Declare the variable that will give you the backup filename
     BAKFILE=$(echo "/home/backup/$USERNAME.$(date '+%Y-%m-%d').tar")
     
     # You will upload an encrypted backup to Drive, so, let's call that file a name
     ENCFILE=$(echo "/tmp/$USERNAME.$(date '+%Y-%m-%d').enc")
     
     # Skip users that don't have backups, like user 'backup' or additional ftp users (don't forget to change the username accordingly)
     if [ "$USERNAME" == "backup" ] || [ "$USERNAME" == "ADD_FTP_USER" ] ; then
      continue;
     fi
     if [ -e "$BAKFILE" ] ; then
      # If the file exists, encrypt it (don't forget to replace the password below with a strong and long password)
      openssl aes-256-cbc -a -salt -in "$BAKFILE" -out "$ENCFILE" -pass 'pass:LONGSTRONGPASSWORDHERE'
      
      # Copy the Drive folder ID where you'll upload your files (the after the https://drive.google.com/.../folders/...). Upload the encrypted file, then delete it from machine.
      gdrive upload -p ID --delete "$ENCFILE"
     fi
    done
    
    exit 0
    
    That is, in fact, the script. Put it in a file (/usr/local/sbin/backupgdrive.sh) and make it executable by root.

    Now, some more automation would be nice, right?
    --
  3. You might experience some short downtime while the encryption takes place (I'm not quite sure on that), so I would set up the cron job late at night. I created a job in /etc/cron.d:

    Code: Select all

    vi /etc/cron.d/backup-gdrive
    Paste the following

    Code: Select all

    MAILTO="" /* You want output mail sent by script, not by crontab */
    15 05 *  *  * root /usr/local/sbin/backupgdrive.sh > output ; mail -s "Backup files uploaded to Google Drive" -r root YOU@YOURDOMAIN.COM < output
    Make sure that the script will be executed at about an hour AFTER Vesta's automatic backup. You will receive an email after the upload is finished, with a summary of the script's output.
And that's about it!

If you ever need to restore the backups (hopefully not), you will have to upload the encrypted files in your /home/backup directory and decrypt them one by one (or make a script :P):

Code: Select all

openssl aes-256-cbc -d -a -in "/home/backup/USERNAME.DATE.enc" -out "/home/backup/USERNAME.DATE.tar" -pass 'pass:YOURLONGSTRONGPASSWORD'
then chown accordingly:

Code: Select all

chown admin:USERNAME /home/backup/USERNAME.DATE.tar
Check that everything is OK, then remove the encrypted files.


Credits go to:
Timothy Quinn (https://timothy-quinn.com/backing-up-fi ... a-the-cli/)
Benjamin Cane (http://bencane.com/2013/10/21/5-bash-fo ... efficient/)
and again Tim (https://timothy-quinn.com/backing-up-li ... gle-drive/)

Re: Automatically upload users (and admin) backup files to Google Drive

Posted: Sat Apr 08, 2017 9:02 pm
by vikhyat
Thanks a lot for this script. I will test it tomorrow and give a feedback.

Re: Automatically upload users (and admin) backup files to Google Drive

Posted: Fri May 05, 2017 10:53 am
by MistaWongX
nice. was seeking answers to the same question. got redirected from another one of the threads

Re: Automatically upload users (and admin) backup files to Google Drive

Posted: Wed Jan 03, 2018 10:42 am
by rhyker2u
thats one way to do things, here's another (which I found out about yesterday due to VestaCP migrations to a new server farm): http://crossftp.com/commander.htm
CrossFTP Commander is a command line tool based on CrossFTP engine to handle data transfer, sync, and backup operations. It has small memory footprint (core library size is about 4MB), and can be integrated in the shell script or system scheduler. The supported protocols include FTP, SFTP, FTPS, WebDav, Amazon S3, and Google Storage protocols.
However will definitely checkout your solution too.

Re: Automatically upload users (and admin) backup files to Google Drive

Posted: Fri Jan 26, 2018 8:13 pm
by blockmarc
Hi batmanu,

Thank you for your post and script.
Since VestaCP backups have been amended with the new VestaCP version could you please give a hint how to update your script, please ?

I quess following lines must be changed ?

from

Code: Select all

upTime=$(date --date="23 hours ago" '+%Y-%m-%dT%T')
to

Code: Select all

upTime=$(date --date="23 hours ago" '+%Y-%m-%dT_%H-%M-%S%T')
?

from

Code: Select all

BAKFILE=$(echo "/home/backup/$USERNAME.$(date '+%Y-%m-%d').tar")
to

Code: Select all

BAKFILE=$(echo "/backup/$USERNAME.$(date '+%Y-%m-%d_%H-%M-%S').tar")
?

from

Code: Select all

ENCFILE=$(echo "/tmp/$USERNAME.$(date '+%Y-%m-%d').enc")
to

Code: Select all

ENCFILE=$(echo "/tmp/$USERNAME.$(date '+%Y-%m-%d_%H-%M-%S').enc")
?

Anything else?
Many thanks in advance for your help !

Re: Automatically upload users (and admin) backup files to Google Drive

Posted: Thu Sep 27, 2018 7:41 pm
by zaka786khan
The following code is fully working with no error on my VPS server Centos 7 vestaCP (backup only admin):

This is my /etc/cron.d/backup-gdrive file {Remember I have install gdrive on my VPS} and I have scheduled {sudo /etc/cron.d/backup-gdrive} under my VestaCP CRON

Code: Select all

#!/bin/bash
IFS=$'\n'

# If you're short on Drive space, you may want to delete previous backups
upTime=$(date --date="5 days ago" '+%Y-%m-%dT_%H-%M-%S%T')
gdrive list -q "name contains 'admin.201' and modifiedTime < '$upTime'" >> drive.txt
filename='drive.txt'
# Read the list file
while read -r line
do
   theId=$(cut -c-2 <<< "$line")
   #basically skip the first line...
   if [ "$theId" != "Id" ]; then
        #send the delete command
        gdrive delete $(cut -c-28 <<< "$line")
   fi
done < $filename
# Remove temp list file
rm -rf drive.txt

FPATH="/home/backup/admin.$(date '+%Y-%m-%d')*.tar"
BAKFILE=`ls ${FPATH}`
# {ID} of My Google Drive is replaced bellow
gdrive upload -p {ID} "$BAKFILE"

exit 0
and The following code is fully working with no error on my VPS server Centos 7 vestaCP (backup all users including admin):

Code: Select all

#!/bin/bash
IFS=$'\n'

# If you're short on Drive space, you may want to delete previous backups
upTime=$(date --date="5 days ago" '+%Y-%m-%dT_%H-%M-%S%T')
gdrive list -q "name contains 'admin.201' and modifiedTime < '$upTime'" >> drive.txt
filename='drive.txt'
# Read the list file
while read -r line
do
   theId=$(cut -c-2 <<< "$line")
   #basically skip the first line...
   if [ "$theId" != "Id" ]; then
        #send the delete command
        gdrive delete $(cut -c-28 <<< "$line")
   fi
done < $filename
# Remove temp list file
rm -rf drive.txt

# For every user you have:
for USERINFO in `grep ":/home" /etc/passwd`
do
 USERNAME=$(echo $USERINFO | cut -d: -f1)
 # Declare the variable that will give you the backup filename
 FPATH="/home/backup/$USERNAME.$(date '+%Y-%m-%d')*.tar"
 BAKFILE=`ls ${FPATH}`
 
 # You will upload an encrypted backup to Drive, so, let's call that file a name
 # ENCFILE=$(echo "/tmp/$USERNAME.$(date '+%Y-%m-%d').enc")
 
 # Skip users that don't have backups, like user 'backup' or additional ftp users (don't forget to change the username accordingly)
 if [ "$USERNAME" == "backup" ] || [ "$USERNAME" == "ADD_FTP_USER" ] ; then
  continue;
 fi
 if [ -e "$BAKFILE" ] ; then
  # If the file exists, encrypt it (don't forget to replace the password below with a strong and long password)
  # openssl aes-256-cbc -a -salt -in "$BAKFILE" -out "$ENCFILE" -pass 'pass:LONGSTRONGPASSWORDHERE'
  
  # Copy the Drive folder ID where you'll upload your files (the after the https://drive.google.com/.../folders/...). Upload the encrypted file, then delete it from machine.
  gdrive upload -p {ID} "$BAKFILE"
 fi
done
exit 0
I didn't used encryption

Re: Automatically upload users (and admin) backup files to Google Drive

Posted: Sun Dec 16, 2018 5:08 pm
by alanwan
Hello All,

I based on above method to install and setup. And I can receive the Backup Email Notification, but I can see any backup file(s)on my Google Driver.

Many thanks

Re: Automatically upload users (and admin) backup files to Google Drive

Posted: Thu Sep 30, 2021 9:44 am
by Awe3423
Thank you for sharing, followed the instruction above and got it working.

Regards,
ROMAN
RIgs Trucking

Re: Automatically upload users (and admin) backup files to Google Drive

Posted: Sat Nov 13, 2021 1:39 am
by bulletproofhosting
Thank you for sharing, it works.

Re: Automatically upload users (and admin) backup files to Google Drive

Posted: Fri Nov 19, 2021 1:41 am
by jhonny022117
Thank you for sharing! I like it a lot and it works really fine! Thanks! concrete driveways