Automatic remote backup of Toon files

Everything about software tools, new software development and toolchains. For developers, mostly.

Moderators: marcelr, TheHogNL, TerrorSource, Toonz

Automatic remote backup of Toon files

Postby Toonz » Sun Jan 28, 2018 12:20 am

Hi all,

I would like to share my latest 'project': automatic backup of Toon data.
With the setup below Toon will make a copy of the /HCBv2 and the /mnt folders (containing the rra databases for instance) on a remote server.
In my case I use a Synology NAS to store the backup. Data is transferred via the rsync tool via scheduled incremental backups.
This will work with any other remote server running a rsync service.
Because the rsync protocol by defaults asks for a password (which you don't want in a batch job obviously) I am using ssh keys to login on the remote server.

Installation instructions (some experience with shells, editors is required):

1. Create an SSH keypair on the Synology NAS:
I followed this guide to do this: https://forum.synology.com/enu/viewtopic.php?t=126166

2. Create an SSH keypair on Toon:
login to your Toon as root
create a keypair by issuing these commands:
Code: Select all
mkdir .ssh
dropbearkey -t rsa -f ~/.ssh/id_dropbear

in the console window the corresponding public key is shown (string starting with "ssh - rsa...."
append this public key string to the authorized_keys file on the Synology NAS which you created in step 1

3. Install rsync on Toon
install the attached packages on Toon:
Code: Select all
opkg install popt_1.14-r3.0_qb2.ipk
opkg install rsync_3.0.6-r0_qb2.ipk


4. edit the attached rsyncjob.sh and replace the destination with your target server address

5. make the file rsyncjob.sh executable if not already.

6. test by running manually:
Code: Select all
./rsyncjob.sh


7. create a cron job to execute frequently (I run it once per day)
Only new/changed files will be copied across (rra databases mainly and changed config files)

Hope you like it. All feedback is welcome of course.

Kind regardz,

Toonz
Attachments
BackupFiles.zip
(208.57 KiB) Downloaded 93 times
member of the Toon Software Collective
User avatar
Toonz
Forum Moderator
Forum Moderator
 
Posts: 1088
Joined: December 2016

Re: Automatic remote backup of Toon files

Postby flamingbob » Thu Feb 01, 2018 12:55 pm

Thanks! That is a good way to store your toon. Maybe will patch it a bit also that rsync is done with ssh tunnels. Thank you for this good solution!
flamingbob
Starting Member
Starting Member
 
Posts: 21
Joined: November 2016

Re: Automatic remote backup of Toon files

Postby michel30 » Sat Feb 03, 2018 2:59 pm

Hello,

I am a litle bit lost here :oops:

Here is I am lost on my NAS:

Code: Select all

ssh-keygen
Generate public and private keys

Press “enter” to accept default file location (should be user’s .ssh folder)
Press “enter” twice to indicate NOT to create passphrase. (The passphrase would prevent the login from working when used by rsync.)
Will add files id_rsa (private key) and id_rsa.pub (public key)

These should automatically be created with the correct permission (read/write by user only, i.e. chmod 600). You shouldn’t need to make any changes.
Configure the Synology’s SSH service to allow login by key

Go to an SSH session (Either an administrator or an SSH user. You may still have one open.)
cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak
makes a backup copy of the config file, just in case something goes wrong
sudo vi /etc/ssh/sshd_config


I have a key created on my NAS but now I am lost.

When I login on my TOON: and I follow the steps from TOONZ

2. Create an SSH keypair on Toon:
login to your Toon as root
create a keypair by issuing these commands:

mkdir .ssh
dropbearkey -t rsa -f ~/.ssh/id_dropbear

And here is where I am lost::::::
Code: Select all
in the console window the corresponding public key is shown (string starting with "ssh - rsa...."
append this public key string to the authorized_keys file on the Synology NAS which you created in step 1



I see the an key but it is compete diffrent than on my NAS because it generate is own key plus one is saying sha256 the other not.
Maybe I did someting wrong but how I can link these two keys ?
michel30
Member
Member
 
Posts: 199
Joined: August 2017

Re: Automatic remote backup of Toon files

Postby Toonz » Sat Feb 03, 2018 3:43 pm

When you execute the dropbearkey command on Toon , you get an output like this int he console window:
eneco-001-xxxxx:~# dropbearkey -t rsa -f ~/.ssh/test
Generating key, this may take a while...
Public key portion is:
ssh-rsa AAAAB3NzaC1yc2EAAAADikhebhierietsveranderdblablablablablablab48l3gEoCzz1kZ1JlGO1446C5ZUnUmBNBRc52+ozYLOWTtS8NeB+wHFhy2zIP1ye5dQh9tYjM6WAQWa66TdfT4b7kADH2EaPGiPaFL5ckVWjPe/GxUYXC/OBCUwC8lJF6CqJFy3mIgoLn8mcXKJiRaPsD0iyEnKyw3QSc1/0oesDItENw8iDqxgQVsfT9b4t0H5ztyBEPOWHUcQttplbA1/IvKsalvXMsLqNlN8A6w3YhV0UXX root@eneco-001-xxxxx
....

Copy the bold part starting with "ssh-rsa......" to the authorized_keys file on the NAS. You actually do not link the keys from the NAS and Toon.
member of the Toon Software Collective
User avatar
Toonz
Forum Moderator
Forum Moderator
 
Posts: 1088
Joined: December 2016


Return to Toon software development

Who is online

Users browsing this forum: No registered users and 1 guest