| View previous topic :: View next topic |
| Author |
Message |
odessit
Apprentice
Joined: 01 Feb 2004
Posts: 180
Location: Current Residency - Server Room - Caution - Frostbite Imminent!
|
 Posted: Mon Aug 28, 2006 7:36 pm Post subject: HOWTO Encrypted Backup System Using Bacula and GnuPG |
 |
|
HOWTO Encrypted Backup System Using Bacula and GnuPG
http://gentoo-wiki.com/HOWTO_Encrypted_Backup_System_Using_Bacula_and_GnuPG
Why do it this way?
Bacula is great backup software, but does not have data encryption just yet (in beta). This particular setup uses two proven software packages to overcome the lack of encryption capability.
Intended Audience
[*]Network Administrators who need to have their backups secured via encryption
[*]Existing users of Bacula who need to have backup volumes encrypted
Why use this setup
[*]Works now and has been tested
[*]Frees up clients' resources by moving encryption process to the server
[*]Uses established and tested mechanisms to secure data
[*]Writes encrypted data to a single file.
[*]Does not leave unencrypted information on the Bacula server
Why NOT use this setup
* Extra overhead on the server to complete backup and restore
* Does not use Bacula's native tape-writing mechanisms to store data
** Use any other tape writing utility
** Transfer file over the network
General Overview of the data flow
The Director will do FULL backup every day for the ease of restoration. Each day it will create a new backup volume.
During the backup process Bacula server (called the Director) requests the files from the client (called File-Daemon or FD for short). During the request Director passes arguments to the FD - Director authentication password, files & folder list, request for compression and MD5/SHA1 sum. The received files being written to the hard drive volume (one large file) and cataloged to the database.
Once the Director is done with all clients, the director calls a script, which will encrypt the backup volume with GnuPG and delete the unencrypted volume. After the encryption is completed, the file will be transferred via FTP or CIFS/SMB for offsite storage. (Optionally it can be written to a tape with a 3rd party utility)
Hardware Configuration
Any sufficiently fast server will do. The requirements vary greatly between different environments. To give you a starting point, here is one example
Dell PowerEdge 2800
[*]Single Xeon 2.8 GHz
[*]1 Gig RAM
[*]36 Gig RAID1 array for the /
[*]169 Gig RAID5 array for the /backups
[*]10/100 switched LAN
This setup backs up and encrypts data for about 50 clients, in my organization it equates to about 40 Gig compressed & encrypted data.
GnuPG Installation
Public / Private Key creation
For a good overview on what GnuPG is all about, please visit our own [http://www.gentoo.org/doc/en/gnupg-user.xml Gentoo GnuPG Guide]
You may or may not follow the suggested values for the key in the guide, but this is a safe setup for a backup system
[*]DSA and ElGamal key pair
[*]Key Size - 4096
[*]Public Key Expiration - 5 years (you will still be able to decrypt after 5 years)
[*]Private Key password or passphrase as complex as you can stand
Please note - you do not need and should NOT store the private key on the backup server. It is a security risk.
The best way is to generate the priv/pub key pairs on a secured workstation. Export the public key only as a text file and import public key only into the backup server. (see [http://www.gentoo.org/doc/en/gnupg-user.xml Gentoo GnuPG Guide] guide on how to do export/import)
Store the following items in a safe place
[*]CD or Flash Drive with Public key, Private key, Passphrase and Revocation Certificate
[*]Paper Printouts with Public key, Private key, Passphrase and Revocation Certificate
If you loose your private key or passphrase - bye-bye data.
Bacula Server Installation
The latest version of Bacula in the portage is outdated (bacula-1.36.3-r3) where the latest version as of writing is 1.38.11. Below is example on how to install Bacula from source.
:1. Download latest bacula source from http://sourceforge.net/projects/bacula
:2. Extract the bacula source
| Code: | | tar -xvzf bacula-x.xx.xx.tar.gz |
:3. Enter the source folder
:4. Run the configure command
| Code: | ./configure \
--prefix=/usr \
--sbindir=/usr/sbin \
--sysconfdir=/etc/bacula \
--with-scriptdir=/etc/bacula \
--enable-smartalloc \
--with-mysql \
--with-working-dir=/var/bacula \
--with-pid-dir=/var/run \
--with-subsys-dir=/var/lock/subsys \
--enable-conio \
--with-openssl \
--enable-largefile \
--enable-wx-console \
--with-python |
:5. Compile and Install
:6. Setup Database (assuming you have MySQL installed and working)
| Code: | /etc/bacula/create_mysql_database
/etc/bacula/make_mysql_tables
/etc/bacula/grant_mysql_privileges |
:7 Add bacula to the /etc/conf.d/local.start
| Code: | | echo /etc/bacula/bacula start >> /etc/conf.d/local.start |
:8 add bacula to the /etc/conf.d/local.stop
| Code: | | echo /etc/bacula/bacula stop >> /etc/conf.d/local.stop |
Server (Director) Configuration (bacula-dir.conf)
This is the most confusing part of the whole process. In the following example, just substitute $VARIABLE with your own value.
| Code: |
#bacula/bacula-dir.conf|<pre>
################### DEFINE DIRECTOR #############################
Director
{
Name = backup-dir #or whatever name you want
Description = "Bacula Director - $MY_LOCATION" # "Bacula Director - Main Office"
DIRport = 9101
QueryFile = "/etc/bacula/query.sql"
WorkingDirectory = "/var/bacula"
PidDirectory = "/var/run"
Maximum Concurrent Jobs = 2
FDConnectTimeout = 1min
SDConnectTimeout = 1min
Password = "$CONSOLE_PASSWORD" # "gsdfgfdsg44"
Messages = Daemon
}
################### DEFINE CLIENTS ##############################
Client {
Name = $SOME_IMPORTANT_CLIENT-fd # accountant-fd
Address = $FQDN_OR_IP # accountant.mydomain.local
FDPort = 9102
Catalog = GiantCatalog
Password = "$CLIENT_ACCESS_PASSWORD" # "strong_password"
File Retention = 6 days
Job Retention = 6 days
AutoPrune = yes
Maximum Concurrent Jobs = 2
}
Client {
Name = $LAST_CLIENT-fd # mainserver-fd
Address = $FQDN_OR_IP # mainserver.mydomain.local
FDPort = 9102
Catalog = GiantCatalog
Password = "$LAST_CLIENT_ACCESS_PASSWORD" # "strong_password2"
File Retention = 6 days
Job Retention = 6 days
AutoPrune = yes
Maximum Concurrent Jobs = 2
}
Client {
Name = $BACKUP_SERVER-fd # the bacula server itself backup-fd
Address = $FQDN_OR_IP # backup.mydomain.lan
FDPort = 9102
Catalog = GiantCatalog
Password = "$BACKUP_SERVER_ACCESS_PASSWORD" # "strong_password2"
File Retention = 6 days
Job Retention = 6 days
AutoPrune = yes
Maximum Concurrent Jobs = 2
}
############### DEFINE DEFAULT JOB PARAMETERS #####################
JobDefs
{
Name = "DefaultBackupJob"
Type = Backup
Level = Full
Client = backup-fd
Schedule = "WeeklyCycle"
Storage = File
Pool = Default
Messages = Standard
Priority = 10
Max Start Delay = 18h
}
####################### DEFINE JOBS ############################
Job
{
Name = "$SOME_IMPORTANT_CLIENT_JOB" # "Accountant Backup Job"
Client = $SOME_IMPORTANT_CLIENT-fd # accountant-fd
JobDefs = "DefaultBackupJob"
FileSet = "Generic User Fileset" #it can be reused for many clients
Write Bootstrap = "/backups/bootstraps/$SOME_IMPORTANT_CLIENT.bsr"
Maximum Concurrent Jobs = 2
}
#Backup the catalog database (after the nightly save)
Job {
Name = "BackupCatalog"
JobDefs = "DefaultBackupJob"
Client = BACKUP_SERVER-fd
FileSet="Catalog"
# This creates an ASCII copy of the catalog
RunBeforeJob = "/etc/bacula/make_catalog_backup bacula bacula"
# This deletes the copy of the catalog
RunAfterJob = "/etc/bacula/delete_catalog_backup"
Write Bootstrap = "/backups/bootstraps/BackupCatalog.bsr"
Priority = 10
}
Job
{
Name = "$LAST_CLIENT_JOB" # "Mainserver Backup Job"
Client = $LAST_CLIENT-fd # mainserver-fd
JobDefs = "DefaultBackupJob"
FileSet = "Server Fileset" #we can define a separate FileSet
Write Bootstrap = "/backups/bootstraps/$LAST_CLIENT.bsr"
Maximum Concurrent Jobs = 2
RunAfterJob = "/backups/encrypted/scripts/EncryptVolume.sh"
## NOTE - we will run the script EncryptVolume.sh after this job completes. ##
## It will encrypt volume, delete unencrypted volume and transfer it via FTP ##
}
################## Define which files to backup #####################
FileSet
{
Name = "Generic User Fileset"
Enable VSS = yes
Include
{
Options
{
signature = MD5 #Create MD5 Signature
compression=GZIP5 #Compress Incoming Data on the client
Exclude = yes
IgnoreCase = yes
# Exclude Mozilla-based programs' file caches
WildDir = "[A-Z]:/Documents and Settings/*/Application Data/*/Profiles/*/*/Cache"
WildDir = "[A-Z]:/Documents and Settings/*/Application Data/*/Profiles/*/*/Cache.Trash"
# Exclude directories full of lots and lots of useless little files
WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
WildDir = "[A-Z]:/Documents and Settings/*/Recent"
WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temporary Internet Files"
# These are always open and unable to be backed up
WildFile = "[A-Z]:/Documents and Settings/All Users/Application Data/Microsoft/Network/Downloader/qmgr[01].dat"
# Temporary directories & files
WildDir = "[A-Z]:/WINNT/Temp"
WildDir = "[A-Z]:/temp"
WildFile = "*.tmp"
# Recycle bins
WildDir = "[A-Z]:/RECYCLER"
# Swap files
WildFile = "[A-Z]:/pagefile.sys"
}
File = "c:/Documents and Settings" #Define Folders to back up
File = "d:/SomePath/To Folder/Or/File.txt"
}
Exclude
{
File = "c:/test" #Define Folders to exclude
}
}
FileSet
{
Name = "Server Fileset"
Enable VSS = yes
Include
{
Options
{
signature = MD5 #Create MD5 Signature
compression=GZIP5 #Compress Incoming Data on the client
Exclude = yes
IgnoreCase = yes
# Swap files
WildFile = "[A-Z]:/pagefile.sys"
}
File = "d:/" #Back Everything for this SPECIAL_USER
}
Exclude
{
File = "c:/test" #Define Folders to exclude
}
}
################### DEFINE SCHEDULE ################################
Schedule {
Name = "WeeklyCycle"
Run = Level=Full Pool=MondayPool Monday at 12:05 #PM That is
Run = Level=Full Pool=TuesdayPool Tuesday at 02:12 #AM that is
Run = Level=Full Pool=WednesdayPool Wednesday at 12:05
Run = Level=Full Pool=ThursdayPool Thursday at 12:05
Run = Level=Full Pool=FridayPool Friday at 12:05
Run = Level=Full Pool=SaturdayPool Saturday at 16:00
Run = Level=Full Pool=SundayPool Sunday at 16:00
}
################# DEFINE STORAGE (file, tape, CDR..) ###################
Storage {
Name = File
Address = FQDN_OR_IP_OF_DIRECTOR
SDPort = 9103
Password = "$STORAGEPASS"
Device = FileStorage
Media Type = File
}
############################ DEFINE CATALOG #############################
Catalog {
Name = GiantCatalog
dbname = bacula; user = bacula; password = ""
}
########################## DEFINE MESSAGES #############################
Messages {
Name = Standard
console = all, !skipped, !saved
append = "/var/bacula/log" = all, !skipped
}
Messages {
Name = Daemon
console = all, !skipped, !saved
append = "/var/bacula/log" = all, !skipped
}
####################### DEFINE POOLS ################################
Pool {
Name = MondayPool
Pool Type = Backup
AutoPrune = yes # Prune expired volumes
Volume Retention = 6 days
Accept Any Volume = yes # write on any volume in the pool
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat = "BackupMondayVolume"
}
Pool {
Recycle = yes
Name = TuesdayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat= "BackupTuesdayVolume"
}
Pool {
Recycle = yes
Name = WednesdayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupWednesdayVolume"
}
Pool {
Recycle = yes
Name = ThursdayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupThursdayVolume"
}
Pool {
Recycle = yes
Name = FridayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupFridayVolume"
}
Pool {
Recycle = yes
Name = SaturdayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupSaturdayVolume"
}
Pool {
Recycle = yes
Name = SundayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupSundayVolume"
}
# Default pool definition
Pool {
Name = Default
Pool Type = Backup
}
# Restricted console used by tray-monitor to get the status of the director
Console {
Name = backup-mon
Password = "$RESTRICTED_CONSOLE_PASSWORD"
CommandACL = status, .status
}
|
Define Storage Daemon (SD)
| Code: |
#/etc/bacula/bacula-sd.conf|
Storage {
Name = backup-sd
SDPort = 9103
WorkingDirectory = "/var/bacula"
Pid Directory = "/var/run"
Maximum Concurrent Jobs = 20
}
# List Directors who are permitted to contact Storage daemon
Director {
Name = backup-dir
Password = "$STORAGEPASS"
}
#
# Restricted Director, used by tray-monitor to get the
# status of the storage daemon
#
Director {
Name = backup-mon
Password = "$RESTRICTED_DIR_PASS"
Monitor = yes
}
Device {
Name = FileStorage
Media Type = File
Archive Device = /backups
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
}
Messages {
Name = Standard
director = backup-dir = all
}
|
Define Console
| Code: |
#/etc/bacula/bacula-sd.conf|
Director {
Name = backup-dir #mainofficebackup-dir
DIRport = 9101
address = $FQDN_OR_IP
Password = "$DIRECTOR_CONSOLE_PASSWORD"
}
|
Create the backup script
This is the script that will encrypt the backup volume and upload it to a different server
This script is to be executed via Bacula's RunAfterJob or Cron. It will encrypt a backup volume with GnuPG using Twofish algorithm. If encryption is to be changed use only encryption with 128-bit BLOCK size. Currently only Twofish, AES, MARS and Serpent have 128bit block.
Ciphers with 64-bit block (3DES, CAST5, BLOWFISH...) are not suitable for encrypting large files. Couple Gig volume is Ok, but as you will approach 64 Gigs, you will increase the risk of leaking decryption information (see http://en.wikipedia.org/wiki/Birthday_attack).
| Code: |
#/backups/encrypted/scripts/EncryptVolume.sh|
# Remove Yesterday’s encrypted backup from HDD
cd /backups/encrypted/
rm -f *.gpg
# encrypt the file with GPG using Public Key.
# use only --cipher-algo aes|aes192|aes256|twofish
gpg --encrypt --batch -r backup@giantmarkets.com --cipher-algo twofish \
--bzip2-compress-level 6 --output /backups/encrypted/BakupVolume.gpg /backups/Backup*
# Remove unencrypted backup volume, we encrypted and saved
# encrypted copy in the above step.
cd /backups/
rm Backup*
# transfer encrypted volume via ftp to the server with tape
cd /backups/encrypted/
ftp -ni $FQDN_OR_IP <<SCRIPT
user Anonymous backup_job@backup
cd $SOME_FOLDER
binary
mdelete *.gpg
mput *.gpg
quit
|
Please note. If the script works manually, but fails to encrypt automatically (trust errors), copy the .gnupg folder in the home directory for the working user (root for example) into the root directory
cp -r /root/.gnupg/ /
Win32 Client Installation and Configuration
Download and install the winbacula.x.x.x.exe
Edit the bacula-fd.conf to let your director to connect to the client
| Code: |
#c:\bacula\bin\bacula-fd.conf
Director {
Name = backup-dir #Director's name as defined on the server
Password = "$CLIENT_ACCESS_PASSWORD" #same as in client definition on server
}
FileDaemon {
Name = $SOME_IMPORTANT_CLIENT-fd
FDport = 9102
WorkingDirectory = /bacula/working
Pid Directory = /bacula/working
}
# Send all messages except skipped files back to Director
Messages {
Name = Standard
director = backup-dir = all, !skipped
}
|
(re)Start the Bacula service and you are ready.
*NIX Client Installation and Configuration
:1. Download latest bacula source from http://sourceforge.net/projects/bacula
:2. Extract the bacula source
| Code: | | tar -xvzf bacula-x.xx.xx.tar.gz |
:3. Enter the source folder
:4. Run the configure command
| Code: | ./configure \
--prefix=/usr \
--sbindir=/usr/sbin \
--sysconfdir=/etc/bacula \
--with-scriptdir=/etc/bacula \
--enable-smartalloc \
--with-mysql \
--with-working-dir=/var/bacula \
--with-pid-dir=/var/run \
--with-subsys-dir=/var/lock/subsys \
--enable-conio \
--with-openssl \
--enable-largefile \
--enable-wx-console \
--with-python \
–-enable-client-only |
:5. Compile and Install
:6. Add bacula to the /etc/conf.d/local.start
| Code: | | echo /etc/bacula/bacula start >> /etc/conf.d/local.start |
:7. add bacula to the /etc/conf.d/local.stop
| Code: | | echo /etc/bacula/bacula stop >> /etc/conf.d/local.stop |
:8. Edit the bacula-fd.conf
| Code: |
#\etc\bacula\bacula-fd.conf
<pre>
Director {
Name = backup-dir #Director's name as defined on the server
Password = "$LAST_CLIENT_ACCESS_PASSWORD" #same as in client definition on server
}
FileDaemon {
Name = $LAST_CLIENT-fd
FDport = 9102
WorkingDirectory = /bacula/working
Pid Directory = /bacula/working
}
# Send all messages except skipped files back to Director
Messages {
Name = Standard
director = backup-dir = all, !skipped
}
|
(re)Start the Bacula service and you are ready.
Restoration Process
1. Decrypt the file
| Code: | | gpg BackupVolume.gpg |
(Assuming that gpg has your private key)
2. Move the decrypted file to the bacula server (/backups directory in this example)
3a. If Bacula database is intact - Just follow the restoration procedures in the Bacula handbook
http://www.bacula.org/dev-manual/Bacula_Consol_Restor_Comman.html
OR
3b If Bacula database is not intact - Use the bscan utility to recreate the database from your volume.
| Code: | cd /etc/bacula/
bscan -V RestoreVolume -v -s -m -c bacula-sd.conf FileStorage
|
and then follow the restoration procedures in the Bacula handbook
http://www.bacula.org/dev-manual/Bacula_Consol_Restor_Comman.html
Last edited by odessit on Mon Sep 11, 2006 2:36 pm; edited 3 times in total |
|
| Back to top |
|
 |
odessit
Apprentice
Joined: 01 Feb 2004
Posts: 180
Location: Current Residency - Server Room - Caution - Frostbite Imminent!
|
| Posted: Mon Aug 28, 2006 7:39 pm Post subject: |
|
|
I created this howto based on internal documents that I wrote for my place of employment. It has been sanitized from any proprietary information.
Maybe you can improve on this, so please give suggestions.
Thanks! |
|
| Back to top |
|
|
jhmartin
Tux's lil' helper
Joined: 03 Sep 2003
Posts: 95
|
| Posted: Thu Aug 31, 2006 4:53 pm Post subject: Backup the catalog |
|
|
| You may also want to back up the mysql database after executing a series of backups, as losing this database makes it significantly harder to restore from a backup. A hot backup of MySQL is not reliable at all. Instead, have a job that exports the DB then backs up the export file. |
|
| Back to top |
|
|
odessit
Apprentice
Joined: 01 Feb 2004
Posts: 180
Location: Current Residency - Server Room - Caution - Frostbite Imminent!
|
| Posted: Thu Aug 31, 2006 7:52 pm Post subject: |
|
|
I actually removed the code to save space (IIRC it is included by default with bacula install), but will add it in a bit incase the config file differs b/n different installs.
Edit - done
Edit - Regarding difficulty to restore w/out database.
Actually it is fairly easy by using bscan utility.
Just decrypt the volume, move it to the bacula server and run
| Code: | cd /etc/bacula/
bscan -V /path/to/RestoreVolume -v -s -m -c bacula-sd.conf FileStorage |
If complaints about name, rename RestoreVolume to whatever name is displayed and change the above bscan command to accommodate the new name. The whole process takes ~20 minutes on 2.8 Gig Xeon working on ~ 30 Gig file (HD speed is more important here btw.) |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
