Documentation

From Nasbackup

Jump to: navigation, search

Contents

Introduction to NasBackup

NasBackup is an open source backup solution. It is a high-performance, enterprise-grade system for backing up MS Windows desktop PCs, laptops and servers to network disks. NasBackup is a highly configurable solution and easy to install and maintain.

NasBackup is a backup solution that can replace traditional tape backup. The information is backed up to network storage using the rsync utility in a way that only the file differences are sent over the network.

NasBackup was designed to be easy to use for the end desktop user, yet scalable to backup enterprise servers. The NasBackup client Windows GUI is easy to maintain and use by end user. Other backup solutions based on rsync are more complicated and have no GUI interface. We had the end user in mind while designing NasBackup. The implementation is very easy to adopt by the end user, including clear and simple backup settings, backup log status report, etc. Still, NasBackup is scales well to backup enterprise servers including SQL and Exchange databases etc.

Given the ever decreasing cost of disks, it is now practical and cost effective to backup a large number of machines onto a server’s local disk or network storage. This is what NasBackup does. For some sites, this might be the complete backup solution. For other sites, additional permanent archives could be created by periodically backing up the server to tape.

Disk usage example: 100 laptops with a full backup averaging 2GB each, and daily difference 20Mb. Storing three weekly full backups per laptop is around 4200GB of raw data, but because incremental backups and compression are used only 120GB is needed. Here is the calculation: 1st day 1 GB of compressed data * 100 laptop is 100 GB. The rest 20 day * 100 laptop * 10 Mb of compressed data is 20 GB. It comes to a total of 120 GB!

NasBackup is written in Perl and backs up data via the rsync protocol. It is robust, reliable, well-documented and freely available as Open Source.

Features

Client Features

Features available on the Client side for the end user and the backup administrator:

  • GUI backup setup and schedule.
  • GUI online backup progress and final status.
  • File backup failure - online alert. An open file causes failure, NasBackup will pause for a while allowing the user to close the open file.
  • Option to compress the backup files.
  • Option to encrypt the backup files.
  • Option to backup locked files like Outlook PST by automatically closing the application at backup time.
  • Easy integration of other utilities during the backup procedure like internet dialer, registry backup etc.
  • Allows selection of any folders on local or network-mapped drives at the click of the mouse.
  • Firewall Friendly, Can be configured to use port 80.
  • Fast. Based on Rsync, the fastest incremental file transfer program available, either open source or commercial. Only the differences between files and directories are transferred between local folders and their corresponding remote mirrors.
  • Scalable. Based on Rsync can mirror directories of any size without degradation of performance.
  • No client-side software is needed for restore. Standard protocols are used: http/s, ftp, smb, etc.


Server Features

Features available on the Server side for the Backup administrator:

  • Control of each client’s backup files version limit.
  • Control of each client’s backup files old versions expiration date.
  • Central daily/periodic summary report via email. Shortcut from the report to the full log of the failed client.


Requirements

Client side

The following OSes are supported on the Client side:

  • Windows NT
  • Windows 2000
  • Windows XP
  • Windows 2003

Other OSes that do not appear in the above list can implement backup using standard rsync client. Some of the client features will be missing!




Server side

The following OSes are supported on the Server side:

  • Linux
  • Windows 2000
  • Windows XP
  • Windows 2003

Other OSes which do not appear in the above list will probably work, but have not been checked. In theory any OS that can run rsync and Perl can be implemented as a backup server. It is highly recommended that the backup server be located at a remote location and not on the same LAN as the client. Rsync should be installed on the server as a prerequisite. The following software should be installed on the server for full functionality: Perl, web server (Apache, IIS etc)


How does it work?

The phase templates

Using the GUI interface, the end user selects the directories to backup and schedules a time to run the backup process. Once this is done, the client wakes up on schedule and starts the process for each of the selected backup directory.

The backup is done in three phases: start, main, and end, which are executed in this order. The start and the end phases are activated only once during the backup session. The main phase can be activated several times, each time for other backup directories the user selects.

The backup administrator can intervene in each of those phases. For example, to activate the internet dialer in the start phase and close the internet connection in the end phase. Another example, in case you want to be sure that Outlook’s pst file is not locked during the backup you can exit the Outlook application in the start phase using a 3rd party utility like killp.


Backup status

During the backup process, the backup windows will show the current working directory. At the end of the backup the end user will see the backup summary of each directory. The user will have to manually close this window. In case you don’t want the end user to be interactively involved it is possible to configure the backup windows to be automatically closed at the end of the backup.

Backup log

The client backup log is transferred at the end of the backup to the server side. The administrator report utility analyzes all the client logs and sends an email report to the backup administrator. In case there is no log from one of the clients because of a communication problem or any other reason, the administrator report utility categorizes it in the report as missing log. As mention above, the backup log is transferred at the end of the backup session from the client to the server, so in case of a severe problem like communication failure the backup log is still available on the client disk for later investigation and analysis.

The backup summary utility sends email in html format, which contains links to the failure log on the server or the client backup log directory on the server (in case of a missing log). In this way the administrator can determine the last time the client backup finished successfully.

NasBackup Components

Client

Backup setting

nasbackup_user_gui.exe allows the end user to select what to backup.

Rsync utility

rsync.exe, use the -h parameter for full syntax. It uses cygwin1.dll.

Backup upload

nasbackup_main.exe the main backup utility. Will start the backup process. Can be scheduled to run or run manually.

Compress and encrypt

nasbackup_dir.exe, use to encrypt files. Use the -help parameter for full syntax. nasbackup_special_gz.exe use to compress files.

File open alert

nasbackup_mbok.exe creates a GUI alert if a file is locked so that the user can close open applications.

Decrypt restore file

nasbackup_decrypt_gui.exe, in case you use encryption you need this utility to decrypt restore files.

Digital key encryption management

nasbackup_key_gui.exe, in case you use encryption you need this utility to create you digital encryption key. Without this key you CAN NOT decrypt restore files. You must backup the key and store it in remote and safe location away from the backup client machine.


Process killer

killp.exe is windows process killer. It can stop processes that lock files, like outlook.exe. The component is optional and can be use from any of the phase files, Probably you will want to use it in the start phase. Killp is freeware from Alex Simonetti Abreu.

Server

Rsync service

Standard Rsync server.

Backup admin report

admin_report.pl utility analyzes the rsync log and other server status information and sends a report to the backup administrator. The report will contain a link to the failure client log and much more.

Backup versioning utility

This utility enforces each client’s backup files version limits and each client’s backup files expiration date.

Restore setting

It is up to the backup administrator to decide how to let the end user restore his files and directories. The recommended way is using a protocol that is imbedded in the MS Windows file explorer, so that the client will not have to learn a new program. The most common ways are ftp, http, and smb (network share).


Installation and setting

Client installation

Download and run nasbackup_v#.##.exe (#.## represents the version number) to install the software. When the installation is finished the Backup setting application will open. You should fill in the following fields:

FieldValue
Backup server name or IPValid IP or DNS name
Share nameRsync module name*
User NameRsync user name*
PasswordRsync user password*
Task loginWindows user name or windows_domain\username
Task passwordWindows password
  • Rsync settings will be discussed in the server section later in this document

Other fields on this screen can be accessed by enabling them via the Advanced menu available on the menu bar. Please note that the default installation process does not enable encryption or compression for backups. Follow the customization section of this document for more information.

Server installation

Prerequisites

Perl 5.6 or newer. Perl module Mail::Sender v0.8.10, earlier versions are reported to cause failure.


Install the server side utilities

The following installation details are for Unix/Linux systems, but can easily be modified to fit MS Windows OSes. Untar the XXX tar file. There are two subdirectories: admin_report and version_control. You should move them to a permanent location, for example: /usr/local/nasbackup Schedule the utilities in those directories to run once a day.

Administrator report

Edit the definition section in the file admin_report.pl

If you would like to access the client backup log files and log directory is via http protocol ($html_log_link parameter), you need to edit the web server configuration file. For example (on Apache web server), to set the URL http://www.nasbackup_server.com/nasbackup_clients_logs/ to point to /home/nasbackup/clients_logs/ add the following line to the httpd.conf file. create the directory /home/nasbackup/clients_logs and then make symbolic link.


Alias /nasbackup_clients_logs /home/nasbackup/clients_logs

<Directory /home/nasbackup/clients_logs>
  Options Indexes +FollowSymLinks 
</Directory>

Each clients_logs directory should be symbolic link to /home/nasbackup/clients_logs:


ln -s /home/users/usera/clients_logs /home/nasbackup/clients_logs/usera


Add the following line to your cron daily schedule 59 23 * * * /usr/local/nasbackup/admin_report/admin_report.pl

Please note the default report of admin_report.pl is last day, so it will be a good idea to run at the end of the day (i.e. 01:00). admin_report.pl accepts a date parameter so that you can create a report for a previous date. For example: ./admin_report.pl 2004/05/24

Version control

You should install and configure this utility only if you want to save and manipulate more then two backup versions of the files.

The default installation (defined in the "main.phase" file) sets previous_version as the directory to store the previous version files in. The version control utility allows you to manage this directory.

Edit the definition in the file "version_control.conf". For each nasbackup client you should set the following:

ParameterExplanation
user_dirclient backup directory, pull path. e.g. /home/users/client1
previous_version_dirname of previous version directory. The default is "previous_version"
max_versionsmaximum versions to save. The default is endless.
expire_days*When to delete old versions of the file. The default is endless.
one_version_per_daySet to "true" if you want more then one version per they, otherwise "false"
  • expire_days overrides the max_versions limit. So files older than expire_days in previous_version_dir will be deleted no matter what the max_versions setting is!.

The counting starts at the date when the file was first moved to the previous version directory.

For clients that are not added to version_control.conf, the default of two versions will be applied.

Add the following line to your cron daily schedule 50 23 * * * /bin/csh /usr/local/nasbackup/version_control/version_control.csh


Restore setting

As mentioned above, the backup administrator will decide how to let the end user restore his files and directories. The recommended way is to use a protocol that is embedded in the MS Windows file explorer, so that the end user will not have to learn a new program. The most common protocols are: ftp, http, and smb (network share).

Here are some definition examples to set appropriate access.

  • ftp on Linux* can be done by adding local user and set the home directory as the backup directory. Give this user ftp access.
  • http on apache can be done by adding the following definition to the apache configuration file, httpd.conf:
<Directory "/home/users/usera ">
  Options Indexes MultiViews
  AuthUserFile /etc/nasbackup/.http_access_pass_file
  AuthName "usera backups"
  AuthType Basic
  require user usera
</Directory>
  • Samba share on Linux can be done by adding the following definition to the samba configuration file, smb.conf:
[usera_backup]
       path = /home/users/usera
       writable = no
       user = usera
       comment = usera_backup
       valid users = usera
       public = no

Rsync config on the server side

in /etc/rsyncd.conf create client /s, client logs and client settings modules. For example:

[client1]
    comment = client1 backup
    path = /home/users/client1
    read only = no
    list = no
    uid = client1
    gid = 0
    auth users = client1
    secrets file = /etc/rsyncd.secrets

Rsync conf file

Information about installing rsync on a windows machine is available at http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html.

Customization

The customization is optional. You can use NasBackup as is without any customization!

Client

nasbackup_main command line options (flags)

--encode Call OEM_None_latin routine.

--exit Auto exit, don’t wait for the use click.

--dashdir This option make that selected folders to backup are stored preserving drive unit(like "C", "D", etc) and change the "\" directory separator to "-".

--test Use it to start the program in test mode. The program only backup the directories specified in entrySrcDirTest.

Dashdir option

This option make that selected folders to backup are stored preserving drive unit(like "C", "D", etc) and change the "\" directory separator to "-".

To use this, is necessary to add "--dashdir" when starting backup: If started from command line: C:\nasbackup\nasbackup_main.exe --dashdir If using desktop shortcut or start menu is necessary to edit target in "Start Backup" properties: "Target: C:\nasbackup\nasbackup_main.exe --dashdir"


Before using this option: With this option Rsync creates a main folder for each directory the user select that differs in name from the original. If you use different folders for many nasbackup users on one single Rsync user, you should create each user folder first, as Rsync can create only one folder before it starts the backup.

This option is used for various things: 1- Use different folders on the remote dir to store folders of the same name but different drive:

Nasbackup currently store files in a folder with the same name of selected folder. Ej: If user select "C:\docs", the it is stored in "docs" under the account on the remote rsync server. If a user select "C:\docs" and "D:\docs", the two folders are stored in the same "docs" directory and they are overwritten.

With "--dashdir" option specified on nasbackup_main.exe files are stored this way:

User select:

"C:\docs"

"D:\docs"

Server dirs:

"C-docs"

"D-docs"

2- Use nasbackup to backup data on Rsync Servers under Windows.

When a user select a subdirectory to backup, nasbackup store files this way:

User select: "C:\docs\exel"

Server dir: "docs\excel"

Under Windows "\" is not allowed as part of the folder name, so backup fails.

With "--dashdir" option specified on nasbackup_main.exe files are stored this way:

User select: "C:\docs\exel" Server dir: "C-docs-excel"

3- Restore data fails As windows fails as an rsync server to store data from subfolders, windows has difficult to read the contents of folders with "\" on the name. Retrieving from ftp with IE(using copy/paste of entire folder) restore folders this way:

Form "docs\excel", windows creates a folder "docs", and inside of it, a folder "excel"; but the two are empty.

phase files

The client setting and customization are done using the nasbackup_user_gui.exe and modifying the *.phase files. You can set the following options using nasbackup_user_gui.exe:

FieldValue
Backup server name or IPValid IP or DNS name
Share nameRsync module name1
User NameRsync user name1
PasswordRsync user password1
Task loginWindow user name or windows_domain\username
Task passwordWindows password
Directory 2Optional, remote subdirectory to backup instead of main directory
File modified from date 2File modified before the setting date will not be backed up. This option is applicable only if using compression and encryption.
File modified in last days2File modified before X days will not be backed up. This option is applicable only if using compression and encryption.

1 Rsync settings are discussed in the server section of this document.

2Those options can be accessed by enabling them via the advance menu.

The backup can be run manually and/or as a Windows scheduled task. To run it manually double click the "Backup Start" (nasbackup_main.exe) icon on your desktop.


Enabling compression and encryption

The default installation process does not enable compression and encryption of backup files. To enable compression and encryption:

  • Rename the file "main.pause_compressing_and_encryption" to "main.phase"
  • Create private digital key for encryption on each client by running "nasbackup_key_gui.exe".

After restoring encrypted files and directories they can be decrypted on the client machine using "nasbackup_decrypt_gui.exe".

The phase files

You can interfere with the backup process by modifying the following three phase files:

1. start.phase

2. main.phase

3. end.phase

Each field the user configures in the backup setting utility can be used in the entire backup process by any 3rd party software.

The following table describe the available parameters:

Parameter nameDescriptionExample
<dir>Directory that the user chose to backupC:\dira\dirb
<tmp_dir>A temporary working directory. Normally used for compression and encryption before backup. Its value automatically contains the local drive name with most available disk space plus "temp_backup"C:\temp_backups
<dir_ul>Name of output directory on the backup server. Its value automatically contains a modified value of <dir> without the characters ‘:’ or ‘\’C_dira_dirb
<tmp_dir_cyg><tmp_dir> name in cygwin style for rsync/cygdrive/C/temp_backup
<dir_ul_cyg><tmp_dir>/<dir_ul>temp_backups/C_dira_dirb
<remote_dir>Remote subdirectory to backup instead of main directoryPC1/
<drive_no_dir>Drive letterC
<dir_no_drive>Directory name without drive letter or ‘:\’dira\dirb
REM HideDon’t send output to the backup window. Will hide from the end user irrelevant information.
Working on:<dir>Will echo the text: Working on: <dir>Working on: C:\dira\dirb
<label_cnt>Running counter of the number of the current backup directory, for goto labels Not in use in the current version
<date>The value of "File modified from date" or "File modified in last days" from the client setting


In order to analyze rsync exit status each rsync command should come after "Working on:<dir>" line for example: ... Working on:<dir>

rsync.exe -r -t -b ...

Examples:

Backup with encryption and compression:

mkdir <tmp_dir>\<dir_ul>

nasbackup_dir.exe -source "<dir>" -target <tmp_dir>\<dir_ul> -e -z -w 120 -v <date>

Working on:<dir>

rsync.exe -r -t -b --backup-dir=previous_version --exclude="~$*" --exclude="*.bak" -v --delete --stats

/cygdrive/<tmp_dir_cyg>/<dir_ul_cyg> <user>@<server>::<user>/<remote_dir>

RMDIR /S /Q <tmp_dir>\<dir_ul>

Backup without encryption:

Working on:<dir> rsync.exe -r -t -b --backup-dir=previous_version --exclude="~$*"

--exclude="*.bak" --delete -v "/cygdrive/<drive_no_dir>/<dir_no_drive>"

<user>@<server>::<user>/<remote_dir>

Exclude files list from text file

rsync.exe --exclude-from=user_name_exclude.txt ....

Backup MS Exchange

Select in the backup setting utility the Exchange data directory for backup. In "start.phase" add "net stop" command to stop all Exchange Server services:

net stop MSExchangeMSMI /y

net stop MSExchangePCMTA /y

net stop MSExchangeFB /y

net stop MSExchangeDX /y

net stop MSExchangeIMC /y

net stop MSExchangeMTA /y

net stop MSExchangeIS /y

net stop MSExchangeDS /y

net stop MSExchangeSA /y

To restart the Exchange Server services, add the following lines to "end.phase"

net start MSExchangeSA

net start MSExchangeDS

net start MSExchangeIS

net start MSExchangeMTA

net start MSExchangeIMC

net start MSExchangeDX

net start MSExchangeFB

net start MSExchangePCMTA

net start MSExchangeMSMI

Note: To decrease the downtime of the Exchange server. After stopping the services, copy the Exchange server data directory to a temporary location, and start the services. The data will be backed up from the temporary location.

Backup Outlook pst file

To stop Outlook 97/2000/2003 application in the beginning of the backup add to the start.phase file the line: C:\nasbackup\killp.exe outlook > NUL

Dial using a modem

Add to the "start.phase" file the line: RDialPro "dialer name"

Add to the "end.phase" file the line: RDialPro "dialer name" /DISCONNECT

RdialPro can be downloaded from the Mu-Consulting web site: http://www.muconsulting.com/

Demo

You can take a "test drive" of the NasBackup client side using our demo account. Please note that other users may also use this single demo account at the same time. To avoid chaos use the "Directory" option in the backup setting utility to create your own subdirectory. The demo account has a limited disk quota, so please backup a small directory of less then 2 megabytes. Once a day the backup files will be removed from the demo storage space.

Download and install the client setup nasbackup_v#.##.exe. When the installation process is finished, the backup setting utility will open. You should fill in the following fields:

FieldValue
Backup server name or IPwww.nasbackup.com
Share namebackupdemo
User Namebackupdemo
Directory *Your subdirectory name on the demo account.
Passworddemo
Task loginYour Windows user name or windows_domain\username
Task passwordYour Windows password
  • To access this option enable it via the advance menu.

The backup can be run manually and/or as a Windows scheduled task. To run it manually double click the "Backup Start" (nasbackup_main.exe) icon on your desktop.

Please note that the default installation process does not enable compression and encryption of backup files. Follow the customization section of this document for more information.

You can restore your backup files using http or ftp protocol.

Use these addresses to restore:

User name: backupdemo Password: demo


Support

NasBackup FAQ

Please take a look at the FAQ section on the NasBackup web site: [1]

Rsync error code

The following table describes most common error codes:

Error codeError messageDescription
5Bad module name?
23partial transferfile-access or a file-writing problem that prevented a file

from being updated. Typically the warning is something like an access permission not allowing the reading of a source file or directory, however it might also be a read error when reading the file, which could indicate a disk problem -- but that is usually not the case. Finding the error in the verbose output should tell you what's going on.

23Permission denied some files could not be transferredThere is no permission to access the backup file or directory
23No such file or directory some files could not be transferredFile name char encoding is not supported
12rsync: read error: Connection reset by peer writing 4 unbuffered bytes - exiting: Connection reset by peerInput file is lock and could not be read. Can be also connection error? or heavy load?
12@ERROR: auth failed on module XYZ rsync: connection unexpectedly closed (89 bytes read so far)

rsync error: error in rsync protocol data stream (code 12) at io.c(150)||Fail to access module XYZ. Check you rsync password (RSYNC_PASSWORD env variable), and rsyncd.secrets file

12error in rsync protocol data stream writefd_unbuffered failed to write 16 bytesFail to write to the server, probably no disk quota
10error in socket IO (code 10) at /tmp/rsync-2.5.6/clientserver.c(83) failed to connect to 10.10.10.10: Connection timed outNo TCP/IP connection to the server


More information can be found on the rsync web site: http://rsync.samba.org/

Rsync documentation: http://rsync.samba.org/documentation.html

Rsync FAQ: http://rsync.samba.org/FAQ.html

Rsync mailing list searchable archive: http://www.mail-archive.com/rsync@lists.samba.org/

Developer Guide

How to make installation kit

TBD

How to run source version using perl

Key needed to add in the regisrty

manually file configuration

How to make exe’s from perl source

TBD

phase files parameter

<working_drive> <working_dir>

Registry keys

InstallPath

The user will choose the install directory when installing the program (for exemple C:\Program Files\NasBackup) and the installer will put this path in InstallPath in the registry.

LanguageFile

It has the name of the file contening the language strings. The file must be in the current directory. The extension of the file must be .pm


entrySrcDirTest

A new key is needed in the registry : entrySrcDirTest It contains the path of the directory (or directories) used for the test.

Configuration files

Nasbackup_conf.pm

Old NasBackup documentation in PDF format

Version: 1.05 | File size: 78.13 Kb

This documentation apply to Server utilities version 1.05 and Client version 1.05.

download


License

NasBackup is available under the GPL license, except the backup setting utility and the encryption component. The backup setting utility is available under the BSD license, since it is based on code which was originally released under the BSD license. See the credit section for more information.

The encryption component is available as freeware. That means it is free to use but the source code is not available. The community is welcome to contribute an open source encryption to NasBackup. Please note that the default configuration of the client in to not use encryption.

Killp is freeware. It is free to use but the source code is not available. This component is optional. Killp is freeware from Alex Simonetti Abreu.

Credits


Appendix: Compression

In order to be rsync compatible, we use a modified version of gzip to compress the files. The compression is optional. You can remove it in "main.phase". Instead of doing the compressing on the client side it can be done on the server side by using a compressed file system.


Appendix: Encryption

At the current version the implemented encryption is low. If you have high security demands it is not recommend to use it. In future releases we plan to introduce several encryption algorithms. The encryption is optional. Please note that the default configuration of the client in to not use encryption.


Personal tools