Backup3 Manual

September 6, 1995

Contents

0: Invocation

    Backup3 [options]
            -l <level> Specify backup level (default is level 0)
            -t <target> One of Backup, Archive, Copyout or CopyRCS
            -d <dest> Destination device, file or directory
            -q Send result via qcp

1: Introduction

Backup3 is a backup utility specifically tailored to backup software source code. When developing software, almost all of the work goes into writing the source files. Using these source files and some instructions, compilers can generate the desired executable code (the "targets") and a number of intermediate files, too ("object" files). The target files are crucial to running your application. The object files are important only during compilation. Since both the object and target files can be automatically generated from the source files, we only need to save the source files in order to have a complete backup. Since source files are generally much smaller than the targets and objects, a source backup is quite small, allowing practical backups to be performed with nothing more complicated than a floppy disk.

2: File Categories

In order for Backup3 to do its work, it must know which files are SOURCE, OBJECT, TARGET or STANDARD files. Backup3 will look for this information in the Makefile. If you are creating a Makefile directly, you will want to look closely at the Makefile syntax that Backup3 requires, but if you are using appgen, your Makefile is automatically generated and you need only be concerned with the appgen syntax.

2.1: SOURCE

SOURCE files are all the files which are required to regenerate everything in the directory (except STANDARD files). These are the only files which Backup3 will save. Within the .spec file and many Makefiles, this category is further broken down into SRC, TOOL, SCRIPT and NONRCS.

SRC files include files which are actual source to some sort of compiler, but which are not also the output of another compiler. For example, a C source file which you type in called doit.c would be a SRC file, but abcdisp.c which is automatically generated by the TMC compiler from .tmc sources would not be.

TOOL files help to define how the SOURCE code can be turned into executable programs. The Makefile and .spec file fall into this category, as do any straight documentation.

SCRIPT files are files which are essentially both SOURCE and TARGET. They are files which you edit directly and are also required by the runtime system. They need to be backed up, and they also need to be distributed to your run directory.

NONRCS files are binary files which would otherwise be SRC files. A binary file is any file with non-ASCII data. The Revision Control System (RCS) which comes with QNX does not handle binary files, so it is useful to make this distinction. SRC, TOOL and SCRIPT files can all be logged into RCS to keep track of revisions over time.

The most important point to take away from this discussion is that you can't always tell whether or not a file is a source file simply by looking at its name. What might appear to be a source file might really be an object file, and in some cases, what might appear to be an object file might really be a source file. Ultimately there is no substitute for actually knowing what you are doing!

This distinction is critical for more than backup. If you edit a file which is really an object file, your changes will not be backed up and they will be be destroyed the next time the real source file is edited.

In really good development environments, the object and target files are segregated into separate directories so there is less chance for confusion. I am currently looking into adopting such an approach, but substantial work is required, so it won't happen soon.

2.2: OBJECT

OBJECT files are intermediate files which are generated by a compiler and may be required for a subsequent stage of the compilation but are not required at runtime for the final application. More generally, OBJECT files are files which can be safely deleted when you are not compiling without affecting the runtime system or the source code.

Within Makefiles, OBJECT might be broken down to include OBJ and TEMP, where OBJ files are the true intermediate files and TEMP are other temporary files which are not required further. Error listings from a compiler are an example of TEMP files.

2.3: TARGET

TARGET files are the final outputs from the compilation process which are required by the runtime system. TARGET files are not backed up and may be safely deleted in a total cleanup (such as when the runtime system is inactive for a long period) because they can be regenerated via compilation.

2.4: STANDARD

The STANDARD category was created to handle files which may need to be in your directory which you didn't create and which don't need to be backed up because they are part of some other distribution which is backed up separately. I use this category to avoid backing up standard operating system files which can be recovered from the standard distribution.

3: Categorize Your Files

Every file in a directory should be classified into one of these categories. Before actually performing the backup operation, Backup3 will check to see that you have properly categorized all the files in the directory. Any file whose type is not known will generate an error: 
      ==> working.file
You should decide whether these files fall into one of the above categories. You should then edit your .spec file or your Makefile to include these files or alternately delete the files if they are obsolete or otherwise don't belong in the directory.

4: Performing a Backup

In order to backup a directory, you must change to that directory and invoke Backup3. Backup3 will create a mirror directory as a subdirectory of /tmp which will contain links to the source files. It is this mirror directory which is actually archived. Backup3 invokes the standard QNX pax utility to pack all the files into a single archive, then it invokes the QNX freeze utility to compress the archive.

There are several options to determine what Backup3 should do with this archive once created.

4.1: Backing up to Floppy

By default, Backup3 will write the compressed archive to the raw floppy disk in /dev/fd0. This floppy must have been low-level formatted, but it does not need to have a file system installed or mounted. A formatted DOS diskette will work, but it will cease to be a DOS diskette after writing this backup to it.

Because this method does not use a file structure, it cannot tolerate any flaws on the diskette, so the diskette should be checked initially to make sure it has no flaws. The DOS format utility does this sort of check and reports the number of bad blocks after the disk is formatted. Any bad blocks indicates that the disk should not be used with Backup3. Since a file system can work around bad blocks, such a disk may be still be useful for other purposes, but don't try to backup to it or you may not be able to restore reliably.

We also have a utility to perform this low-level format and check the disk under QNX called flinit, though at present it is somewhat lacking in general applicability.

An alternate floppy device can be selected either by using the -d option to Backup3 or by setting the ARCDEV environment variable.

In order to write to the raw floppy, you must have write permission on the floppy device. This can be tricky, and I haven't developed a general approach. For this reason, it may be generally more convenient to backup to a file.

4.2: Backing up to a File

To write the compressed archive to a file, simply specify the filename with the -d argument: 
      Backup3 -d ~/abc.pax.F
Specifying a file type of .pax.F marks the archive as a compressed archive in case you forget. Using this approach, the archive could be written to a DOS floppy (provided the name was constrained to DOS's 8.3 format): 
      Backup3 -d /dos/a/abcpax.f

4.3: Backing up Remotely

If you have a QNX machine at a remote location and dial in using qtalk, you can use the -q option to automatically transmit the compressed archive using QNX's qcp protocol. This will not work if you are dialing in from a DOS machine which doesn't support qcp.

5: Incremental Backups

Backup3 supports incremental backups through the -l option. By default, Backup3 performs a level 0 backup, saving all the source files into a single archive. A level 1 backup, designated by "-l 1" backs up only those source files which have changed since the most recent level 0 backup. Similarly, a level 2 backup saves only files which have changed since the more recent level 1 backup.

Needless to say, a level 1 backup is of relatively little value without the corresponding level 0 backup, so be sure not to overwrite your level 0 backup with your level 1 backup.

For most single directories, there is little need to use incremental backups since the full backup is relatively quick.

6: Alternate Invocations

The -t option controls how much work Backup3 does to guarantee a comprehensive backup of your application. The four optional modes are Backup, Archive, Copyout and CopyRCS. Backup and Archive check each file in the directory agains the SOURCE, OBJECT, TARGET and STANDARD categories. Copyout and CopyRCS do not perform that test, but simply backup those files which are already designated as SOURCE. Archive and CopyRCS will backup any RCS subdirectories in their entirety; Backup and Copyout will not. The default mode is Archive, and I recommend that you always use the default except in dire circumstances.

7: Restoring from a Backup

The utility to restore a backup is Copyin3. It restores the files into the mirror directory, then compares each file against the existing directory to see if the file is newer. If the file is newer, it asks whether the old file should be overwritten.

When you get quite confident of Copyin3's operation, you can use the -R (Rambo) option, which will overwrite older files without asking.

The -s option to Copyin3 allows you to specify the location of the compressed archive.

A.A: Appendix A: Environment Variables

ARCDEV specifies where the compressed archive should be written. The default is /dev/fd0, the raw floppy device on the local node.

TMPDIR specifies the directory in which to build the mirror directories. This is defined in the standard environment and is usually /tmp.

MKVERBOSE, if defined to be a nonempty string, will cause all sorts of verbose output to appear on your screen, obliterating what you really want to see. I used it when initially debugging these applications.

A.B: Appendix B: Caveats

The /tmp directory and the directory being backed up must reside on the same filesystem (disk) or the method of using hard links between the mirror directory and the source directory will fail. This is an issue if you are backing up a directory from a remote machine. One way to work around this is to change your root: 
      cd //4/home/clono2/src
	  on -r //4/ Backup3 -d ~/clono2.pax.F

Return to Manuals Guide

last updated: Thu Jan 2 10:56:25 2003 webmaster@huarp.harvard.edu
Copyright 2002 by the President and Fellows of Harvard College