CleanCode Perl Libraries

Perl

Java

JavaScript

Certified
Class

Testable
Class

Standalone
Mode

Diagnostic
Enabled

NAME

copydir.pl - Copy a directory recursively, preserving time stamps.

SYNOPSIS

copydir.pl [options] sourceDir targetDir

OPTIONS

 -d date       start date
 -D date       end date
 -t time       start time (default is start of day 00:00:00)
 -T time       end time (default is end of day 23:59:59)
 -m period     alternative to specifying dates by period (d, w, m, or y)
 -l logdir     write output to dated file in logdir instead of STDOUT
 -x            show operations only; do not execute
 -X            show summary of operations only; do not execute
 -e pathlist   exclude all directories on semicolon-separated path list
 -f extlist    exclude all files with any extension on semicolon-separated list
 -a            copy all files (default is copy changed files only)
 -i            incremental copy (copies to targetDir-incr/date)
 -h            help
 -H            longer help (manual page)

REQUIRES

Cwd, Date::Calc, File::Find, File::Copy, File::stat, Time::Local, Pod::Usage, Fcntl, Getopt::Std, Data::Handy, File::Handy, Time::SimpleFormat

DESCRIPTION

The sourceDir is recursively copied to targetDir including all files that fall within the date/time parameters selected. If you specify no date/time parameters, then all files will be copied.

The target directory leaf (i.e. the last component on the targetDir) need not exist; it will be created by the program. (Any subdirectories of the target needed to replicate the source tree will also be created.)

Default behavior writes verbose output to STDOUT. You may send this instead to a dated log file using the -l parameter. This will redirect STDOUT into a file named copydir-date.log in the directory you specify. This is a recommended option to always use so you have a record of what was copied, including any errors that may have been encountered. (Errors are also reported on STDERR.) Use the -l option in conjunction with -x and the program will record the entire log file without actually copying any files, allowing you to make sure your filters are correct in a dry run.

When capturing the output in a log file, the program will also display a simple progress monitor on STDERR, displaying a period for each directory processed. You will typically get a long stream of dots, then, showing that the program is working.

Full vs Incremental Backup

By default, copydir will copy files to the directory you specify. If you did the same copy last week, only files that have changed will be copied, unless you specify the -a option to copy all files regardless of changed status. Either with or without the -a, this can be considered a full backup, since the net result is a complete copy of the source, overwriting whatever previous complete copy of the source you made.

A typical backup schedule, however, might typically do a full backup weekly, while an incremental backup daily. An incremental copies changed files only but into a separate backup set, not into the same target directory. Copying just the changed files typically keeps the incremental backup quite compact. Then, if you need to recover a version of a file from 3 days ago, you can go back to the backup set from 3 days ago and retrieve it. Contrast that with always doing full backups every day into the same directory. While it might seem simpler, you can then never recover a file from more than one day ago. (Alternately, if you do a full backup into a different directory each day, you avoid that problem, but then you need huge amounts of storage capacity.)

To direct copydir to use the incremental approach of writing to a separate but related directory, use the -i option. If your target directory is, for example, /usr/bkup, then -i will instead write to the directory /usr/bkup-incr/date where date is the current date in the form yyyy-mm-dd.

Finally, it is important to be aware of the interaction of -a and -i. Or, more accurately, the absence of -a, indicating only newer files should be copied. This "newness" check is still done against the principal target directory (/usr/bkup in the previous example). Then, if newer, the copy is placed in the incremental target (/usr/bkup-incr/date).

Pruning by Date

There are several parameters that allow you to tailor the date range to your needs. Use -d and -t to specify starting date and starting time, respectively. Similarly use -D and -T to specify ending date and time.

The time may be specified as hh or hh:mm or hh:mm:ss. The date is interpreted using a flexible library routine that can analyze a variety of common formats, including the basic mm/dd/yyyy. Experiment to see what works for you by omitting source and target dirs, and just specifying date/time options.

The final date option (-m) allows you to specify the starting date as a relative figure; that is, to select all files modified in the last n units, where units may be days, weeks, months, or years.

Note that -m and -d both set the starting date only (not the starting time). If both are used, -d has precedence.

Pruning by Directory

A second pruning technique is via the exclusion list option (-e). Here you specify a specific list of subdirectories of your source directory to prune. Say for example your source directory is /usr/abc. Then using

        copydir.pl -e /usr/abc/tmp;/usr/abc/archive /usr/abc /bkup

will skip the tmp subdirectory and the archive subdirectory completely. Note the use of absolute paths in that example. You may also specify just a leaf directory name to prune that leaf wherever it may occur in the source tree. That is, if you instead use

        copydir.pl -e tmp;/usr/abc/archive /usr/abc /bkup

then not only will /usr/abc/archive and /usr/abc/tmp be skipped, but any other subdirectory named tmp will also be skipped. Finally, note that when using absolute paths, your exclusion list must be consistent with your source directory specification. You could use either:

        copydir.pl -e /usr/x/dir1;/usr/x/dir2 /usr/x /bkup

or (assuming /usr is on the C: drive):

        copydir.pl -e C:/usr/x/dir1;C:/usr/x/dir2 C:/usr/x /bkup

but this will fail to prune dir1 or dir2:

        copydir.pl -e /usr/x/dir1;/usr/x/dir2 C:/usr/x /bkup

Pruning by File Type

Similar to pruning by directory, you may prune on a file-by-file basis by specifying one or more file extensions. Separate multiple items with semicolons. This example will exclude any file ending in tmp, bak, or old:

        copydir.pl -f tmp;bak;old /usr/abc /bkup

Pruning by Dated Name

On huge directories, the scan-and-copy of a tree can take a while, since each file in each subdirectory must be scanned for its datestamp. One way to optimize is by having dated directory names, e.g. having a directory named log-2004-04-03 for example. The system will skip that directory completely (and its subdirectories) if the dated name is outside the time range you specify.

BUGS

Does not restore datestamps on created directories.

AUTHOR

Michael Sorens

VERSION

$Revision: 278 $ $Date: 2008-05-26 22:44:56 -0700 (Mon, 26 May 2008) $

SINCE

CleanCode 0.9

CleanCode Perl Libraries