digup drawing with shovel, files and a digest list

ShoutBox

No posts yet.

digup - A Digest Updating Tool

Description

digup is a console tool to update md5sum or shasum digest files. It will read existing digest files, check the current directory for new, updated, modified, renamed or deleted files and query the user with a summary of changes. After reviewing the updates, they can be written back to the digest file.

One of the envisioned applications of digup is to update and verify incremental archives like chronological data storages or music collections. By using digup possibly undesired changes or lost files can easily be detected while quickly adding new files.

Another purpose could be to automatically verify the integrity of hard disk copies of archives, as backups to hard disks are becoming increasingly popular. Using a full file digest scan even slowly creeping bad blocks on old hard disks can be detected. By using a crontab entry, this check can be performed unattended and routinely.

In normal operation only touched files with newer modification times are fully read and their digest compared. Optionally a full scan can be performed to test all file contents against their expected digests.

Symbolic links are supported by either following the link and reading the target's digest or by saving only the link target path and verifying it against the old one.

The digest files written by digup are compatible with those generated and read by md5sum and similar programs from the coreutils package. Additional information like file size and modification time or symlink targets are stored on comment lines.

Four digest algorithms are supported: MD5, SHA1, SHA256 and SHA512. The digest file itself is also checksummed using CRC32 against unintentional changes. A fast red-black binary tree is used for the internal file list, allowing fast operation on a large number of files.

Alternatives

digup is targeted at simple digest updating and verification without filtering capabilities or configuration files. For more complex applications like host-based intrusion detection, refer to tools like Tripwire (commercial), AIDE (open-source) or debsums for Debian/Linux.

Downloads

digup 0.6.40 (current) released 2011-01-30

Binaries:

Windows 95/98/Me/2000/NT/XP/Vista/7 Stand-alone 32-bit Binary

digup-0.6.40-win32.zip (52 KB)


Binaries packages for the Linux distributions above are available in the idlebox Repository. Refer to the individual installation instructions of your distribution (click the icon). If your Linux distribution is not listed, it may be based on one of the above. Otherwise fall back to compiling the source tarball below.
Alternatively to the dynamically linked binaries above, a statically linked, stand-alone Linux binary is also available: digup-0.6.40-static.tar.gz (304 KB)

Currently no pre-compiled binaries are available for further systems. However, the source code below was successfully compiled on FreeBSD, OpenBSD and MacOSX.

Source code:

Download digup-0.6.40.tar.bz2 (127 KB)

The source code is released under the GNU General Public License v2 (GPL). Some parts of the source tarball are copyrighted by different authors.

All sources and packages are available in the git repository via git clone https://github.com/bingmann/digup.git

See bottom of this page for older downloads.

Documentation

Please refer to the manpage of digup(1): also available online in HTML and PDF formats.

Screenshots

Screenshot of two digup runs on Linux

ChangeLog

2011-01-30 19:05 - Timo Bingmann - 0.6.40

src/digup.c: Adding shortcut option -w, --windows for --modify-window=1 as needed to check files on FAT filesystems.
src/digup.c: New feature: --restrict=substring to temporarily ignore all files not matching the substring. Required a new fileinfo state FS_SKIPPED and appropriate commands.

2011-01-13 17:11 - Timo Bingmann - 0.6.40

src/digup.c: Fixing problems with 64-bit file sizes on win32 under (mingw32): using _stat64 on mingw now.
src/digup.c: Simple variable rename node -> digestiter for clarity.

2011-01-13 17:04 - Timo Bingmann - 0.6.40

src/digup.c: Adding check in digest_file to verify that the complete file was read.

2011-01-13 17:01 - Timo Bingmann - 0.6.40

src/digup.c: Using intptr_t in rbtree tests to fix warnings about pointer conversion on 64-bit systems.

2010-11-29 18:05 - Timo Bingmann - 0.6.40

src/digup.c: Adding new flags --modify-window inspired by rsync: allow modification time deltas larger than zero e.g. for backuping on FAT filesystems.
src/digup.c: Adding new option --exclude-marker=file. If file is found in a directory during the recursive scan, the directory itself and all subdirectories are skipped.

2010-10-03 11:28 - Timo Bingmann - 0.6.30

src/digup.c: Applying contributed patch which fixes segfault on amd64 due to va_start(), va_end() use pattern.

2010-08-20 10:49 - Timo Bingmann - 0.6.27

src/digup.c: Fixed string allocation bug causing a segfault when using -t followed by -f parameters.

2010-03-20 14:27 - Timo Bingmann - 0.6.27

src/digup.c: Changing size variable from ssize_t to long long to enable support for large files.

2009-11-10 14:57 - Timo Bingmann - 0.6.23

Initial release of 0.6.23

Older Downloads

digup 0.6.30 released 2010-10-03
Source code:	Download digup-0.6.30.tar.bz2 (125 KB)

digup 0.6.27 released 2010-08-20
Source code:	Download digup-0.6.27.tar.bz2 (125 KB)

digup 0.6.23 released 2009-11-10
Source code:	Download digup-0.6.23.tar.bz2 (123 KB)