Siegfried is a signature-based file format identification tool, implementing:
- the National Archives UK's PRONOM file format signatures
- freedesktop.org's MIME-info file format signatures
- the Library of Congress's FDD file format signatures (beta).
- Wikidata (beta).
1.10.1
sf file.ext
sf DIR
sf -csv file.ext | DIR // Output CSV rather than YAML
sf -json file.ext | DIR // Output JSON rather than YAML
sf -droid file.ext | DIR // Output DROID CSV rather than YAML
sf -nr DIR // Don't scan subdirectories
sf -z file.zip | DIR // Decompress and scan zip, tar, gzip, warc, arc
sf -zs gzip,tar file.tar.gz | DIR // Selectively decompress and scan
sf -hash md5 file.ext | DIR // Calculate md5, sha1, sha256, sha512, or crc hash
sf -sig custom.sig file.ext // Use a custom signature file
sf - // Scan stream piped to stdin
sf -name file.ext - // Provide filename when scanning stream
sf -f myfiles.txt // Scan list of files and directories
sf -v | -version // Display version information
sf -home c:\junk -sig custom.sig file.ext // Use a custom home directory
sf -serve hostname:port // Server mode
sf -throttle 10ms DIR // Pause for duration (e.g. 1s) between file scans
sf -multi 256 DIR // Scan multiple (e.g. 256) files in parallel
sf -log [comma-sep opts] file.ext | DIR // Log errors etc. to stderr (default) or stdout
sf -log e,w file.ext | DIR // Log errors and warnings to stderr
sf -log u,o file.ext | DIR // Log unknowns to stdout
sf -log d,s file.ext | DIR // Log debugging and slow messages to stderr
sf -log p,t DIR > results.yaml // Log progress and time while redirecting results
sf -log fmt/1,c DIR > results.yaml // Log instances of fmt/1 and chart results
sf -replay -log u -csv results.yaml // Replay results file, convert to csv, log unknowns
sf -setconf -multi 32 -hash sha1 // Save flag defaults in a config file
sf -setconf -serve :5138 -conf srv.conf // Save/load named config file with '-conf filename'
By default, siegfried uses the latest PRONOM signatures without buffer limits (i.e. it may do full file scans). To use MIME-info or LOC signatures, or to add buffer limits or other customisations, use the roy tool to build your own signature file.
go install github.com/richardlehane/siegfried/cmd/sf@latest
sf -update
Download a pre-built binary from the releases page. Unzip to a location in your system path. Then run:
sf -update
Mac Homebrew (or Linuxbrew):
brew install mistydemeo/digipres/siegfried
Or, for the most recent updates, you can install from this fork:
brew install richardlehane/digipres/siegfried
curl -sL "http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x20F802FE798E6857" | gpg --dearmor | sudo tee /usr/share/keyrings/siegfried-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/siegfried-archive-keyring.gpg] https://www.itforarchivists.com/ buster main" | sudo tee -a /etc/apt/sources.list.d/siegfried.list
sudo apt-get update && sudo apt-get install siegfried
pkg install siegfried
git clone https://aur.archlinux.org/siegfried.git
cd siegfried
makepkg -si
- format classification included as "class" field in PRONOM results. Requested by Robin François. Implemented by Ross Spencer
-noclassflag added to roy build command. Use this flag to build signatures that omit the new "class" field from results.- glob paths can be used in place of file or directory paths for identification (e.g.
sf *.jpg). Implemented by Ross Spencer -multi droidsetting for roy build command. Applies priorities after rather than during identificaiton for more DROID-like results. Reported by David Clipsham/updatecommand for server mode. Requested by Luis Faria
- new algorithm for dynamic multi-sequence matching for improved wildcard performance
- update PRONOM to v111
- update LOC to 2023-01-27
- update tika-mimetypes to v2.7.0
- archivematica extensions built into wikidata signatures. Reported by Ross Spencer
- trailing slash for folder paths in URI field in droid output. Reported by Philipp Wittwer
- crash when using
sf -replaywith droid output
- update PRONOM to v109
roy inspectnow takes a-droidflag to allow easier inspection of old or custom DROID files- github action to update siegfried docker deployment [https://github.com/keeps/siegfried-docker]. Implemented by Keep Solutions
- update PRONOM to v108
- update tika-mimetype signatures to v2.4.1
- update LOC signatures to 2022-09-01
- incorrect encoding of YAML strings containing line endings; #202.
- parse signatures with offsets and offsets in patterns e.g. fmt/1741; #203
- new pkg/static and static builds. This allows direct use of sf API and self-contained binaries without needing separate signature files.
- update PRONOM to v106
- inconsistent output for
roy inspect priorities. Reported by Dave Clipsham
- JS/WASM build support contributed by Andy Jackson
- wikidata signature added to
-update. Contributed by Ross Spencer -nopronomflag added toroy inspectsubcommand. Contributed by Ross Spencer
- update PRONOM to v104
- update LOC signatures to 2022-05-09
- update Wikidata to 2022-05-20
- update tika-mimetypes signatures to v2.4.0
- update freedesktop.org signatures to v2.2
- invalid JSON output for fmt/1472 due to tab in MIME field. Reported by Robert Schultz
- panic on corrupt Zip containers. Reported by A. Diamond
- Wikidata definition file specification has been updated and now includes endpoint (users will need to harvest Wikidata again)
- Custom Wikibase endpoint can now be specified for harvesting when paired with a custom SPARQL query and property mappings
- Wikidata identifier includes permalinks in results
- Wikidata revision history visible using
roy inspect - roy inspect returns format ID with name
- update PRONOM to v100
- update LOC signatures to 2022-02-01
- update tika-mimetypes signatures to v2.1
- update freedesktop.org signatures to v2.2.1
- parse issues for container files where zero indexing used for Position. Spotted by Ross Spencer
- sf -droid output can't be read by sf (e.g. for comparing results). Reported by ostnatalie
- panic when running in server mode due to race condition. Reported by Miguel Guimarães
- panic when reading malformed MSCFB files. Reported by Greg Lepore
- unescaped control characters in JSON output. Reported by Sebastian Lange
- zip file names with null terminated strings prevent ID of Serif formats. Reported by Tyler Thorsted
- update PRONOM to v97
- zs flag now activates -z flag
- details text in PRONOM identifier
roypanic when building signatures with empty sequences. Reported by Greg Lepore
- a new Wikidata identifier, harvesting information from the Wikidata Query Service. Implemented by Ross Spencer.
- select which archive types (zip, tar, gzip, warc, or arc) are unpacked using the -zs flag (sf -zs tar,zip). Implemented by Ross Spencer.
- update LOC signatures to 2020-09-21
- update tika-mimetypes signatures to v1.24
- update freedesktop.org signatures to v2.0
- incorrect basis for some signatures with multiple patterns. Reported and fixed by Ross Spencer.
See the CHANGELOG for the full history.
Copyright 2020 Richard Lehane, Ross Spencer
Licensed under the Apache License, Version 2.0
Join the Google Group for updates, signature releases, and help.
Like siegfried and want to get involved in its development? That'd be wonderful! There are some notes on the wiki to get you started, and please get in touch.
Thanks TNA for http://www.nationalarchives.gov.uk/pronom/ and http://www.nationalarchives.gov.uk/information-management/projects-and-work/droid.htm
Thanks Ross for https://github.com/exponential-decay/skeleton-test-suite-generator and http://exponentialdecay.co.uk/sd/index.htm, both are very handy!
Thanks Misty for the brew and ubuntu packaging
Thanks Steffen for the FreeBSD and Arch Linux packaging