Skip to content

Latest commit

 

History

History
90 lines (72 loc) · 2.96 KB

README.md

File metadata and controls

90 lines (72 loc) · 2.96 KB
Raw

dbn-cli

build license Current Crates.io Version

This crate provides a CLI tool dbn for converting Databento Binary Encoding (DBN) files to text formats, as well as updating legacy DBZ files to DBN.

For more information about DBN, read our introduction to DBN.

Installation

To install the latest version, run the following command:

cargo install dbn-cli

Usage

dbn currently supports CSV and JSON (technically newline-delimited JSON) as output formats. By default, dbn outputs the result to standard output for ease of use with text-based command-line utilities. Running

dbn some.dbn --csv --limit 5

will print the header row and the first 5 data rows in some.dbn in CSV format to the console. Similarly, running

dbn ohlcv-1d.dbn.zst --json | jq '.high'

Will extract only the high prices from ohlcv-1d.dbn.zst. dbn works with both uncompressed and Zstandard-compressed DBN files.

You can also save the results directly to another file by running

dbn some.dbn.zst --json --output some.json

dbn will create a new file some.csv with the data from some.dbn.zst formatted as JSON.

When the file name passed --output or -o ends in .json or .csv, you can omit the --json and --csv flags.

dbn another.dbn.zst -o data.csv

This writes the contents of another.dbn.zst to data.json in CSV format.

By default, dbn will not overwrite an existing file. To replace the contents of an existing file and allow overwriting files, pass the -f or --force flag.

Compressing the output

In addition to reading Zstandard-compressed files, dbn can also write compressed JSON and CSV.

dbn ohlcv-1d.dbn -o ohclv-1d.json.zst

or explicitly

dbn ohlcv-1d.dbn --json --zstd -o ohlcv-1d.json.zst

Converting DBZ files to DBN

DBN is an evolution of DBZ, which required Zstandard. To update an old DBZ file to Zstandard-compressed DBN, run

dbn 20221212.mbo.dbz -o 20221212.dbn.zst

or pass --dbn to set the output encoding explicitly.

Reading and writing fragments

dbn can also read and write DBN files without the metadata header, these are called "DBN fragments". Pass the --input-fragment or --input-zstd-fragment flag to read a DBN file without a metadata header.

dbn 120000_121000.mbo.dbn.frag --input-fragment --json

You can also write DBN files without the metadata header with the --fragment or -F flag.

dbn ohlcv-1d.dbn -F -o ohlcv-1d.dbn.frag

License

Distributed under the Apache 2.0 License.