Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python bindings #48

Closed
wants to merge 59 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
87ca5f1
Foundations for Python bindings
larsnaesbye Nov 16, 2022
9a2f25d
python-bindings: basics
gwbres Nov 28, 2022
3b6754b
clarify maturin building
larsnaesbye Nov 28, 2022
d9e447d
python-bindings: basics
gwbres Nov 28, 2022
0dd07ea
add pip instructions
larsnaesbye Nov 28, 2022
90c1ab5
Merge branch 'main' into python-bindings
larsnaesbye Nov 30, 2022
e7fea37
python: correct Cargo.toml
gwbres Nov 30, 2022
e925c25
python: basics
gwbres Nov 30, 2022
e4dee0c
python: github workflow to generate the python wheel
gwbres Nov 30, 2022
b660dee
python.yml: add linux-x86 wheel
gwbres Nov 30, 2022
16e3567
python.yml: linux x86 does not work
gwbres Nov 30, 2022
9da50ce
python.yml: test advanced script, remove publication
gwbres Nov 30, 2022
2b51f9d
python.yml: sanitization
gwbres Nov 30, 2022
4a6ff20
Update python.yml
gwbres Nov 30, 2022
fc55030
python: fix wheel compilation on windows
gwbres Nov 30, 2022
c234b95
github: fix windows wheel build
gwbres Nov 30, 2022
7c59727
remove unused and outcommented
larsnaesbye Nov 30, 2022
cc45bf3
Merge branch 'main' into python-bindings
larsnaesbye Nov 30, 2022
0fcc683
observation: basics
gwbres Nov 30, 2022
2e231cc
python.yml: fix windows-x86 build
gwbres Nov 30, 2022
6690dfe
python.yml
gwbres Dec 1, 2022
60b0489
python: support and supply py3.8
gwbres Dec 1, 2022
105c5ac
python.yml
gwbres Dec 1, 2022
a43ff4b
rust.yml
gwbres Dec 1, 2022
d68d797
fix testbench: crinex day is 2 digit number
gwbres Dec 1, 2022
f7935b1
release.yml: cli binary packaging (#61)
gwbres Dec 1, 2022
c1e4b50
Merge branch 'gwbres:main' into python-bindings
larsnaesbye Dec 1, 2022
931bfe8
python.rs: add a few more bindings
gwbres Dec 2, 2022
98d926c
python.rs: add a few more bindings
gwbres Dec 2, 2022
834518f
add topic
larsnaesbye Dec 2, 2022
9de6af2
python.yml: archive, github release
gwbres Dec 2, 2022
1e3f528
python.yml: archive, github release
gwbres Dec 2, 2022
2aab1c7
python.yml
gwbres Dec 3, 2022
0e1376b
python.yml
gwbres Dec 3, 2022
97e1104
python.yml
gwbres Dec 3, 2022
061cff4
python.yml: package and release
gwbres Dec 3, 2022
3504e3e
python.yml
gwbres Dec 3, 2022
b25150a
simplify and align a bit with upstream
larsnaesbye Dec 18, 2022
5b0496e
Merge branch 'main' into python-bindings
larsnaesbye Dec 18, 2022
7a420fb
fix a few compile errors due to wrong merge
larsnaesbye Dec 18, 2022
4a69bb7
formatting
larsnaesbye Dec 18, 2022
31a43d5
add the new classes
larsnaesbye Dec 18, 2022
c5dbb0e
Merge branch 'main' into python-bindings
larsnaesbye Dec 27, 2022
b48bd47
Merge branch 'main' into python-bindings
larsnaesbye Mar 27, 2023
91974cc
Merge branch 'gwbres:main' into python-bindings
larsnaesbye May 18, 2023
7e28993
Merge branch 'gwbres:main' into python-bindings
larsnaesbye May 18, 2023
fb52f5d
update PyO3 to match HiFiTime
larsnaesbye May 28, 2023
fc26d0f
update to latest branch
gwbres Jun 11, 2023
bb5329b
declare ephemeris type
gwbres Jun 11, 2023
665c38b
use latest pyo3
gwbres Jun 11, 2023
381f584
version bindings
gwbres Jun 11, 2023
9951a82
python: correct build instructions
gwbres Jun 11, 2023
a86f836
add more bindings
gwbres Jun 11, 2023
71826d9
add more bindings
gwbres Jun 11, 2023
5e0ba37
working on complex cast
gwbres Jun 11, 2023
35e7ee1
Merge branch 'georust:main' into python-bindings
larsnaesbye Jun 22, 2023
db34c17
Merge branch 'georust:main' into python-bindings
larsnaesbye Jun 25, 2023
ed51e97
Merge branch 'georust:main' into python-bindings
larsnaesbye Jul 10, 2023
c9bbf0c
Merge branch 'georust:main' into python-bindings
larsnaesbye Jul 31, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 119 additions & 0 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
name: python

on:
push:
pull_request:
release:
types: [created]

jobs:
macos:
runs-on: macos-latest
continue-on-error: true
strategy:
matrix:
py_ver: ['3.8', '3.9', '3.10']
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.py_ver }}
- uses: dtolnay/rust-toolchain@stable
with:
targets: aarch64-apple-darwin
- name: Build
uses: PyO3/maturin-action@v1
with:
target: x86_64-apple-darwin
args: --release -i python${{ matrix.py_ver }} --all-features -m rinex/Cargo.toml --out dist --sdist
- name: Test wheel
run: |
pip install rinex --no-index --find-links dist --force-reinstall
python -c "import rinex"
- name: Prepare for upload
run: tar czvf rinex.tar.gz dist/*.whl
- name: Github Release
uses: svenstaro/upload-release-action@v2
if: "startsWith(github.ref, 'refs/tags/')"
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
asset_name: rinex-python${{ matrix.pyver }}-apple-darwin.tar.gz
file: rinex.tar.gz
tag: ${{ github.ref }}

windows:
runs-on: windows-latest
continue-on-error: true
strategy:
matrix:
target: [x64, x86]
py_ver: ['3.8', '3.9', '3.10']
steps:
- uses: actions/checkout@v3
- uses: dtolnay/rust-toolchain@stable
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.py_ver }}
architecture: ${{ matrix.target }}
- name: Build
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --all-features -m rinex/Cargo.toml --out dist
- name: Test wheel
run: |
pip install rinex --no-index --find-links dist --force-reinstall
python -c "import rinex"
- name: Prepare for upload
run: |
Compress-Archive dist/*.whl rinex.zip
dir
- name: Github Release
uses: svenstaro/upload-release-action@v2
if: "startsWith(github.ref, 'refs/tags/')"
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
asset_name: rinex-python${{ matrix.pyver }}-windows-${{ matrix.target }}.zip
file: rinex.zip
tag: ${{ github.ref }}

linux:
runs-on: ubuntu-latest
continue-on-error: true
strategy:
matrix:
target: [x64]
py_ver: ['3.8', '3.9', '3.10']
steps:
- uses: actions/checkout@v3
- uses: dtolnay/rust-toolchain@stable
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.py_ver }}
architecture: ${{ matrix.target }}
- name: Build
uses: PyO3/maturin-action@main
with:
target: x64
manylinux: auto
args: --release -i python${{ matrix.py_ver }} --all-features -m rinex/Cargo.toml -o dist
- name: Build
run: |
pip3 install rinex --no-index --find-links dist --force-reinstall
python -c "import rinex"
- name: Test wheel
run: |
pip install rinex --no-index --find-links dist --force-reinstall
python -c "import rinex"
- name: Prepare for upload
run: |
tar czvf rinex.tar.gz dist/*.whl
ls -lah rinex.tar.gz
- name: Github Release
uses: svenstaro/upload-release-action@v2
if: "startsWith(github.ref, 'refs/tags/')"
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
asset_name: rinex-python${{ matrix.pyver }}-linux-${{ matrix.target }}.tar.gz
file: rinex.tar.gz
tag: ${{ github.ref }}
4 changes: 4 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ name: release
on:
release:
types: [created]
pull_request:

env:
PKG_CONFIG_PATH: "/usr/lib/pkgconfig"

env:
PKG_CONFIG_PATH: "/usr/lib/pkgconfig"
Expand Down
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ RINEX
Rust tool suites to parse, analyze and process `RINEX` files

* [`rinex`](rinex/) is the core library

* [`rinex-cli`](rinex-cli/) is a command line application based on the core library.
It can be used to process RINEX files or perform operations similar to `teqc`.
The application is auto-generated for a few architectures, download it from the
Expand All @@ -28,6 +29,14 @@ and [rinex](rinex/) crates.

By default, all timestamps are in UTC with leap seconds correctly managed.

:warning: Years encoded on two digits in files generated prior Jan 1 2000,
get falsely offset to the 21st century. This only applies to OBS(V2)
and NAV(V2) files generated prior year 2000.

## `RINEX` in Python

Refer to the [python package](doc/python.md) to understand how to build the python3 wheel.

## Supported `RINEX` types

| Type | Parser | Writer | CLI | UBX | Notes |
Expand Down Expand Up @@ -91,6 +100,9 @@ select a SBAS for a given location on Earth.
* `--flate2`
allow native parsing of .gz compressed RINEX files. Otherwise, user must uncompress manually the `.gz` extension first.

* `--pyo3` (experimental)
Add Python bindings via `PyO3`. To build the Python package, you must first install maturin and then build it with the pyo3 feature flag. For example, `maturin build -F pyo3`. Maturin will then build and place the resulting .whl file in `/target/wheels/`, after which you can install the package with `pip install rinex`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be awesome to be able to auto generate the wheel

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it gets included with the release, and later uploaded to pypi.org ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it gets included with the release

yes, like the application binaries

and later uploaded to pypi.org ?

i did not think about going this far, but I don't foresee any reasons that would not be doable.

Copy link
Collaborator

@gwbres gwbres Nov 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use maturin-action

The wheels are generated for windows and linux.
The generation for macos is currently failing, due to a linking error when the bindings reach "hifitime".
I turned the publication off but it will easily be added.
First we will attach the wheels to the released package, like we do for the compiled -cli.
We can also add a publication to pypi, if you follow the previous link, some people do that.


## Benchmark

Test | Results
Expand Down
25 changes: 25 additions & 0 deletions doc/python.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Python3 wheel
=============

Install requirements: python3 (>3.8) and maturin

Then build the python bindings: use `maturin` to build the crate
with the `pyo3` feature

```bash
cd rinex/
maturin build -F pyo3
```

A pip3 wheel is generated for your architecture.
Use pip3 to install the library

```bash
pip3 install --force-reinstall rinex/target/wheels/rinex-xxx.whl
```

Now move on to the provided examples, run the basics with

```bash
python3 rinex/examples/python/basic.py
```
4 changes: 3 additions & 1 deletion rinex/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ rust-version = "1.61"
[features]
default = [] # no features by default
sbas = ["geo", "wkt"]
pyo3 = ["dep:pyo3", "hifitime/python"]
tests = [] # used to import shared test methods

[build-dependencies]
Expand All @@ -37,7 +38,8 @@ geo = { version = "0.26", optional = true }
wkt = { version = "0.10.0", default-features = false, optional = true }
serde = { version = "1.0", optional = true, default-features = false, features = ["derive"] }
flate2 = { version = "1.0.24", optional = true, default-features = false, features = ["zlib"] }
hifitime = { version = "3.8", features = ["serde", "std"] }
pyo3 = { version = "0.19", default-features = false, features = ["extension-module"], optional = true}
hifitime = { git = "https://github.com/nyx-space/hifitime", branch = "master", features = ["serde", "std"] }
horrorshow = { version = "0.8" }
statrs = "0.16"

Expand Down
36 changes: 36 additions & 0 deletions rinex/examples/python/basic.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
from rinex import *

# rinex::prelude basic examples,
# This example program depicts how you can interact with
# all the basic structures from the rust crate

def parser_example(fp):
# parse a RINEX file
rinex = Rinex(fp)
# use header section
print("is_crinex: ", rinex.header.is_crinex())
print("header : \n{:s}".format(str(rinex.header)))
# use record section
print(rinex.record)

def rinex_manual_constructor():
# Manual construction example.
# This is handy in data production contexts
header = Header.basic_obs()
print(header.is_crinex())

def sv_example():
pass

def constellation_example():
pass

def epoch_example():
print("Epoch.system_now(): ", Epoch.system_now())

if __name__ == "__main__":
parser_example("../test_resources/OBS/V3/DUTH0630.22O")
epoch_example()
sv_example()
rinex_manual_constructor()
constellation_example()
11 changes: 11 additions & 0 deletions rinex/examples/python/observation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
from rinex import *

if __name__ == "__main__":
crinex = Crinex()
assert(crinex.version.major == 3)
assert(crinex.version.minor == 0)

observation = ObservationData(10.0)
assert(observation.obs == 10.0)
assert(observation.snr == None)
assert(observation.lli == None)
16 changes: 16 additions & 0 deletions rinex/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[build-system]
requires = ["maturin>=0.14"]
build-backend = "maturin"

[project]
name = "rinex"
requires-python = ">=3.8"
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: Apache Software License",
"License :: OSI Approved :: MIT License",
gwbres marked this conversation as resolved.
Show resolved Hide resolved
"Topic :: Scientific/Engineering :: Physics",
"Topic :: Scientific/Engineering :: Atmospheric Science",
]

8 changes: 6 additions & 2 deletions rinex/src/constellation/augmentation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,17 @@
//! mainly used for high precision positioning
use strum_macros::EnumString;

#[cfg(feature = "pyo3")]
use pyo3::prelude::*;

#[cfg(feature = "serde")]
use serde::{Deserialize, Serialize};

#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, EnumString)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
/// GNSS Augmentation systems,
/// must be used based on current location
#[cfg_attr(feature = "pyo3", pyclass)]
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, EnumString)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub enum Augmentation {
/// Augmentation Unknown
Unknown,
Expand Down
56 changes: 56 additions & 0 deletions rinex/src/constellation/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ pub use augmentation::selection_helper;
#[cfg(feature = "serde")]
use serde::{Deserialize, Serialize};

#[cfg(feature = "pyo3")]
use pyo3::prelude::*;

#[derive(Error, Clone, Debug, PartialEq)]
/// Constellation parsing & identification related errors
pub enum Error {
Expand Down Expand Up @@ -47,6 +50,15 @@ pub enum Constellation {
Mixed,
}

#[cfg(feature = "pyo3")]
#[cfg_attr(feature = "pyo3", pyclass)]
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum PyConstellation {
GPS,
Glonass,
Geo,
}

impl std::fmt::Display for Constellation {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
f.write_str(self.to_3_letter_code())
Expand Down Expand Up @@ -185,6 +197,50 @@ impl std::str::FromStr for Constellation {
}
}

#[cfg(feature = "pyo3")]
impl IntoPy<PyObject> for Constellation {
fn into_py(self, py: Python<'_>) -> PyObject {
let pyc: PyConstellation = self.into();
pyc.into_py(py)
}
}

#[cfg(feature = "pyo3")]
impl From<Constellation> for PyConstellation {
fn from(c: Constellation) -> Self {
match c {
Constellation::SBAS(_) => Self::Geo,
Constellation::GPS => Self::GPS,
Constellation::Glonass => Self::Glonass,
_ => panic!("gnss not supported yet"),
}
}
}

#[cfg(feature = "pyo3")]
impl From<PyConstellation> for Constellation {
fn from(c: PyConstellation) -> Self {
match c {
PyConstellation::GPS => Self::GPS,
PyConstellation::Glonass => Self::Glonass,
PyConstellation::Geo => Self::Geo,
}
}
}

/*#[cfg(feature = "pyo3")]
use crate::prelude::Sv;
#[cfg(feature = "pyo3")]
impl From<&PyCell<Sv>> for Constellation {
fn from(cell: &PyCell<Sv>) -> Self {
if let Ok(sv) = cell.extract::<Sv>() {
sv.constellation
} else {
Self::default()
}
}
}*/

#[cfg(test)]
mod tests {
use super::*;
Expand Down
9 changes: 7 additions & 2 deletions rinex/src/epoch/flag.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,14 @@ pub enum Error {
UnknownFlag,
}

#[cfg(feature = "pyo3")]
use pyo3::prelude::*;

/// `EpochFlag` validates an epoch,
/// or describes possible events that occurred
#[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[cfg_attr(feature = "pyo3", pyclass)]
pub enum EpochFlag {
/// Epoch is sane
Ok,
Expand All @@ -37,10 +41,11 @@ impl Default for EpochFlag {
}
}

#[cfg_attr(feature = "pyo3", pymethods)]
impl EpochFlag {
/// Returns True if self is a valid epoch
pub fn is_ok(self) -> bool {
self == Self::Ok
pub fn is_ok(&self) -> bool {
*self == Self::Ok
}
}

Expand Down
Loading
Loading