Python bindings #48

larsnaesbye · 2022-11-16T16:47:29Z

expose basic structures like epoch::EpochFlag or observation::LliFlags
modules
convert record structures (Vec<V> or Map<K, V>)
we need a solution for the SBAS nested enum contained in Constellation, pyclass auto derivation does not allow that
provide meaningful examples in python
wheel auto generation and release
readme/docs

* Cargo.toml: we need hifitime with "python" * epoch naturally comes included * add basis for "Rinex" struct Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

rinex/Cargo.toml

* Cargo.toml: we need hifitime with "python" * epoch naturally comes included * convert epoch flag to pyobject Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

gwbres · 2022-11-28T21:45:12Z

rinex/src/epoch/flag.rs

 /// `EpochFlag` validates an epoch, 
 /// or describes possible events that occurred
 #[derive(Copy, Clone, Debug)]
 #[derive(PartialEq, Eq, PartialOrd, Ord, Hash)]
+#[cfg_attr(feature = "pyo3", pyclass)]


@larsnaesbye ,
let's start easy with basic structures that do not have any dependents.

We'll have to solve several issues before we can derive pyclass for Rinex.

We can't derive pyclass for Constellation because Pyo3 does not allow nested enums like we have for SBAS description for example. Hopefully this is an open topic on their side? i don't know.

Then we need to describe how to convert "complex" structures. Mainly: all record types and vectorized stuff. Apparently this is doable! but I just don't understand how to do it. An auto derivation method does not seem to exist. I think we need to move on to manual derivation (Trait implementation)

Here is the conversion guideline we should follow for Python<->Rust conversion

that would be something like IntoPyList for vectors, and IntoPyDict for all record types ?

Finally, Errors must be converted to PyErr or exceptions. This is more documented and we can find examples easily on the web. If you look at the Hifitime/python.rs, they convert their main Error type to a PyErr object with a simple manual trait implementation. We will probably have to do that for Rinex::Error

I just pushed the basics, it works :)
An example is attached and will get incremented.
Now we need to tackle the hardest parts 😄

and we probably have to submit a PR to PyO3 for the nested Enum case

we don't necessarily have to go this far. The immediate problem is the pyclass auto derivation does not support nested enums, that means this macro cannot dictate the code in this scenario.
We just have to to the job ourselves.
And it's probably a manual IntoPyObject implementation like these ones

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

gwbres · 2022-11-30T10:06:27Z

README.md

@@ -90,6 +95,9 @@ select a SBAS for a given location on Earth.
 * `--flate2`  
 allow native parsing of .gz compressed RINEX files. Otherwise, user must uncompress manually the `.gz` extension first.

+* `--pyo3` (experimental)
+Add Python bindings via `PyO3`. To build the Python package, you must first install maturin and then build it with the pyo3 feature flag. For example, `maturin build -F pyo3`. Maturin will then build and place the resulting .whl file in `/target/wheels/`, after which you can install the package with `pip install rinex`.


It would be awesome to be able to auto generate the wheel

So it gets included with the release, and later uploaded to pypi.org ?

So it gets included with the release

yes, like the application binaries

and later uploaded to pypi.org ?

i did not think about going this far, but I don't foresee any reasons that would not be doable.

We use maturin-action

The wheels are generated for windows and linux.
The generation for macos is currently failing, due to a linking error when the bindings reach "hifitime".
I turned the publication off but it will easily be added.
First we will attach the wheels to the released package, like we do for the compiled -cli.
We can also add a publication to pypi, if you follow the previous link, some people do that.

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

gwbres · 2022-11-30T22:34:35Z

rinex/src/python.rs

+     */
+    m.add_class::<Epoch>()?;
+    m.add_class::<EpochFlag>()?;
+    /*


we should design python modules here, following the crate architecture. This would allow for example, when processing Observations:

from rinex import prelude from rinex import observation

gwbres · 2022-12-01T08:25:51Z

For the moment, I will only focus on solving the MacOS builder and unlock the wheel packager.
We have the basis for most of the crate.
We still need to figure out what needs to be done for Record structures (dictionaries, Map<K, V>) and we probably have to submit a PR to PyO3 for the nested Enum case

larsnaesbye · 2022-12-01T08:31:49Z

and we probably have to submit a PR to PyO3 for the nested Enum case.

Strange that they have not encountered this need before. But I agree.

larsnaesbye · 2022-12-18T16:35:26Z

@gwbres I resolved the conflict - now on to solving the errors :-D

larsnaesbye · 2023-01-29T10:02:44Z

Rebase done (in a roundabout way, I know...)

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

larsnaesbye · 2023-09-07T13:20:31Z

Should this work start all over, with the new codebase, new PyO3 and maturin etc.? It's beginning to get a bit hairy....

gwbres · 2023-09-07T13:49:08Z

Should this work start all over, with the new codebase, new PyO3 and maturin etc.? It's beginning to get a bit hairy....

We only have a couple of useful commits on this branch, it should not take too long to start again from scratch, and is probably easier.

Now I understand that we must provide alternative structures, for structures that cannot be bound right away.
Cast in both ways must be provided

larsnaesbye · 2023-09-07T14:02:57Z

Okay, in that case I will close the pull request, so we can start over when we have the time and energy :-)

larsnaesbye · 2023-09-07T14:03:40Z

Closing to make way for a better PR later.

gwbres added enhancement New feature provided v1.0.0 labels Nov 16, 2022

gwbres added this to the v1.0.0 milestone Nov 21, 2022

larsnaesbye and others added 2 commits November 28, 2022 21:05

Foundations for Python bindings

87ca5f1

python-bindings: basics

9a2f25d

* Cargo.toml: we need hifitime with "python" * epoch naturally comes included * add basis for "Rinex" struct Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

gwbres reviewed Nov 28, 2022

View reviewed changes

rinex/Cargo.toml Outdated Show resolved Hide resolved

larsnaesbye and others added 3 commits November 28, 2022 22:32

clarify maturin building

3b6754b

python-bindings: basics

d9e447d

* Cargo.toml: we need hifitime with "python" * epoch naturally comes included * convert epoch flag to pyobject Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

add pip instructions

0dd07ea

gwbres reviewed Nov 28, 2022

View reviewed changes

gwbres self-requested a review November 29, 2022 08:26

larsnaesbye and others added 3 commits November 30, 2022 08:09

Merge branch 'main' into python-bindings

90c1ab5

python: correct Cargo.toml

e7fea37

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

python: basics

e925c25

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

gwbres reviewed Nov 30, 2022

View reviewed changes

gwbres and others added 11 commits November 30, 2022 19:59

python: github workflow to generate the python wheel

e4dee0c

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

python.yml: add linux-x86 wheel

b660dee

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

python.yml: linux x86 does not work

16e3567

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

python.yml: test advanced script, remove publication

9da50ce

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

python.yml: sanitization

2b51f9d

Update python.yml

4a6ff20

python: fix wheel compilation on windows

fc55030

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

github: fix windows wheel build

c234b95

remove unused and outcommented

7c59727

Merge branch 'main' into python-bindings

cc45bf3

observation: basics

0fcc683

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

gwbres reviewed Nov 30, 2022

View reviewed changes

python.yml: fix windows-x86 build

2e231cc

larsnaesbye added 4 commits December 18, 2022 17:51

fix a few compile errors due to wrong merge

7a420fb

formatting

4a69bb7

add the new classes

31a43d5

Merge branch 'main' into python-bindings

c5dbb0e

larsnaesbye closed this Jan 27, 2023

larsnaesbye reopened this Jan 28, 2023

Merge branch 'main' into python-bindings

b48bd47

larsnaesbye added the help wanted Extra attention is needed label May 17, 2023

larsnaesbye and others added 15 commits May 18, 2023 15:39

Merge branch 'gwbres:main' into python-bindings

91974cc

Merge branch 'gwbres:main' into python-bindings

7e28993

update PyO3 to match HiFiTime

fb52f5d

update to latest branch

fc26d0f

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

declare ephemeris type

bb5329b

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

use latest pyo3

665c38b

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

version bindings

381f584

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

python: correct build instructions

9951a82

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

add more bindings

a86f836

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

add more bindings

71826d9

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

working on complex cast

5e0ba37

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>

Merge branch 'georust:main' into python-bindings

35e7ee1

Merge branch 'georust:main' into python-bindings

db34c17

Merge branch 'georust:main' into python-bindings

ed51e97

Merge branch 'georust:main' into python-bindings

c9bbf0c

larsnaesbye closed this Sep 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python bindings #48

Python bindings #48

larsnaesbye commented Nov 16, 2022 •

edited by gwbres

Loading

gwbres Nov 28, 2022 •

edited

Loading

gwbres Nov 30, 2022 •

edited

Loading

gwbres Dec 2, 2022

gwbres Nov 30, 2022

larsnaesbye Nov 30, 2022

gwbres Nov 30, 2022

gwbres Nov 30, 2022 •

edited

Loading

gwbres Nov 30, 2022 •

edited

Loading

gwbres commented Dec 1, 2022

larsnaesbye commented Dec 1, 2022

larsnaesbye commented Dec 18, 2022

larsnaesbye commented Jan 29, 2023

larsnaesbye commented Sep 7, 2023

gwbres commented Sep 7, 2023 •

edited

Loading

larsnaesbye commented Sep 7, 2023

larsnaesbye commented Sep 7, 2023

Python bindings #48

Python bindings #48

Conversation

larsnaesbye commented Nov 16, 2022 • edited by gwbres Loading

gwbres Nov 28, 2022 • edited Loading

Choose a reason for hiding this comment

gwbres Nov 30, 2022 • edited Loading

Choose a reason for hiding this comment

gwbres Dec 2, 2022

Choose a reason for hiding this comment

gwbres Nov 30, 2022

Choose a reason for hiding this comment

larsnaesbye Nov 30, 2022

Choose a reason for hiding this comment

gwbres Nov 30, 2022

Choose a reason for hiding this comment

gwbres Nov 30, 2022 • edited Loading

Choose a reason for hiding this comment

gwbres Nov 30, 2022 • edited Loading

Choose a reason for hiding this comment

gwbres commented Dec 1, 2022

larsnaesbye commented Dec 1, 2022

larsnaesbye commented Dec 18, 2022

larsnaesbye commented Jan 29, 2023

larsnaesbye commented Sep 7, 2023

gwbres commented Sep 7, 2023 • edited Loading

larsnaesbye commented Sep 7, 2023

larsnaesbye commented Sep 7, 2023

larsnaesbye commented Nov 16, 2022 •

edited by gwbres

Loading

gwbres Nov 28, 2022 •

edited

Loading

gwbres Nov 30, 2022 •

edited

Loading

gwbres Nov 30, 2022 •

edited

Loading

gwbres Nov 30, 2022 •

edited

Loading

gwbres commented Sep 7, 2023 •

edited

Loading