Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Made libumccr distribution light weight #24

Merged
merged 1 commit into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 28 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ UMCCR Reusable Python modules
## Usage

- Install through ``pip`` like so
```commandline
pip install libumccr
```
pip install libumccr[aws]
```

- Somewhere in your Python code
Expand All @@ -31,6 +31,32 @@ for obj in libs3.get_matching_s3_objects(bucket, prefix=key_prefix, suffix=key_s
print(f"s3://{bucket}/{obj['Key']}")
```

## Transitive Dependencies

### Self Service

- Starting from `v0.4` onwards, by default, the libumccr does not mandate any transitive dependencies installation for you.
- That is, if you `pip install libumccr` then it just installs the libumccr package itself only.
- If you do this, you will have to manage transitive dependencies at your application as _end-of-chain_ requirement as you see fit.
- This depends on which libumccr modules you need to depend on at your code.
- The _bare metal_ libumccr should only have some hundred KB (~180KB) by size. However, in this case, you can only use those modules that leverage built-in Python standard library (that does not require extra 3rd party library). Which might be perfect for use case like [libregex](libumccr/libregex.py) - well-known sample ID pattern parser contributed by team member ([Alexis](https://github.com/alexiswl)). You may contribute here similar and, reuse it at your end...

### Feature Flag

- If you would like to have all transitive dependencies requirement along with the libumccr, then do so `pip install libumccr[all]`.
- For any aws related module usage with boto3 and botocore, then `pip install libumccr[aws]`.
- Similarly, for gspread, gdrive and Pandas dataframe stuff, then `pip install libumccr[libgdrive]`.
- Do check out [setup.py](setup.py) `extras_require` sections for all available feature flags.

_This arrangement is sometime also known as deferring feature flag toggle at runtime requirement. Google for more..._

### AWS Lambda

- The AWS Lambda runtime for Python environment already contain the boto3 and botocore.
- Hence, if you are bundling your application for AWS Lambda Python runtime environment then you just need to `pip install libumccr`.
- The AWS Lambda Python runtime is already satisfied the `libumccr[aws]` feature requirement, and you can still leverage the [libumccr.aws](libumccr/aws) modules with the lowest asset bundle footprint.
- In this case, you just put `boto3` in your local development environment only; such as `reqirements-dev.txt` or equivalent.

## Development

- Create Python virtual environment
Expand Down
31 changes: 22 additions & 9 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

setup(
name="libumccr",
version="0.4.0rc1",
version="0.4.0rc2",
author="UMCCR and Contributors",
author_email="services@umccr.org",
description="UMCCR Reusable Python modules",
Expand Down Expand Up @@ -49,18 +49,31 @@
"nose2",
"awscli-local",
],
"libgdrive": [
"requests",
"pandas",
"gspread",
"gspread-pandas",
"google-auth",
],
"aws": [
"boto3",
"botocore",
"cachetools",
],
"all": [
"Django",
"boto3",
"botocore",
"cachetools",
"requests",
"pandas",
"gspread",
"gspread-pandas",
"google-auth",
],
},
install_requires=[
"boto3",
"botocore",
"cachetools",
"requests",
"pandas",
"gspread",
"gspread-pandas",
"google-auth",
# silent night
],
)
4 changes: 2 additions & 2 deletions tox.ini
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
[tox]
envlist = py{36,37,38,39}
envlist = py{36,37,38,39,310,311,312}
skip_missing_interpreters=True

[testenv]
passenv = GITHUB_*
deps =
.[test]
.[test,all]
commands =
pytest