134 lines
3.8 KiB
Markdown
134 lines
3.8 KiB
Markdown
# CoreLibs for Python
|
|
|
|
> [!warning]
|
|
> This is pre-production, location of methods and names of paths can change
|
|
>
|
|
> This will be split up into modules per file and this will be just a collection holder
|
|
> See [Deprecated](#deprecated) below
|
|
|
|
This is a pip package that can be installed into any project and covers the following parts
|
|
|
|
- logging update with exception logs
|
|
- requests wrapper for easier auth pass on access
|
|
- dict fingerprinting
|
|
- sending email
|
|
- jmespath search
|
|
- json helpers for conten replace and output
|
|
- dump outputs for data for debugging
|
|
- progress printing
|
|
- string formatting, time creation, byte formatting
|
|
- Enum base class
|
|
- SQLite simple IO class
|
|
- Symmetric encryption
|
|
|
|
## Current list
|
|
|
|
- config_handling: simple INI config file data loader with check/convert/etc
|
|
- csv_interface: csv dict writer/reader helper
|
|
- debug_handling: various debug helpers like data dumper, timer, utilization, etc
|
|
- db_handling: SQLite interface class
|
|
- encyption_handling: symmetric encryption
|
|
- email_handling: simple email sending
|
|
- file_handling: crc handling for file content and file names, progress bar
|
|
- json_handling: jmespath support and json date support, replace content in dict with json paths
|
|
- iterator_handling: list and dictionary handling support (search, fingerprinting, etc)
|
|
- logging_handling: extend log and also error message handling
|
|
- requests_handling: requests wrapper for better calls with auth headers
|
|
- script_handling: pid lock file handling, abort timer
|
|
- string_handling: byte format, datetime format, datetime compare, hashing, string formats for numbers, double byte string format, etc
|
|
- var_handling: var type checkers, enum base class
|
|
|
|
## Unfinished
|
|
|
|
- csv_handling/csv_interface: The CSV DictWriter interface is just in a very basic way implemented
|
|
- script_handling/script_helpers: No idea if there is need for this, tests are written but not finished
|
|
|
|
## Deprecated
|
|
|
|
All content in this module will move to stand alone libraries, as of now the following entries have moved and will throw deprecated warnings if used
|
|
|
|
- var_handling.enum_base: corelibs-enum-base
|
|
- var_handling.var_helpers: corelibs-var
|
|
- datetime_handling: corelibs-datetime
|
|
- string_handling.text_colors: corelibs-text-colors
|
|
|
|
## UV setup
|
|
|
|
uv must be [installed](https://docs.astral.sh/uv/getting-started/installation/)
|
|
|
|
## How to publish
|
|
|
|
Have the following setup in `project.toml`
|
|
|
|
```toml
|
|
[[tool.uv.index]]
|
|
name = "opj-pypi"
|
|
url = "https://git.egplusww.jp/api/packages/PyPI/pypi/simple/"
|
|
publish-url = "https://git.egplusww.jp/api/packages/PyPI/pypi"
|
|
explicit = true
|
|
```
|
|
|
|
```sh
|
|
uv build
|
|
uv publish --index opj-pypi --token <gitea token>
|
|
```
|
|
|
|
## Use package
|
|
|
|
We must set the full index URL here because we run with "--no-project"
|
|
|
|
```sh
|
|
uv run --with corelibs --index opj-pypi=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/ --no-project -- python -c "import corelibs"
|
|
```
|
|
|
|
### Python tests
|
|
|
|
All python tests are the tests/ folder. They are structured by the source folder layout
|
|
|
|
run them with
|
|
|
|
```sh
|
|
uv run pytest
|
|
```
|
|
|
|
Get a coverate report
|
|
|
|
```sh
|
|
uv run pytest --cov=corelibs
|
|
uv run pytest --cov=corelibs --cov-report=term-missing
|
|
```
|
|
|
|
### Other tests
|
|
|
|
In the test-run folder usage and run tests are located, runt them below
|
|
|
|
```sh
|
|
uv run test-run/<script>
|
|
```
|
|
|
|
## How to install in another project
|
|
|
|
This will also add the index entry
|
|
|
|
```sh
|
|
uv add corelibs --index opj-pypi=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/
|
|
```
|
|
|
|
## Python venv setup
|
|
|
|
After clone, run the command below to install all dependenciss
|
|
|
|
```sh
|
|
uv sync
|
|
```
|
|
|
|
## NOTE on TLS problems
|
|
|
|
> [!warning] TLS problems with Netskope
|
|
|
|
If the Netskope service is running all uv runs will fail unless either --native-tls is set or the enviroment variable SSL_CERT_FILE is set, see blow
|
|
|
|
```sh
|
|
export SSL_CERT_FILE='/Library/Application Support/Netskope/STAgent/data/nscacert_combined.pem'
|
|
```
|