a76eae0cc7
previous string_handling located datetime and timestamp files have been moved to the datetime handling folder Update readme file with more information about currently covered areas
138 lines
3.3 KiB
Markdown
138 lines
3.3 KiB
Markdown
# CoreLibs for Python
|
|
|
|
> [!warning]
|
|
> This is pre-production, location of methods and names of paths can change
|
|
|
|
This is a pip package that can be installed into any project and covers the following parts
|
|
|
|
- logging update with exception logs
|
|
- requests wrapper for easier auth pass on access
|
|
- dict fingerprinting
|
|
- jmespath search
|
|
- json helpers for conten replace and output
|
|
- dump outputs for data for debugging
|
|
- progress printing
|
|
- string formatting, time creation, byte formatting
|
|
- Enum base class
|
|
- SQLite simple IO class
|
|
- Symmetric encryption
|
|
|
|
## Current list
|
|
|
|
- config_handling: simple INI config file data loader with check/convert/etc
|
|
- csv_handling: csv dict writer helper
|
|
- debug_handling: various debug helpers like data dumper, timer, utilization, etc
|
|
- db_handling: SQLite interface class
|
|
- encyption_handling: symmetric encryption
|
|
- file_handling: crc handling for file content and file names, progress bar
|
|
- json_handling: jmespath support and json date support, replace content in dict with json paths
|
|
- iterator_handling: list and dictionary handling support (search, fingerprinting, etc)
|
|
- logging_handling: extend log and also error message handling
|
|
- requests_handling: requests wrapper for better calls with auth headers
|
|
- script_handling: pid lock file handling, abort timer
|
|
- string_handling: byte format, datetime format, datetime compare, hashing, string formats for numbers, double byte string format, etc
|
|
- var_handling: var type checkers, enum base class
|
|
|
|
## UV setup
|
|
|
|
uv must be [installed](https://docs.astral.sh/uv/getting-started/installation/)
|
|
|
|
## How to publish
|
|
|
|
Have the following setup in `project.toml`
|
|
|
|
```toml
|
|
[[tool.uv.index]]
|
|
name = "egra-gitea"
|
|
url = "https://git.egplusww.jp/api/packages/PyPI/pypi/simple/"
|
|
publish-url = "https://git.egplusww.jp/api/packages/PyPI/pypi"
|
|
explicit = true
|
|
```
|
|
|
|
```sh
|
|
uv build
|
|
uv publish --index egra-gitea --token <gitea token>
|
|
```
|
|
|
|
## Test package
|
|
|
|
We must set the full index URL here because we run with "--no-project"
|
|
|
|
```sh
|
|
uv run --with corelibs --index egra-gitea=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/ --no-project -- python -c "import corelibs"
|
|
```
|
|
|
|
### Python tests
|
|
|
|
All python tests are the tests/ folder. They are structured by the source folder layout
|
|
|
|
run them with
|
|
|
|
```sh
|
|
uv run pytest
|
|
```
|
|
|
|
Get a coverate report
|
|
|
|
```sh
|
|
uv run pytest --cov=corelibs
|
|
```
|
|
|
|
### Other tests
|
|
|
|
In the test-run folder usage and run tests are located
|
|
|
|
#### Progress
|
|
|
|
```sh
|
|
uv run test-run/progress/progress_test.py
|
|
```
|
|
|
|
#### Double byte string format
|
|
|
|
```sh
|
|
uv run test-run/double_byte_string_format/double_byte_string_format.py
|
|
```
|
|
|
|
#### Strings helpers
|
|
|
|
```sh
|
|
uv run test-run/timestamp_strings/timestamp_strings.py
|
|
```
|
|
|
|
```sh
|
|
uv run test-run/string_handling/string_helpers.py
|
|
```
|
|
|
|
#### Log
|
|
|
|
```sh
|
|
uv run test-run/logging_handling/log.py
|
|
```
|
|
|
|
## How to install in another project
|
|
|
|
This will also add the index entry
|
|
|
|
```sh
|
|
uv add corelibs --index egra-gitea=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/
|
|
```
|
|
|
|
## Python venv setup
|
|
|
|
After clone, run the command below to install all dependenciss
|
|
|
|
```sh
|
|
uv sync
|
|
```
|
|
|
|
## NOTE on TLS problems
|
|
|
|
> [!warning] TLS problems with Netskope
|
|
|
|
If the Netskope service is running all uv runs will fail unless either --native-tls is set or the enviroment variable SSL_CERT_FILE is set, see blow
|
|
|
|
```sh
|
|
export SSL_CERT_FILE='/Library/Application Support/Netskope/STAgent/data/nscacert_combined.pem'
|
|
```
|