c559a6bafb
list_helpers: convert to list, any input, output is always a list compare to lists, check what elements from A are not in B, type safe string helpers add is_int, is_float checker add string to bool converter for true/True/false/False strings config reader with parsing and checking The simple config reader is now in the corelibs with the basic content check, convert to list for entries, convert to value for entries, etc log updates: Add Log type Enum for better log level checks and convert Add a get int for requested log level, and return default if not found Make the validate log level a static function Add tests for list helpers and new string helpers
128 lines
3.0 KiB
Markdown
128 lines
3.0 KiB
Markdown
# CoreLibs for Python
|
|
|
|
This is a pip package that can be installed into any project and covers the following pars
|
|
|
|
- logging update with exception logs
|
|
- requests wrapper for easier auth pass on access
|
|
- dict fingerprinting
|
|
- jmespath search
|
|
- dump outputs for data
|
|
- progress printing
|
|
- string formatting, time creation, byte formatting
|
|
|
|
## Current list
|
|
|
|
- config_handling: simple INI config file data loader with check/convert/etc
|
|
- csv_handling: csv dict writer helper
|
|
- debug_handling: various debug helpers like data dumper, timer, utilization, etc
|
|
- file_handling: crc handling for file content and file names, progress bar
|
|
- json_handling: jmespath support and json date support
|
|
- iterator_handling: list and dictionary handling support (search, fingerprinting, etc)
|
|
- logging_handling: extend log and also error message handling
|
|
- requests_handling: requests wrapper for better calls with auth headers
|
|
- script_handling: pid lock file handling, abort timer
|
|
- string_handling: byte format, datetime format, hashing, string formats for numbrers, double byte string format, etc
|
|
|
|
## UV setup
|
|
|
|
uv must be [installed](https://docs.astral.sh/uv/getting-started/installation/)
|
|
|
|
## How to publish
|
|
|
|
Have the following setup in `project.toml`
|
|
|
|
```toml
|
|
[[tool.uv.index]]
|
|
name = "egra-gitea"
|
|
url = "https://git.egplusww.jp/api/packages/PyPI/pypi/simple/"
|
|
publish-url = "https://git.egplusww.jp/api/packages/PyPI/pypi"
|
|
explicit = true
|
|
```
|
|
|
|
```sh
|
|
uv build
|
|
uv publish --index egra-gitea --token <gitea token>
|
|
```
|
|
|
|
## Test package
|
|
|
|
We must set the full index URL here because we run with "--no-project"
|
|
|
|
```sh
|
|
uv run --with corelibs --index egra-gitea=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/ --no-project -- python -c "import corelibs"
|
|
```
|
|
|
|
### Python tests
|
|
|
|
All python tests are the tests/ folder. They are structured by the source folder layout
|
|
|
|
run them with
|
|
|
|
```sh
|
|
uv run pytest
|
|
```
|
|
|
|
Get a coverate report
|
|
|
|
```sh
|
|
uv run pytest --cov=corelibs
|
|
```
|
|
|
|
### Other tests
|
|
|
|
In the test-run folder usage and run tests are located
|
|
|
|
#### Progress
|
|
|
|
```sh
|
|
uv run test-run/progress/progress_test.py
|
|
```
|
|
|
|
#### Double byte string format
|
|
|
|
```sh
|
|
uv run test-run/double_byte_string_format/double_byte_string_format.py
|
|
```
|
|
|
|
#### Strings helpers
|
|
|
|
```sh
|
|
uv run test-run/timestamp_strings/timestamp_strings.py
|
|
```
|
|
|
|
```sh
|
|
uv run test-run/string_handling/string_helpers.py
|
|
```
|
|
|
|
#### Log
|
|
|
|
```sh
|
|
uv run test-run/logging_handling/log.py
|
|
```
|
|
|
|
## How to install in another project
|
|
|
|
This will also add the index entry
|
|
|
|
```sh
|
|
uv add corelibs --index egra-gitea=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/
|
|
```
|
|
|
|
## Python venv setup
|
|
|
|
After clone, run the command below to install all dependenciss
|
|
|
|
```sh
|
|
uv sync
|
|
```
|
|
|
|
## NOTE on TLS problems
|
|
|
|
> [!warning] TLS problems with Netskope
|
|
|
|
If the Netskope service is running all uv runs will fail unless either --native-tls is set or the enviroment variable SSL_CERT_FILE is set, see blow
|
|
|
|
```sh
|
|
export SSL_CERT_FILE='/Library/Application Support/Netskope/STAgent/data/nscacert_combined.pem'
|
|
```
|