# CoreLibs for Python

> [!warning]
> This is pre-production, location of methods and names of paths can change
>
> This will be split up into modules per file and this will be just a collection holder
> See [Deprecated](#deprecated) below

This is a pip package that can be installed into any project and covers the following parts

- logging update with exception logs
- requests wrapper for easier auth pass on access
- dict fingerprinting
- sending email
- jmespath search
- json helpers for conten replace and output
- dump outputs for data for debugging
- progress printing
- string formatting, time creation, byte formatting
- Enum base class
- SQLite simple IO class
- Symmetric encryption

## Current list

- config_handling: simple INI config file data loader with check/convert/etc
- csv_interface: csv dict writer/reader helper
- debug_handling: various debug helpers like data dumper, timer, utilization, etc
- db_handling: SQLite interface class
- encyption_handling: symmetric encryption
- email_handling: simple email sending
- file_handling: crc handling for file content and file names, progress bar
- json_handling: jmespath support and json date support, replace content in dict with json paths
- iterator_handling: list and dictionary handling support (search, fingerprinting, etc)
- logging_handling: extend log and also error message handling
- requests_handling: requests wrapper for better calls with auth headers
- script_handling: pid lock file handling, abort timer
- string_handling: byte format, datetime format, datetime compare, hashing, string formats for numbers, double byte string format, etc
- var_handling: var type checkers, enum base class

## Unfinished

- csv_handling/csv_interface: The CSV DictWriter interface is just in a very basic way implemented
- script_handling/script_helpers: No idea if there is need for this, tests are written but not finished

## Deprecated

All content in this module will move to stand alone libraries, as of now the following entries have moved and will throw deprecated warnings if used

- check_handling.regex_constants_compiled: corelibs-regex-checks
- check_handling.regex_constants: corelibs-regex-checks
- csv_handling.csv_interface: corelibs-csv
- datetime_handling.datetime_helpers: corelibs-datetime
- datetime_handling.timestamp_convert: corelibs-datetime
- datetime_handling.timestamp_strings: corelibs-datetime
- debug_handling.debug_helpers: corelibs-stack-trace
- debug_handling.dump_data: corelibs-dump-data
- debug_handling.profiling: corelibs-debug
- debug_handling.timer: corelibs-debug
- debug_handling.writeline: corelibs-debug
- encryption_handling.symmetrix_encryption: corelibs-encryption
- exceptions.csv_exceptions: orelibs-csv
- file_handling.file_bom_encoding: corelibs-file
- file_handling.file_crc: corelibs-file
- file_handling.file_handling: corelibs-file
- iterator_handling.data_search: corelibs-search
- iterator_handling.dict_helpers: corelibs-iterator
- iterator_handling.dict_mask: corelibs-dump-data
- iterator_handling.fingerprint: corelibs-hash
- iterator_handling.list_helpers: corelibs-iterator
- json_handling.jmespath_helper: corelibs-search
- json_handling.json_helper: corelibs-json
- math_handling.math_helpers: python.math
- requests_handling.auth_helpers: corelibs-requests
- requests_handling.caller: corelibs-requests
- string_handling.text_colors: corelibs-text-colors
- var_handling.enum_base: corelibs-enum-base
- var_handling.var_helpers: corelibs-var

## UV setup

uv must be [installed](https://docs.astral.sh/uv/getting-started/installation/)

## How to publish

Have the following setup in `project.toml`

```toml
[[tool.uv.index]]
name = "opj-pypi"
url = "https://git.egplusww.jp/api/packages/PyPI/pypi/simple/"
publish-url = "https://git.egplusww.jp/api/packages/PyPI/pypi"
explicit = true
```

```sh
uv build
uv publish --index opj-pypi --token <gitea token>
```

## Use package

We must set the full index URL here because we run with "--no-project"

```sh
uv run --with corelibs --index opj-pypi=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/ --no-project -- python -c "import corelibs"
```

### Python tests

All python tests are the tests/ folder. They are structured by the source folder layout

run them with

```sh
uv run pytest
```

Get a coverate report

```sh
uv run pytest --cov=corelibs
uv run pytest --cov=corelibs --cov-report=term-missing
```

### Other tests

In the test-run folder usage and run tests are located, runt them below

```sh
uv run test-run/<script>
```

## How to install in another project

This will also add the index entry

```sh
uv add corelibs --index opj-pypi=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/
```

## Python venv setup

After clone, run the command below to install all dependenciss

```sh
uv sync
```

## NOTE on TLS problems

> [!warning] TLS problems with Netskope

If the Netskope service is running all uv runs will fail unless either --native-tls is set or the enviroment variable SSL_CERT_FILE is set, see blow

```sh
export SSL_CERT_FILE='/Library/Application Support/Netskope/STAgent/data/nscacert_combined.pem'
```