CoreLibs for Python

Warning

This is pre-production, location of methods and names of paths can change

This will be split up into modules per file and this will be just a collection holder See Deprecated below

This is a pip package that can be installed into any project and covers the following parts

• logging update with exception logs
• requests wrapper for easier auth pass on access
• dict fingerprinting
• sending email
• jmespath search
• json helpers for conten replace and output
• dump outputs for data for debugging
• progress printing
• string formatting, time creation, byte formatting
• Enum base class
• SQLite simple IO class
• Symmetric encryption

Current list

• config_handling: simple INI config file data loader with check/convert/etc
• csv_interface: csv dict writer/reader helper
• debug_handling: various debug helpers like data dumper, timer, utilization, etc
• db_handling: SQLite interface class
• encyption_handling: symmetric encryption
• email_handling: simple email sending
• file_handling: crc handling for file content and file names, progress bar
• json_handling: jmespath support and json date support, replace content in dict with json paths
• iterator_handling: list and dictionary handling support (search, fingerprinting, etc)
• logging_handling: extend log and also error message handling
• requests_handling: requests wrapper for better calls with auth headers
• script_handling: pid lock file handling, abort timer
• string_handling: byte format, datetime format, datetime compare, hashing, string formats for numbers, double byte string format, etc
• var_handling: var type checkers, enum base class

Unfinished

• csv_handling/csv_interface: The CSV DictWriter interface is just in a very basic way implemented
• script_handling/script_helpers: No idea if there is need for this, tests are written but not finished

Deprecated

All content in this module will move to stand alone libraries, as of now the following entries have moved and will throw deprecated warnings if used

• var_handling.enum_base: corelibs-enum-base
• var_handling.var_helpers: corelibs-var
• datetime_handling: corelibs-datetime
• string_handling.text_colors: corelibs-text-colors

UV setup

uv must be installed

How to publish

Have the following setup in project.toml

[[tool.uv.index]]
name = "opj-pypi"
url = "https://git.egplusww.jp/api/packages/PyPI/pypi/simple/"
publish-url = "https://git.egplusww.jp/api/packages/PyPI/pypi"
explicit = true

uv build
uv publish --index opj-pypi --token <gitea token>

Use package

We must set the full index URL here because we run with "--no-project"

uv run --with corelibs --index opj-pypi=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/ --no-project -- python -c "import corelibs"

Python tests

All python tests are the tests/ folder. They are structured by the source folder layout

run them with

uv run pytest

Get a coverate report

uv run pytest --cov=corelibs
uv run pytest --cov=corelibs --cov-report=term-missing

Other tests

In the test-run folder usage and run tests are located, runt them below

uv run test-run/<script>

How to install in another project

This will also add the index entry

uv add corelibs --index opj-pypi=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/

Python venv setup

After clone, run the command below to install all dependenciss

uv sync

NOTE on TLS problems

[!warning] TLS problems with Netskope

If the Netskope service is running all uv runs will fail unless either --native-tls is set or the enviroment variable SSL_CERT_FILE is set, see blow

export SSL_CERT_FILE='/Library/Application Support/Netskope/STAgent/data/nscacert_combined.pem'