CoreLibs for Python

Warning

This is pre-production, location of methods and names of paths can change

This will be split up into modules per file and this will be just a collection holder

This is a pip package that can be installed into any project and covers the following parts

logging update with exception logs
requests wrapper for easier auth pass on access
dict fingerprinting
sending email
jmespath search
json helpers for conten replace and output
dump outputs for data for debugging
progress printing
string formatting, time creation, byte formatting
Enum base class
SQLite simple IO class
Symmetric encryption

Current list

config_handling: simple INI config file data loader with check/convert/etc
csv_interface: csv dict writer/reader helper
debug_handling: various debug helpers like data dumper, timer, utilization, etc
db_handling: SQLite interface class
encyption_handling: symmetric encryption
email_handling: simple email sending
file_handling: crc handling for file content and file names, progress bar
json_handling: jmespath support and json date support, replace content in dict with json paths
iterator_handling: list and dictionary handling support (search, fingerprinting, etc)
logging_handling: extend log and also error message handling
requests_handling: requests wrapper for better calls with auth headers
script_handling: pid lock file handling, abort timer
string_handling: byte format, datetime format, datetime compare, hashing, string formats for numbers, double byte string format, etc
var_handling: var type checkers, enum base class

Unfinished

csv_handling/csv_interface: The CSV DictWriter interface is just in a very basic way implemented
script_handling/script_helpers: No idea if there is need for this, tests are written but not finished

UV setup

uv must be installed

How to publish

Have the following setup in project.toml

[[tool.uv.index]]
name = "egra-gitea"
url = "https://git.egplusww.jp/api/packages/PyPI/pypi/simple/"
publish-url = "https://git.egplusww.jp/api/packages/PyPI/pypi"
explicit = true

uv build
uv publish --index egra-gitea --token <gitea token>

Test package

We must set the full index URL here because we run with "--no-project"

uv run --with corelibs --index egra-gitea=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/ --no-project -- python -c "import corelibs"

Python tests

All python tests are the tests/ folder. They are structured by the source folder layout

run them with

uv run pytest

Get a coverate report

uv run pytest --cov=corelibs
uv run pytest --cov=corelibs --cov-report=term-missing

Other tests

In the test-run folder usage and run tests are located, runt them below

uv run test-run/<script>

How to install in another project

This will also add the index entry

uv add corelibs --index egra-gitea=https://git.egplusww.jp/api/packages/PyPI/pypi/simple/

Python venv setup

After clone, run the command below to install all dependenciss

uv sync

NOTE on TLS problems

[!warning] TLS problems with Netskope

If the Netskope service is running all uv runs will fail unless either --native-tls is set or the enviroment variable SSL_CERT_FILE is set, see blow

export SSL_CERT_FILE='/Library/Application Support/Netskope/STAgent/data/nscacert_combined.pem'

3.4 KiB Raw Blame History