Merge branch 'development' into refactor/TTD-3115/shellcheck-cleanup

Update gitignore file with backup lock files
Bump version to v4.8.0
2025-12-04 11:46:31 +09:00 · 2025-12-04 09:57:54 +09:00 · 2025-12-04 09:35:08 +09:00 · 2025-12-04 09:28:57 +09:00 · 2025-12-03 15:49:45 +09:00 · 2025-12-03 15:31:55 +09:00
20 changed files with 1403 additions and 702 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -1,8 +1,11 @@
 .cache/
 .config/
+# various borg files that need to be ignored
 borg.backup.settings
 borg.backup.*.settings
 borg.backup.*.include
 borg.backup.*.exclude
 borg.backup.*.schema-only
 borg.backup.*.init
+borg.backup.*.compact
+borg.backup.*.lock
--- a/.shellcheckrc
+++ b/.shellcheckrc
@@ -0,0 +1,2 @@
+shell=bash
+external-sources=true
--- a/Readme.md
+++ b/Readme.md
@@ -13,9 +13,11 @@ Version 4.0 introduces default borg repository name with `-file` for the `file`
 *Example:*

 Old backup name
+
 ```sh
 BACKUP_FILE="some-backup-data.borg"
 ```
+
 Then the file need to be renamed the following way:
 `mv some-backup-data.borg some-backup-data-file.borg`

@@ -42,48 +44,79 @@ No settings files will be overwritten
 ## Possible command line options

 ### `-c <config folder>`
+
 if this is not given, /usr/local/scripts/borg/ is used

 ### `-L <log folder>`
+
 override config set and default log folder

 ### `-T <tag>`
+
 create one time stand alone backup prefixed with tag name

 ### `-D <tag backup set>`
+
 remove a tagged backup set, full name must be given

 ### `-b <borg executable>`
+
 override the default borg executable found in path

 ### `-P`
+
 print list of archives created

-### `-C`
-check if repository exists, if not abort
+### `-V`

-### `-E`
-exit after check
+verify if repository exists, if not abort
+
+### `-e`
+
+exit after running verify `-V`

 ### `-I`
+
 init repository (must be run first)

+### `-Z`
+
+run `borg compact` over given repository
+
+### `-C`
+
+run `borg check` over given repository
+
+#### `-y`
+
+Add `--verify-data` to `borg check`. Only works with `-C`
+
+#### `-p <prefix|glob>`
+
+Only `borg check` data that has given prefix or glob (with *). Only works with `-C`
+
 ### `-i`
+
 print out only info

 ### `-l`
+
 list files during backup

 ### `-v`
+
 be verbose

 ### `-d`
+
 debug output all commands

 ### `-n`
+
 only do dry run

 ### `-h`
+
 this help page

 ## Basic Settings
@@ -97,7 +130,6 @@ LOG_FOLDER: default `/var/log/borg.backup/`
 TARGET_FOLDER: must be set to a path where the backups can be written
 BACKUP_FILE: the folder inside the TARGET_FOLDER that is the target for borg. Must end with `.borg`

-
 Note: BACKUP_FILE is the base name. For all except file (current) a module suffix will be added:

 eg:
@@ -106,35 +138,41 @@ eg:
 If `FILE_REPOSITORY_COMPATIBLE` is set to `false` in the borg.backup.file.settings then the file borg name will have `-file` added too. Currently this is not added to stay compatible with older scripts

 All below have default values if not set in the main settings file
- * COMPRESSION: zstd
- * COMPRESSION_LEVEL: 3
- * ENCRYPTION: none
- * FORCE_CHECK: false
- * KEEP_LAST: 0
- * KEEP_HOURS: 0
- * KEEP_DAYS: 7
- * KEEP_WEEKS: 4
- * KEEP_MONTHS: 6
- * KEEP_YEARS: 1
+
+* COMPRESSION: zstd
+* COMPRESSION_LEVEL: 3
+* ENCRYPTION: keyfile (with no password)
+* FORCE_VERIFY: false
+* COMPACT_INTERVAL: 1
+* CHECK_INTERVAL: none
+* KEEP_LAST: 0
+* KEEP_HOURS: 0
+* KEEP_DAYS: 7
+* KEEP_WEEKS: 4
+* KEEP_MONTHS: 6
+* KEEP_YEARS: 1

 All module settings files can have the following prefixed with `SUB_` to override master settings:
- * SUB_BACKUP_FILE
- * SUB_COMPRESSION
- * SUB_COMPRESSION_LEVEL
- * SUB_BACKUP_SET
- * SUB_KEEP_LAST
- * SUB_KEEP_HOURS
- * SUB_KEEP_DAYS
- * SUB_KEEP_WEEKS
- * SUB_KEEP_MONTHS
- * SUB_KEEP_YEARS
- * SUB_KEEP_WITHIN
+
+* SUB_BACKUP_FILE
+* SUB_COMPRESSION
+* SUB_COMPRESSION_LEVEL
+* SUB_COMPACT_INTERVAL
+* SUB_CHECK_INTERVAL
+* SUB_BACKUP_SET
+* SUB_KEEP_LAST
+* SUB_KEEP_HOURS
+* SUB_KEEP_DAYS
+* SUB_KEEP_WEEKS
+* SUB_KEEP_MONTHS
+* SUB_KEEP_YEARS
+* SUB_KEEP_WITHIN

 ## Setup backup via SSH to remote host on `borg.backup.settings`

 For this the following settings are from interest

-```
+```sh
 TARGET_USER="";
 TARGET_HOST="";
 TARGET_PORT="";
@@ -148,6 +186,14 @@ and `TARGET_BORG_PATH="";` if the target borg is in a non default path

 `BORG_EXECUTABLE="<full path to borg>"`

+## Note on CHECK_INTERVAL and SUB_CHECK_INTERVAL
+
+If set to empty or 0 it will not run an automatic check. If set to 1 it will run a check after each backup. Any other value means days differente to the last check.
+
+Running check manually (`-C`) will not reset the last check timestamp.
+
+Automatic checks always add `--verify-data`, with manual `-C` the option `-y` has to be set.
+
 ## File backup settings

 On new setups it is recommended to use the `borg.backup.file.setings` and set
@@ -156,10 +202,9 @@ to `true`

 ### Config variables

-
 ### Control files

-```
+```sh
 backup.borg.file.include
 backup.borg.file.exclude
 ```
@@ -173,16 +218,16 @@ backup.borg.file.exclude
 This script must be run as the postgres user, normaly `postgres`.
 The postgres user must be added to the backup group for this, so that the basic init file can be created in the borg base folder.

-### Config variables
+### PostgreSQL Config variables

-Variable | Default | Description
+| Variable | Default | Description |
 | - | - | - |
-DATABASE_FULL_DUMP | | if empty, dump per databse, if set dump all in one file, if set to schema dump only schema
-DATABASE_USER | | overide username to connect to database
+| DATABASE_FULL_DUMP | | if empty, dump per databse, if set dump all in one file, if set to schema dump only schema |
+| DATABASE_USER | | overide username to connect to database |

-### Control files
+### PostgreSQL Control files

-```
+```sh
 backup.borg.pgsql.include
 backup.borg.pgsql.exclude
 backup.borg.pgsql.schema-only
@@ -192,16 +237,16 @@ backup.borg.pgsql.schema-only

 If non root ident authentication run is used, be sure that the `mysql` user is in the backup group.

-### Config variables
+### MySQL Config variables

-Variable | Default | Description
+| Variable | Default | Description |
 | - | - | - |
-DATABASE_FULL_DUMP | | if empty, dump per databse, if set dump all in one file, if set to schema dump only schema
-MYSQL_DB_CONFIG | | override file for connection. In modern mariaDB installations it is rcommended to run the script as root or mysql user and use the ident authentication instead.
+| DATABASE_FULL_DUMP | | if empty, dump per databse, if set dump all in one file, if set to schema dump only schema |
+| MYSQL_DB_CONFIG | | override file for connection. In modern mariaDB installations it is rcommended to run the script as root or mysql user and use the ident authentication instead. |

-### Control files
+### MySQLControl files

-```
+```sh
 backup.borg.mysql.include
 backup.borg.mysql.exclude
 backup.borg.mysql.schema-only
@@ -212,31 +257,52 @@ backup.borg.mysql.schema-only
 Note that the backup needs the GIT_USER set that runs gitea.
 This user is neede to create the temporary dump folder and access for the git files and database.

-### Config Variables
+### gitea Config Variables

-Variable | Default | Description
+| Variable | Default | Description |
 | - | - | - |
-GIT_USER | git | The user that runs gitea |
-GITEA_TMP | /tmp/gitea/ | Where the temporary dump files from the backup are stored, as user git |
-GITEA_BIN | /usr/local/bin/gitea | Where the gitea binary is located |
-GITEA_CONFIG | /etc/gitea/app.ini | The configuration file for gitea |
+| GIT_USER | git | The user that runs gitea |
+| GITEA_WORKING_DIR | /var/tmp/gitea/ | Where the temporary dump files from the backup are stored, as user git |
+| GITEA_TEMP_DIR | /var/tmp/ | General temporary folder |
+| GITEA_BIN | /usr/local/bin/gitea | Where the gitea binary is located |
+| GITEA_CONFIG | /etc/gitea/app.ini | The configuration file for gitea |

-
-### Control files
+### gitea Control files

 There are no control files for gitea backup

 ## zabbix config backup settings

-### Config Variables
+The `zabbix-dump` dump script must be installed from: <https://github.com/gullevek/zabbix-backup>

-Variable | Default | Description
+### zabbix Config Variables
+
+| Variable | Default | Description |
 | - | - | - |
-ZABBIX_DUMP | /usr/local/bin/zabbix-dump |
-ZABBIX_DATABASE | '' | Must be set as either psql or mysql
-ZABBIX_CONFIG | '' | if not set uses default location
-ZABBIX_UNKNOWN_TABLES | '' | if set, changed to -f (force)
+| ZABBIX_DUMP | /usr/local/bin/zabbix-dump | |
+| ZABBIX_DATABASE | '' | Must be set as either psql or mysql |
+| ZABBIX_CONFIG | '' | if not set uses default location |
+| ZABBIX_UNKNOWN_TABLES | '' | if set, changed to -f (force) |
+| ZABBIX_DB_PORT | '' | if set, sets -P parameter on zabbix backup script to override auto get from zabbix server config |

-### Control files
+### zabbix Control files

 There are no control files for zabbix settings backup
+
+## File connection
+
+Running any of the commands below
+
+* borg.backup.file.sh
+* borg.backup.gitea.sh
+* borg.backup.mysql.sh
+* borg.backup.pgsql.sh
+* borg.backup.zabbix.sh
+
+1) Run `borg.backup.functions.init.sh` (always)
+2) Run `borg.backup.functions.verify.sh` (always)
+3) (other code in "file" module)
+4) Run `borg.backup.functions.info.sh` (always)
+5) Run `borg.backup.functions.compact.sh` (not if one time tag)
+6) Run `borg.backup.functions.check.sh` (not if one time tag)
+7) Run `borg.backup.functions.close.sh` (always, can be called on error too)
--- a/_borg_backup_set_prefix_cleanup.sh
+++ b/_borg_backup_set_prefix_cleanup.sh
@@ -8,14 +8,20 @@
 # pgsql
 # zabbix-settings-

-export BORG_BASE_DIR="borg/";
+# debug and dry run
 DEBUG=0;
 DRYRUN=0;
+PGSQL_SUDO=1;
+# options
+OPT_REMOTE="";
+PGSQL_SUDO_USER="postgres";
+# basic settings needed
 TARGET_USER="";
 TARGET_HOST="";
 TARGET_PORT="";
 TARGET_BORG_PATH="";
 TARGET_FOLDER="";
+# base folder
 BASE_FOLDER="/usr/local/scripts/borg/";
 # those are the valid modules
 MODULE_LIST="file gitea mysql pgsql zabbix"
@@ -23,15 +29,18 @@ MODULE_LIST="file gitea mysql pgsql zabbix"
 # basic options
 # -c for config file override
 # -n for dry run test
-while getopts ":c:nd" opt; do
+while getopts ":c:snd" opt; do
 	case "${opt}" in
-		c|config)
-			BASE_FOLDER=${OPTARG}"/";
+		c) # config
+			BASE_FOLDER=${OPTARG};
 			;;
-		d|debug)
+		s) # nosudo
+			PGSQL_SUDO=0;
+			;;
+		d) # debug
 			DEBUG=1;
 			;;
-		n|dryrun)
+		n) # dryrun
 			DRYRUN=1;
 			;;
 		:)
@@ -55,7 +64,16 @@ if [ ! -f "${BASE_FOLDER}${SETTINGS_FILE}" ]; then
 	echo "Could not find: ${BASE_FOLDER}${SETTINGS_FILE}";
 	exit;
 fi;
+# shellcheck source="borg.backup.settings"
 . "${BASE_FOLDER}${SETTINGS_FILE}";
+
+if [ -n "${TARGET_BORG_PATH}" ]; then
+	OPT_REMOTE="--remote-path="$(printf "%q" "${TARGET_BORG_PATH}");
+fi;
+export BORG_BASE_DIR="${BASE_FOLDER}";
+export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes";
+export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes";
+
 ORIG_BACKUPFILE=${BACKUP_FILE};
 for MODULE in ${MODULE_LIST}; do
 	echo "************* MODULE: ${MODULE}";
@@ -65,37 +83,57 @@ for MODULE in ${MODULE_LIST}; do
 	TARGET_FOLDER=${TARGET_FOLDER#/}
 	# and add slash front and back and escape the path
 	TARGET_FOLDER=$(printf "%q" "/${TARGET_FOLDER}/");
-	if [ ! -z "${TARGET_USER}" ] && [ ! -z "${TARGET_HOST}" ] && [ ! -z "${TARGET_PORT}" ]; then
+	if [ -n "${TARGET_USER}" ] && [ -n "${TARGET_HOST}" ] && [ -n "${TARGET_PORT}" ]; then
 		TARGET_SERVER="ssh://${TARGET_USER}@${TARGET_HOST}:${TARGET_PORT}/";
 	# host/port
-	elif [ ! -z "${TARGET_HOST}" ] && [ ! -z "${TARGET_PORT}" ]; then
+	elif [ -n "${TARGET_HOST}" ] && [ -n "${TARGET_PORT}" ]; then
 		TARGET_SERVER="ssh://${TARGET_HOST}:${TARGET_PORT}/";
 	# user/host
-	elif [ ! -z "${TARGET_USER}" ] && [ ! -z "${TARGET_HOST}" ]; then
+	elif [ -n "${TARGET_USER}" ] && [ -n "${TARGET_HOST}" ]; then
 		TARGET_SERVER="${TARGET_USER}@${TARGET_HOST}:";
 	# host
-	elif [ ! -z "${TARGET_HOST}" ]; then
+	elif [ -n "${TARGET_HOST}" ]; then
 		TARGET_SERVER="${TARGET_HOST}:";
 	fi;
 	# we dont allow special characters, so we don't need to special escape it
 	REPOSITORY="${TARGET_SERVER}${TARGET_FOLDER}${BACKUP_FILE}";
+	# set sudo prefix for postgres so the cache folder stays the same
+	# if run as root then the foloders below have to have the user set to postgres again
+	# .config/borg/security/<postgresql repo id>
+	# .cache/borg/<postgresql repo id>
+	CMD_PREFIX="";
+	# only sudo to pgsql if sudo is set to true
+	if [ "${MODULE}" = "pgsql" ] && [ "${PGSQL_SUDO}" = "1" ]; then
+		CMD_PREFIX="sudo -E -u ${PGSQL_SUDO_USER} ";
+	fi;
 	echo "==== REPOSITORY: ${REPOSITORY}";
-	borg list --format '{archive}{NL}' ${REPOSITORY}|grep -v "${MODULE},"|
+	borg list ${OPT_REMOTE} --format '{archive}{NL}' ${REPOSITORY}|grep -v "${MODULE},"|
 	while read i; do
 		# for gitea, zabbix we do not ADD we RENAME
-		if [ "${MODILE}" = "gitea" ]; then
-			target_name=$(echo $i | sed -e 's/gitea-/gitea,/');
-		elif [ "${MODILE}" = "zabbix" ]; then
-			target_name=$(echo $i | sed -e 's/zabbix-settings-/zabbix,settings-/');
+		if [ "${MODULE}" = "gitea" ]; then
+			# if just date, add gitea,
+			# else rename
+			if [ -n "${i##gitea*}" ]; then
+				target_name="${MODULE},${i}";
+			else
+				target_name=$(echo $i | sed -e 's/gitea-/gitea,/');
+			fi;
+		elif [ "${MODULE}" = "zabbix" ]; then
+			# if zabbix is missing, prefix
+			if [ -n "${i##zabbix*}" ]; then
+				target_name="${MODULE},${i}";
+			else
+				target_name=$(echo $i | sed -e 's/zabbix-settings-/zabbix,settings-/');
+			fi;
 		else
 			target_name="${MODULE},${i}";
 		fi;
 		echo "- Rename from: ${i} to: ${target_name}";
 		if [ ${DEBUG} -eq 1 ]; then
-			echo "borg rename -p -v ${REPOSITORY}::${i} ${target_name}";
+			echo "${CMD_PREFIX}borg rename ${OPT_REMOTE} -p -v ${REPOSITORY}::${i} ${target_name}";
 		fi;
 		if [ ${DRYRUN} -eq 0 ]; then
-			borg rename -p -v ${REPOSITORY}::${i} ${target_name};
+			${CMD_PREFIX}borg rename ${OPT_REMOTE} -p -v ${REPOSITORY}::${i} ${target_name};
 		fi;
 	done;
 done;
--- a/borg.backup.file.sh
+++ b/borg.backup.file.sh
@@ -1,23 +1,30 @@
 #!/usr/bin/env bash

-# Run -I first to initialize repository
-# There are no automatic repository checks unless -C is given
+# allow variables in printf format string
+# shellcheck disable=SC2059
+
+# Plain file backup

 # set last edit date + time
 MODULE="file";
-MODULE_VERSION="1.2.0";
+MODULE_VERSION="1.2.3";

 DIR="${BASH_SOURCE%/*}"
 if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 . "${DIR}/borg.backup.functions.init.sh";

 # include and exclude file
-INCLUDE_FILE="borg.backup.file.include";
-EXCLUDE_FILE="borg.backup.file.exclude";
-# init check file
-BACKUP_INIT_CHECK="borg.backup.file.init";
+INCLUDE_FILE="borg.backup.${MODULE}.include";
+EXCLUDE_FILE="borg.backup.${MODULE}.exclude";
+# init verify, compact and check file
+BACKUP_INIT_FILE="borg.backup.${MODULE}.init";
+BACKUP_COMPACT_FILE="borg.backup.${MODULE}.compact";
+BACKUP_CHECK_FILE="borg.backup.${MODULE}.check";
+# lock file
+BACKUP_LOCK_FILE="borg.backup.${MODULE}.lock";

-. "${DIR}/borg.backup.functions.check.sh";
+# verify valid data
+. "${DIR}/borg.backup.functions.verify.sh";

 # exit if include file is missing
 if [ ! -f "${BASE_FOLDER}${INCLUDE_FILE}" ]; then
@@ -25,13 +32,13 @@ if [ ! -f "${BASE_FOLDER}${INCLUDE_FILE}" ]; then
 	. "${DIR}/borg.backup.functions.close.sh" 1;
 	exit 1;
 fi;
-echo "--- [INCLUDE: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+printf "${PRINTF_SUB_BLOCK}" "INCLUDE" "$(date +'%F %T')" "${MODULE}";
 # folders to backup
 FOLDERS=();
 # this if for debug output with quoted folders
 FOLDERS_Q=();
 # include list
-while read include_folder; do
+while read -r include_folder; do
 	# strip any leading spaces from that folder
 	include_folder=$(echo "${include_folder}" | sed -e 's/^[ \t]*//');
 	# check that those folders exist, warn on error,
@@ -41,7 +48,7 @@ while read include_folder; do
 		echo "# [C] Comment: '${include_folder}'";
 	else
 		# skip if it is empty
-		if [ ! -z "${include_folder}" ]; then
+		if [ -n "${include_folder}" ]; then
 			# if this is a glob, do a double check that the base folder actually exists (?)
 			if [[ "${include_folder}" =~ $REGEX_GLOB ]]; then
 				# if this is */ then allow it
@@ -88,14 +95,14 @@ while read include_folder; do
 	fi;
 done<"${BASE_FOLDER}${INCLUDE_FILE}";

-# exclude list
-if [ -f "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
-	echo "--- [EXCLUDE: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+# exclude list, only if file exists and is larger than zero
+if [ -s "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
+	printf "${PRINTF_SUB_BLOCK}" "EXCLUDE" "$(date +'%F %T')" "${MODULE}";
 	# check that the folders in that exclude file are actually valid,
 	# remove non valid ones and warn
 	#TMP_EXCLUDE_FILE=$(mktemp --tmpdir ${EXCLUDE_FILE}.XXXXXXXX); # non mac
 	TMP_EXCLUDE_FILE=$(mktemp "${TEMPDIR}${EXCLUDE_FILE}".XXXXXXXX);
-	while read exclude_folder; do
+	while read -r exclude_folder; do
 		# strip any leading spaces from that folder
 		exclude_folder=$(echo "${exclude_folder}" | sed -e 's/^[ \t]*//');
 		# folder or any type of file is ok
@@ -104,10 +111,10 @@ if [ -f "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
 			echo "# [C] Comment: '${exclude_folder}'";
 		else
 			# skip if it is empty
-			if [ ! -z "${exclude_folder}" ]; then
+			if [ -n "${exclude_folder}" ]; then
 				# if it DOES NOT start with a / we assume free folder and add as is
 				if [[ "${exclude_folder}" != /* ]]; then
-					echo "${exclude_folder}" >> ${TMP_EXCLUDE_FILE};
+					echo "${exclude_folder}" >> "${TMP_EXCLUDE_FILE}";
 					echo "+ [E] General exclude: '${exclude_folder}'";
 				# if this is a glob, do a double check that the base folder actually exists (?)
 				elif [[ "${exclude_folder}" =~ $REGEX_GLOB ]]; then
@@ -117,7 +124,7 @@ if [ -f "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
 					if [ ! -d "${_exclude_folder}" ]; then
 						echo "- [E] Exclude folder with glob '${exclude_folder}' does not exist or is not accessable";
 					else
-						echo "${exclude_folder}" >> ${TMP_EXCLUDE_FILE};
+						echo "${exclude_folder}" >> "${TMP_EXCLUDE_FILE}";
 						echo "+ [E] Exclude folder with glob '${exclude_folder}'";
 					fi;
 				# do a warning for a possible invalid folder
@@ -125,7 +132,7 @@ if [ -f "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
 				elif [ ! -d "${exclude_folder}" ] && [ ! -e "${exclude_folder}" ]; then
 					echo "- [E] Exclude folder or file '${exclude_folder}' does not exist or is not accessable";
 				else
-					echo "${exclude_folder}" >> ${TMP_EXCLUDE_FILE};
+					echo "${exclude_folder}" >> "${TMP_EXCLUDE_FILE}";
 					# if it is a folder, remove the last / or the symlink check will not work
 					if [ -d "${exclude_folder}" ]; then
 						_exclude_folder=${exclude_folder%/*};
@@ -143,7 +150,7 @@ if [ -f "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
 		fi;
 	done<"${BASE_FOLDER}${EXCLUDE_FILE}";
 	# avoid blank file add by checking if the tmp file has a size >0
-	if [ -s "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
+	if [ -s "${TMP_EXCLUDE_FILE}" ]; then
 		OPT_EXCLUDE="--exclude-from=${TMP_EXCLUDE_FILE}";
 	fi;
 fi;
@@ -159,7 +166,7 @@ COMMAND=${COMMAND}${REPOSITORY}::${ONE_TIME_TAG}${BACKUP_SET_PREFIX}${BACKUP_SET
 . "${DIR}/borg.backup.functions.info.sh";

 if [ $FOLDER_OK -eq 1 ]; then
-	echo "--- [BACKUP: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+	printf "${PRINTF_SUB_BLOCK}" "BACKUP" "$(date +'%F %T')" "${MODULE}";
 	# show command
 	if [ ${DEBUG} -eq 1 ]; then
 		echo $(echo ${COMMAND} | sed -e 's/[ ][ ]*/ /g') ${FOLDERS_Q[*]};
@@ -182,7 +189,7 @@ fi;

 # clean up, always verbose, but only if we do not run one time tag
 if [ -z "${ONE_TIME_TAG}" ]; then
-	echo "--- [PRUNE : $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+	printf "${PRINTF_SUB_BLOCK}" "PRUNE" "$(date +'%F %T')" "${MODULE}";
 	# build command
 	COMMAND="${BORG_COMMAND} prune ${OPT_REMOTE} -v --list ${OPT_PROGRESS} ${DRY_RUN_STATS} -P ${BACKUP_SET_PREFIX} ${KEEP_OPTIONS[*]} ${REPOSITORY}";
 	echo "Prune repository with keep${KEEP_INFO:1}";
@@ -192,7 +199,9 @@ if [ -z "${ONE_TIME_TAG}" ]; then
 	# for the IFS="#" to work we need to replace options spaces with exactly ONE #
 	$(echo "${COMMAND}" | sed -e 's/[ ][ ]*/#/g') 2>&1 || echo "[!] Borg prune aborted";
 	# if this is borg version >1.2 we need to run compact after prune
-	. "${DIR}/borg.backup.functions.compact.sh";
+	. "${DIR}/borg.backup.functions.compact.sh" "auto";
+	# check in auto mode
+	. "${DIR}/borg.backup.functions.check.sh" "auto";
 else
 	echo "[#] No prune with tagged backup";
 fi;
--- a/borg.backup.functions.check.sh
+++ b/borg.backup.functions.check.sh
@@ -1,376 +1,86 @@
 #!/usr/bin/env bash

-# start time in seconds
-START=$(date +'%s');
-# start logging from here
-exec &> >(tee -a "${LOG}");
-echo "=== [START : $(date +'%F %T')] ==[${MODULE}]====================================>";
-# show info for version always
-echo "Script version: ${VERSION}";
-# show type
-echo "Backup module : ${MODULE}";
-echo "Module version: ${MODULE_VERSION}";
-# borg version
-echo "Borg version  : ${BORG_VERSION}";
-# show base folder always
-echo "Base folder   : ${BASE_FOLDER}";
+# allow variables in printf format string
+# shellcheck disable=SC2059

-# if force check is true set CHECK to 1unless INFO is 1
-# Needs bash 4.0 at lesat for this
-if [ "${FORCE_CHECK,,}" = "true" ] && [ ${INFO} -eq 0 ]; then
-	CHECK=1;
-	if [ ${DEBUG} -eq 1 ]; then
-		echo "Force repository check";
-	fi;
-fi;
-
-# remote borg path
-if [ ! -z "${TARGET_BORG_PATH}" ]; then
-	if [[ "${TARGET_BORG_PATH}" =~ \ |\' ]]; then
-		echo "Space found in ${TARGET_BORG_PATH}. Aborting";
-		echo "There are issues with passing on paths with spaces"
-		echo "as parameters"
-		exit;
-	fi;
-	OPT_REMOTE="--remote-path="$(printf "%q" "${TARGET_BORG_PATH}");
-fi;
-
-if [ -z "${TARGET_FOLDER}" ]; then
-	echo "[! $(date +'%F %T')] No target folder has been set yet";
+if [ -z "${MODULE}" ]; then
+	echo "Script cannot be run on its own";
 	exit 1;
-else
-	# There are big issues with TARGET FOLDERS with spaces
-	# we should abort anything with this
-	if [[ "${TARGET_FOLDER}" =~ \ |\' ]]; then
-		echo "Space found in ${TARGET_FOLDER}. Aborting";
-		echo "There is some problem with passing paths with spaces as";
-		echo "repository base folder"
-		exit;
+fi;
+
+# run borg check (NOT REPAIR)
+
+RUN_CHECK=0;
+if [ $# -ge 1 ] && [ "$1" = "auto" ]; then
+	# strip any spaces and convert to int
+	CHECK_INTERVAL=$(echo "${CHECK_INTERVAL}" | sed -e 's/ //g');
+	# not a valid check interval, do no check
+	if [ -z "${CHECK_INTERVAL##*[!0-9]*}" ]; then
+		CHECK_INTERVAL=0;
 	fi;
-
-	# This does not care for multiple trailing or leading slashes
-	# it just makes sure we have at least one set
-	# for if we have a single slash, remove it
-	TARGET_FOLDER=${TARGET_FOLDER%/}
-	TARGET_FOLDER=${TARGET_FOLDER#/}
-	# and add slash front and back and escape the path
-	TARGET_FOLDER=$(printf "%q" "/${TARGET_FOLDER}/");
-fi;
-
-# if we have user/host then we build the ssh command
-TARGET_SERVER='';
-# allow host only (if full setup in .ssh/config)
-# user@host OR ssh://user@host:port/ IF TARGET_PORT is set
-# user/host/port
-if [ ! -z "${TARGET_USER}" ] && [ ! -z "${TARGET_HOST}" ] && [ ! -z "${TARGET_PORT}" ]; then
-	TARGET_SERVER="ssh://${TARGET_USER}@${TARGET_HOST}:${TARGET_PORT}/";
-# host/port
-elif [ ! -z "${TARGET_HOST}" ] && [ ! -z "${TARGET_PORT}" ]; then
-	TARGET_SERVER="ssh://${TARGET_HOST}:${TARGET_PORT}/";
-# user/host
-elif [ ! -z "${TARGET_USER}" ] && [ ! -z "${TARGET_HOST}" ]; then
-	TARGET_SERVER="${TARGET_USER}@${TARGET_HOST}:";
-# host
-elif [ ! -z "${TARGET_HOST}" ]; then
-	TARGET_SERVER="${TARGET_HOST}:";
-fi;
-# we dont allow special characters, so we don't need to special escape it
-REPOSITORY="${TARGET_SERVER}${TARGET_FOLDER}${BACKUP_FILE}";
-echo "Repository    : ${REPOSITORY}";
-
-# check compression if given is valid and check compression level is valid if given
-OPT_COMPRESSION='';
-if [ ! -z "${COMPRESSION}" ]; then
-	# valid compression
-	if [ "${COMPRESSION}" = "lz4" ] || [ "${COMPRESSION}" = "zlib" ] || [ "${COMPRESSION}" = "lzma" ] || [ "${COMPRESSION}" = "zstd" ]; then
-		OPT_COMPRESSION="-C=${COMPRESSION}";
-		# if COMPRESSION_LEVEL, check it is a valid regex
-		# for zlib, zstd, lzma
-		if [ ! -z "${COMPRESSION_LEVEL}" ] && ([ "${COMPRESSION}" = "zlib" ] || [ "${COMPRESSION}" = "lzma" ] || [ "${COMPRESSION}" = "zstd" ]); then
-			MIN_COMPRESSION=0;
-			MAX_COMPRESSION=0;
-			case "${COMPRESSION}" in
-				zlib|lzma)
-					MIN_COMPRESSION=0;
-					MAX_COMPRESSION=9;
-					;;
-				zstd)
-					MIN_COMPRESSION=1;
-					MAX_COMPRESSION=22;
-					;;
-				*)
-					MIN_COMPRESSION=0;
-					MAX_COMPRESSION=0;
-					;;
-			esac;
-			# if [ "${COMPRESSION}" = "zlib" ] || [ "${COMPRESSION}" = "lzma" ]
-			# 	MIN_COMPRESSION=0;
-			# 	MAX_COMPRESSION=9;
-			# elif [ "${COMPRESSION}" = "zstd" ]; then
-			# 	MIN_COMPRESSION=1;
-			# 	MAX_COMPRESSION=22;
-			# fi;
-			error_message="[! $(date +'%F %T')] Compression level for ${COMPRESSION} needs to be a numeric value between ${MIN_COMPRESSION} and ${MAX_COMPRESSION}: ${COMPRESSION_LEVEL}";
-			if ! [[ "${COMPRESSION_LEVEL}" =~ ${REGEX_NUMERIC} ]]; then
-				echo ${error_message};
-				exit 1;
-			elif [ ${COMPRESSION_LEVEL} -lt ${MIN_COMPRESSION} ] || [ ${COMPRESSION_LEVEL} -gt ${MAX_COMPRESSION} ]; then
-				echo ${error_message};
-				exit 1;
-			else
-				OPT_COMPRESSION=${OPT_COMPRESSION}","${COMPRESSION_LEVEL};
-			fi;
+	# get current date timestmap
+	CURRENT_DATE=$(date +%s);
+	# if =1 always ok
+	if [ "${CHECK_INTERVAL}" -eq 1 ]; then
+		RUN_CHECK=1;
+		# always add verify data for automatic check
+		OPT_CHECK_VERIFY_DATA="--verify-data";
+		# set new check time here
+		echo "${CURRENT_DATE}" > "${BASE_FOLDER}${BACKUP_CHECK_FILE}";
+	elif [ "${CHECK_INTERVAL}" -gt 1 ]; then
+		# else load last timestamp and check if today - last time stamp > days
+		if [ -z "${LAST_CHECK_DATE}" ]; then
+			LAST_CHECK_DATE=$(cat "${BASE_FOLDER}${BACKUP_CHECK_FILE}" 2>/dev/null | sed -e 's/ //g');
 		fi;
-	else
-		echo "[! $(date +'%F %T')] Compress setting need to be lz4, zstd, zlib or lzma. Or empty for no compression: ${COMPRESSION}";
-		exit 1;
-	fi;
-fi;
-
-# home folder, needs to be set if there is eg a HOME=/ in the crontab
-if [ ! -w "${HOME}" ] || [ "${HOME}" = '/' ]; then
-	HOME=$(eval echo "$(whoami)");
-fi;
-
-# keep optionfs (for files)
-KEEP_OPTIONS=();
-# keep info string (for files)
-KEEP_INFO="";
-# override standard keep for tagged backups
-if [ ! -z "${ONE_TIME_TAG}" ]; then
-	BACKUP_SET="{now:%Y-%m-%dT%H:%M:%S}";
-else
-	# build options and info string,
-	# also flag BACKUP_SET check if hourly is set
-	BACKUP_SET_CHECK=0;
-	if [ ${KEEP_LAST} -gt 0 ]; then
-		KEEP_OPTIONS+=("--keep-last=${KEEP_LAST}");
-		KEEP_INFO="${KEEP_INFO}, last: ${KEEP_LAST}";
-	fi;
-	if [ ${KEEP_HOURS} -gt 0 ]; then
-		KEEP_OPTIONS+=("--keep-hourly=${KEEP_HOURS}");
-		KEEP_INFO="${KEEP_INFO}, hourly: ${KEEP_HOURS}";
-		BACKUP_SET_CHECK=1;
-	fi;
-	if [ ${KEEP_DAYS} -gt 0 ]; then
-		KEEP_OPTIONS+=("--keep-daily=${KEEP_DAYS}");
-		KEEP_INFO="${KEEP_INFO}, daily: ${KEEP_DAYS}";
-	fi;
-	if [ ${KEEP_WEEKS} -gt 0 ]; then
-		KEEP_OPTIONS+=("--keep-weekly=${KEEP_WEEKS}");
-		KEEP_INFO="${KEEP_INFO}, weekly: ${KEEP_WEEKS}";
-	fi;
-	if [ ${KEEP_MONTHS} -gt 0 ]; then
-		KEEP_OPTIONS+=("--keep-monthly=${KEEP_MONTHS}");
-		KEEP_INFO="${KEEP_INFO}, monthly: ${KEEP_MONTHS}";
-	fi;
-	if [ ${KEEP_YEARS} -gt 0 ]; then
-		KEEP_OPTIONS+=("--keep-yearly=${KEEP_YEARS}");
-		KEEP_INFO="${KEEP_INFO}, yearly: ${KEEP_YEARS}";
-	fi;
-	if [ ! -z "${KEEP_WITHIN}" ]; then
-		# check for invalid string. can only be number + H|d|w|m|y
-		if [[ "${KEEP_WITHIN}" =~ ^[0-9]+[Hdwmy]{1}$ ]]; then
-			KEEP_OPTIONS+=("--keep-within=${KEEP_WITHIN}");
-			KEEP_INFO="${KEEP_INFO}, within: ${KEEP_WITHIN}";
-			if [[ "${KEEP_WITHIN}" == *"H"* ]]; then
-				BACKUP_SET_CHECK=1;
-			fi;
-		else
-			echo "[! $(date +'%F %T')] KEEP_WITHIN has invalid string.";
-			exit 1;
+		# file date is not a timestamp
+		if [ -z "${LAST_CHECK_DATE##*[!0-9]*}" ]; then
+			LAST_CHECK_DATE=0;
+		fi;
+		# if the difference greate than check date, run. CHECK INTERVAL is in days
+		if [ $((CURRENT_DATE - LAST_CHECK_DATE)) -ge $((CHECK_INTERVAL * 86400)) ]; then
+			RUN_CHECK=1;
+			# always add verify data for automatic check
+			OPT_CHECK_VERIFY_DATA="--verify-data";
+			# set new check time here
+			echo "${CURRENT_DATE}" > "${BASE_FOLDER}${BACKUP_CHECK_FILE}";
 		fi;
 	fi;
-	# abort if KEEP_OPTIONS is empty
-	if [ -z "${KEEP_OPTIONS}" ]; then
-		echo "[! $(date +'%F %T')] It seems no KEEP_* entries where set in a valid format.";
-		exit 1;
-	fi;
-	# set BACKUP_SET if empty, set to Year-month-day
-	if [ -z "${BACKUP_SET}" ]; then
-		BACKUP_SET="{now:%Y-%m-%d}";
-	fi;
-	# backup set check, and there is no hour entry (%H) in the archive string
-	# we add T%H:%M:%S in this case, before the last }
-	if [ ${BACKUP_SET_CHECK} -eq 1 ] && [[ "${BACKUP_SET}" != *"%H"* ]]; then
-		BACKUP_SET=$(echo "${BACKUP_SET}" | sed -e "s/}/T%H:%M:%S}/");
-	fi;
+elif [ "${CHECK}" -eq 1 ]; then
+	RUN_CHECK=1;
 fi;

-
-# for folders list split set to "#" and keep the old setting as is
-_IFS=${IFS};
-IFS="#";
-# turn off for non file
-if [ "${MODULE}" != "file" ]; then
+if [ "${RUN_CHECK}" -eq 1 ]; then
+	# run borg check command
 	IFS=${_IFS};
-fi;
-
-# borg call, replace ##...## parts during run
-# used in all modules, except 'file'
-_BORG_CALL="${BORG_COMMAND} create ${OPT_REMOTE} -v ${OPT_LIST} ${OPT_PROGRESS} ${OPT_COMPRESSION} -s --stdin-name ##FILENAME## ${REPOSITORY}::##BACKUP_SET## -";
-_BORG_PRUNE="${BORG_COMMAND} prune ${OPT_REMOTE} -v --list ${OPT_PROGRESS} ${DRY_RUN_STATS} -P ##BACKUP_SET_PREFIX## ${KEEP_OPTIONS[*]} ${REPOSITORY}";
-
-# general borg settings
-# set base path to config directory to keep cache/config separated
-export BORG_BASE_DIR="${BASE_FOLDER}";
-# ignore non encrypted access
-export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=${_BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK};
-# ignore moved repo access
-export BORG_RELOCATED_REPO_ACCESS_IS_OK=${_BORG_RELOCATED_REPO_ACCESS_IS_OK};
-# and for debug print that tout
-if [ ${DEBUG} -eq 1 ]; then
-	echo "export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=${_BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK};";
-	echo "export BORG_RELOCATED_REPO_ACCESS_IS_OK=${_BORG_RELOCATED_REPO_ACCESS_IS_OK};";
-	echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";";
-fi;
-# prepare debug commands only
-COMMAND_EXPORT="export BORG_BASE_DIR=\"${BASE_FOLDER}\";"
-COMMAND_INFO="${COMMAND_EXPORT}${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY}";
-# if the is not there, call init to create it
-# if this is user@host, we need to use ssh command to check if the file is there
-# else a normal check is ok
-# unless explicit given, check is skipped
-if [ ${CHECK} -eq 1 ] || [ ${INIT} -eq 1 ]; then
-	echo "--- [CHECK : $(date +'%F %T')] --[${MODULE}]------------------------------------>";
-	if [ ! -z "${TARGET_SERVER}" ]; then
-		if [ ${DEBUG} -eq 1 ]; then
-			echo "${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY} 2>&1|grep \"Repository ID:\"";
-		fi;
-		# use borg info and check if it returns "Repository ID:" in the first line
-		REPO_CHECK=$(${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY} 2>&1|grep "Repository ID:");
-		# this is currently a hack to work round the error code in borg info
-		# this checks if REPO_CHECK holds this error message and then starts init
-		if [[ -z "${REPO_CHECK}" ]] || [[ "${REPO_CHECK}" =~ ${REGEX_ERROR} ]]; then
-			INIT_REPOSITORY=1;
-		fi;
-	elif [ ! -d "${REPOSITORY}" ]; then
-		INIT_REPOSITORY=1;
+	printf "${PRINTF_SUB_BLOCK}" "CHECK" "$(date +'%F %T')" "${MODULE}";
+	# repare command
+	OPT_GLOB="";
+	if [[ "${CHECK_PREFIX}" =~ $REGEX_GLOB ]]; then
+		OPT_GLOB="-a '${CHECK_PREFIX}'"
+	elif [ -n "${CHECK_PREFIX}" ]; then
+		OPT_GLOB="-P ${CHECK_PREFIX}";
 	fi;
-	# if check but no init and repo is there but init file is missing set it
-	if [ ${CHECK} -eq 1 ] && [ ${INIT} -eq 0 ] && [ ${INIT_REPOSITORY} -eq 0 ] &&
-		[ ! -f "${BASE_FOLDER}${BACKUP_INIT_CHECK}" ]; then
-		# write init file
-		echo "[!] Add missing init check file";
-		echo "$(date +%s)" > "${BASE_FOLDER}${BACKUP_INIT_CHECK}";
-	fi;
-	# end if checked but repository is not here
-	if [ ${CHECK} -eq 1 ] && [ ${INIT} -eq 0 ] && [ ${INIT_REPOSITORY} -eq 1 ]; then
-		echo "[! $(date +'%F %T')] No repository. Please run with -I flag to initialze repository";
-		. "${DIR}/borg.backup.functions.close.sh" 1;
-		exit 1;
-	fi;
-	if [ ${EXIT} -eq 1 ] && [ ${CHECK} -eq 1 ] && [ ${INIT} -eq 0 ]; then
-		echo "Repository exists";
-		echo "For more information run:"
-		echo "${COMMAND_INFO}";
-		. "${DIR}/borg.backup.functions.close.sh";
-		exit;
-	fi;
-fi;
-if [ ${INIT} -eq 1 ] && [ ${INIT_REPOSITORY} -eq 1 ]; then
-	echo "--- [INIT  : $(date +'%F %T')] --[${MODULE}]------------------------------------>";
-	if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
-		echo "${BORG_COMMAND} init ${OPT_REMOTE} -e ${ENCRYPTION} ${OPT_VERBOSE} ${REPOSITORY}";
-	fi
-	if [ ${DRYRUN} -eq 0 ]; then
-		# should trap and exit properly here
-		${BORG_COMMAND} init ${OPT_REMOTE} -e ${ENCRYPTION} ${OPT_VERBOSE} ${REPOSITORY};
-		# write init file
-		echo "$(date +%s)" > "${BASE_FOLDER}${BACKUP_INIT_CHECK}";
-		echo "Repository initialized";
-		echo "For more information run:"
-		echo "${COMMAND_INFO}";
-	fi
-	. "${DIR}/borg.backup.functions.close.sh";
-	# exit after init
-	exit;
-elif [ ${INIT} -eq 1 ] && [ ${INIT_REPOSITORY} -eq 0 ]; then
-	echo "[! $(date +'%F %T')] Repository already initialized";
-	echo "For more information run:"
-	echo "${COMMAND_INFO}";
-	. "${DIR}/borg.backup.functions.close.sh" 1;
-	exit 1;
-fi;
-
-# check for init file
-if [ ! -f "${BASE_FOLDER}${BACKUP_INIT_CHECK}" ]; then
-	echo "[! $(date +'%F %T')] It seems the repository has never been initialized."
-	echo "Please run -I to initialize or if already initialzed run with -C for init update."
-	. "${DIR}/borg.backup.functions.close.sh" 1;
-	exit 1;
-fi;
-
-# PRINT OUT current data, only do this if REPO exists
-if [ ${PRINT} -eq 1 ]; then
-	echo "--- [PRINT : $(date +'%F %T')] --[${MODULE}]------------------------------------>";
-	FORMAT="{archive} {comment:6} {start} - {end} [{id}] ({username}@{hostname}){NL}"
-	# show command on debug or dry run
-	if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
-		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} list ${OPT_REMOTE} --format ${FORMAT} ${REPOSITORY}";
+	# debug/dryrun
+	if [ "${DEBUG}" -eq 1 ] || [ "${DRYRUN}" -eq 1 ]; then
+		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} check ${OPT_REMOTE} ${OPT_PROGRESS} ${OPT_CHECK_VERIFY_DATA} ${OPT_GLOB} ${REPOSITORY}";
 	fi;
 	# run info command if not a dry drun
-	if [ ${DRYRUN} -eq 0 ]; then
-		borg list ${OPT_REMOTE} --format "${FORMAT}" ${REPOSITORY} ;
+	if [ "${DRYRUN}" -eq 0 ]; then
+		# if glob add glob command directly
+		if [[ "${CHECK_PREFIX}" =~ $REGEX_GLOB ]]; then
+		${BORG_COMMAND} check ${OPT_REMOTE} ${OPT_PROGRESS} ${OPT_CHECK_VERIFY_DATA} -a "${CHECK_PREFIX}" "${REPOSITORY}";
+		else
+			${BORG_COMMAND} check ${OPT_REMOTE} ${OPT_PROGRESS} ${OPT_CHECK_VERIFY_DATA} ${OPT_GLOB} "${REPOSITORY}";
+		fi;
 	fi;
-	if [ ${VERBOSE} -eq 1 ]; then
+	# print additional info for use --repair command
+	# but only for manual checks
+	if [ "${VERBOSE}" -eq 1 ] && [ "${CHECK}" -eq 1 ]; then
 		echo "";
-		echo "Base command info:"
-		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} [COMMAND] ${OPT_REMOTE} ${REPOSITORY}::[BACKUP] [PATH]";
-		echo "Replace [COMMAND] with list for listing or extract for restoring backup data."
-		echo "Replace [BACKUP] with archive name."
-		echo "If no [PATH] is given then all files will be restored."
-		echo "Before extracting -n (dry run) is recommended to use."
-		echo "If archive size is needed the info command with archive name has to be used."
-		echo "When listing files in an archive set (::SET) the --format command can be used."
-		echo "Example: \"{mode} {user:6} {group:6} {size:8d} {csize:8d} {dsize:8d} {dcsize:8d} {mtime} {path}{extra} [{health}]{NL}\""
-	else
-		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} [COMMAND] ${OPT_REMOTE} [FORMAT] ${REPOSITORY}::[BACKUP] [PATH]";
+		echo "In case of needed repair: "
+		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} check ${OPT_REMOTE} ${OPT_PROGRESS} --repair ${OPT_GLOB} ${REPOSITORY}";
+		echo "Before running repair, a copy from the backup should be made because repair might damage a backup"
 	fi;
-	. "${DIR}/borg.backup.functions.close.sh";
-	exit;
-fi;
-
-# DELETE ONE TIME TAG
-if [ ! -z "${DELETE_ONE_TIME_TAG}" ]; then
-	echo "--- [DELETE: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
-	# if a "*" is inside we don't do ONE archive, but globbing via -a option
-	DELETE_ARCHIVE=""
-	OPT_GLOB="";
-	# this is more or less for debug only
-	if [[ "${DELETE_ONE_TIME_TAG}" =~ $REGEX_GLOB ]]; then
-		OPT_GLOB="-a '${DELETE_ONE_TIME_TAG}'"
-	else
-		DELETE_ARCHIVE="::"${DELETE_ONE_TIME_TAG};
-	fi
-	# if this is borg <1.2 OPT_LIST does not work
-	if [ $(version $BORG_VERSION) -lt $(version "1.2.0") ]; then
-		OPT_LIST="";
-	fi;
-	# if exists, delete and exit
-	# show command on debug or dry run
-	if [ ${DEBUG} -eq 1 ]; then
-		echo "${BORG_COMMAND} delete ${OPT_REMOTE} ${OPT_LIST} -s ${OPT_GLOB} ${REPOSITORY}${DELETE_ARCHIVE}";
-	fi;
-	# run delete command if not a dry drun
-	# NOTE seems to be glob is not working if wrapped into another variable
-	if [[ "${DELETE_ONE_TIME_TAG}" =~ $REGEX_GLOB ]]; then
-		${BORG_COMMAND} delete ${OPT_REMOTE} ${OPT_LIST} ${DRY_RUN_STATS} -a "${DELETE_ONE_TIME_TAG}" ${REPOSITORY};
-	else
-		${BORG_COMMAND} delete ${OPT_REMOTE} ${OPT_LIST} ${DRY_RUN_STATS} ${REPOSITORY}${DELETE_ARCHIVE};
-	fi;
-	# if not a dry run, compact repository after delete
-	# not that compact only works on borg 1.2
-	if [ $(version $BORG_VERSION) -ge $(version "1.2.0") ]; then
-		if [ ${DRYRUN} -eq 0 ]; then
-			${BORG_COMMAND} compact ${REPOSITORY};
-		fi;
-		if [ ${DEBUG} -eq 1 ]; then
-			echo "${BORG_COMMAND} compact ${REPOSITORY}";
-		fi;
-	fi;
-	. "${DIR}/borg.backup.functions.close.sh";
-	exit;
 fi;

 # __END__
--- a/borg.backup.functions.close.sh
+++ b/borg.backup.functions.close.sh
@@ -1,14 +1,27 @@
 #!/usr/bin/env bash

+# allow variables in printf format string
+# shellcheck disable=SC2059
+
+if [ -z "${MODULE}" ]; then
+	echo "Script cannot be run on its own";
+	exit 1;
+fi;
+
 # unset borg settings
 unset BORG_BASE_DIR BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK BORG_RELOCATED_REPO_ACCESS_IS_OK
+# delete lock file if it exists
+if [ -f "${BASE_FOLDER}${BACKUP_LOCK_FILE}" ]; then
+	rm "${BASE_FOLDER}${BACKUP_LOCK_FILE}";
+fi;
 # error abort without duration and error notice
-if [ $1 == 1 ]; then
-	echo "=== [ERROR: $(date +'%F %T')] ==[${MODULE}]====================================>";
+if [ $# -ge 1 ] && [ "$1" = "1" ]; then
+	printf "${PRINTF_MASTER_BLOCK}" "ERROR" "$(date +'%F %T')" "${MODULE}";
 else
-	DURATION=$[ $(date +'%s')-$START ];
+	# running time calculation
+	DURATION=$(( $(date +'%s') - START ));
 	echo "=== [Run time: $(convert_time ${DURATION})]";
-	echo "=== [END  : $(date +'%F %T')] ==[${MODULE}]====================================>";
+	printf "${PRINTF_MASTER_BLOCK}" "END" "$(date +'%F %T')" "${MODULE}";
 fi;

 # __END__
--- a/borg.backup.functions.compact.sh
+++ b/borg.backup.functions.compact.sh
@@ -1,18 +1,66 @@
 #!/usr/bin/env bash

+# allow variables in printf format string
+# shellcheck disable=SC2059
+
+if [ -z "${MODULE}" ]; then
+	echo "Script cannot be run on its own";
+	exit 1;
+fi;
+
 # compact (only if BORG COMPACT is set)
 # only for borg 1.2
-
-# reset to normal IFS, so command works here
-IFS=${_IFS};
-if [ $(version $BORG_VERSION) -ge $(version "1.2.0") ]; then
-	echo "--- [COMPACT:$(date +'%F %T')] --[${MODULE}]------------------------------------>";
-	BORG_COMPACT="${BORG_COMMAND} compact -v ${OPT_PROGRESS} ${REPOSITORY}";
-	if [ ${DEBUG} -eq 1 ]; then
-			echo "${BORG_COMPACT}";
+if [ "$(version "$BORG_VERSION")" -ge "$(version "1.2.0")" ]; then
+if [ "$(version "$BORG_VERSION")" -ge "$(version "1.2.0")" ]; then
+	RUN_COMPACT=0;
+	if [ $# -ge 1 ] && [ "$1" = "auto" ]; then
+		# strip any spaces and convert to int
+		COMPACT_INTERVAL=$(echo "${COMPACT_INTERVAL}" | sed -e 's/ //g');
+		# not a valid compact interval, do no compact
+		if [ -z "${COMPACT_INTERVAL##*[!0-9]*}" ]; then
+			COMPACT_INTERVAL=0;
+		fi;
+		# get current date timestmap
+		CURRENT_DATE=$(date +%s);
+		if [ "${COMPACT_INTERVAL}" -eq 1 ]; then
+		if [ "${COMPACT_INTERVAL}" -eq 1 ]; then
+			RUN_COMPACT=1;
+			# set new compact time here
+			echo "${CURRENT_DATE}" > "${BASE_FOLDER}${BACKUP_COMPACT_FILE}";
+		elif [ "${COMPACT_INTERVAL}" -gt 1 ]; then
+			echo "${CURRENT_DATE}" > "${BASE_FOLDER}${BACKUP_COMPACT_FILE}";
+		elif [ "${COMPACT_INTERVAL}" -gt 1 ]; then
+			# else load last timestamp and check if today - last time stamp > days
+			if [ -z "${LAST_COMPACT_DATE}" ]; then
+				LAST_COMPACT_DATE=$(cat "${BASE_FOLDER}${BACKUP_COMPACT_FILE}" 2>/dev/null | sed -e 's/ //g');
+			fi;
+			# file date is not a timestamp
+			if [ -z "${LAST_COMPACT_DATE##*[!0-9]*}" ]; then
+				LAST_COMPACT_DATE=0;
+			fi;
+			# if the difference greate than compact date, run. COMPACT INTERVAL is in days
+			if [ $((CURRENT_DATE - LAST_COMPACT_DATE)) -ge $((COMPACT_INTERVAL * 86400)) ]; then
+				RUN_COMPACT=1;
+				# set new compact time here
+				echo "${CURRENT_DATE}" > "${BASE_FOLDER}${BACKUP_COMPACT_FILE}";
+			fi;
+		fi;
+	elif [ "${COMPACT}" -eq 1 ]; then
+		RUN_COMPACT=1;
 	fi;
-	if [ ${DRYRUN} -eq 0 ]; then
-		${BORG_COMPACT};
+
+	if [ "${RUN_COMPACT}" -eq 1 ]; then
+		# reset to normal IFS, so command works here
+		IFS=${_IFS};
+		# shellcheck disable=SC2059
+		printf "${PRINTF_SUB_BLOCK}" "COMPACT" "$(date +'%F %T')" "${MODULE}";
+		BORG_COMPACT="${BORG_COMMAND} compact ${OPT_REMOTE} -v ${OPT_PROGRESS} ${REPOSITORY}";
+		if [ "${DEBUG}" -eq 1 ]; then
+				echo "${BORG_COMPACT}";
+		fi;
+		if [ "${DRYRUN}" -eq 0 ]; then
+			${BORG_COMPACT};
+		fi;
 	fi;
 fi;

--- a/borg.backup.functions.info.sh
+++ b/borg.backup.functions.info.sh
@@ -1,20 +1,33 @@
 #!/usr/bin/env bash

-if [ ${INFO} -eq 1 ]; then
-	echo "--- [INFO  : $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+# allow variables in printf format string
+# shellcheck disable=SC2059
+
+if [ -z "${MODULE}" ]; then
+	echo "Script cannot be run on its own";
+	exit 1;
+fi;
+
+if [ "${INFO}" -eq 1 ]; then
+	printf "${PRINTF_SUB_BLOCK}" "INFO" "$(date +'%F %T')" "${MODULE}";
 	# show command on debug or dry run
-	if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
+	if [ "${DEBUG}" -eq 1 ] || [ "${DRYRUN}" -eq 1 ]; then
 		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY}";
 	fi;
 	# run info command if not a dry drun
-	if [ ${DRYRUN} -eq 0 ]; then
-		${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY};
+	if [ "${DRYRUN}" -eq 0 ]; then
+		${BORG_COMMAND} info ${OPT_REMOTE} "${REPOSITORY}";
+		if [ "${VERBOSE}" -eq 1 ]; then
+			# print key information
+			echo "------------------------------------------------------------------------------";
+			${BORG_COMMAND} key export "${REPOSITORY}";
+		fi;
 	fi;
 	if [ "${MODULE}" = "files" ]; then
-		if [ $FOLDER_OK -eq 1 ]; then
+		if [ "${FOLDER_OK}" -eq 1 ]; then
 			echo "--- [Run command]:";
 			#IFS="#";
-			echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${COMMAND} "${FOLDERS_Q[*]};
+			echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${COMMAND} ${FOLDERS_Q[*]}";
 		else
 			echo "[!] No folders where set for the backup";
 		fi;
--- a/borg.backup.functions.init.sh
+++ b/borg.backup.functions.init.sh
@@ -1,15 +1,28 @@
 #!/usr/bin/env bash

-set -ETu #-e -o pipefail
-trap cleanup SIGINT SIGTERM ERR
+if [ -z "${MODULE}" ]; then
+	echo "Script cannot be run on its own";
+	exit 1;
+fi;

-cleanup() {
+# E: inherit trap ERR
+# T: DEBUG and RETURN traps are inherited
+# u: unset variables ere error
+set -ETu #-e -o pipefail
+trap _catch ERR
+trap _cleanup SIGINT SIGTERM
+
+_cleanup() {
 	# script cleanup here
-	echo "Some part of the script failed with an error: $? @LINE: $(caller)";
+	echo "Script abort: $? @LINE: $(caller)";
 	# unset exported vars
 	unset BORG_BASE_DIR BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK BORG_RELOCATED_REPO_ACCESS_IS_OK;
 	# end trap
-	trap - SIGINT SIGTERM ERR
+	trap - SIGINT SIGTERM
+}
+_catch() {
+	local last_exit_code=$?;
+	echo "Some part of the script failed with ERROR: $last_exit_code @COMMAND: '$BASH_COMMAND' @LINE: $(caller)" >&2;
 }
 # on exit unset any exported var
 trap "unset BORG_BASE_DIR BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK BORG_RELOCATED_REPO_ACCESS_IS_OK" EXIT;
@@ -19,7 +32,7 @@ function version {
 }

 # version for all general files
-VERSION="4.1.2";
+VERSION="4.8.0";

 # borg version and borg comamnd
 BORG_VERSION="";
@@ -30,6 +43,8 @@ _LOG_FOLDER="/var/log/borg.backup/";
 LOG_FOLDER="";
 # should be there on everything
 TEMPDIR="/tmp/";
+# HOSTNAME (as set on server)
+HOSTNAME=$(hostname);
 # creates borg backup based on the include/exclude files
 # if base borg folder (backup files) does not exist, it will automatically init it
 # base folder
@@ -39,18 +54,28 @@ SETTINGS_FILE="borg.backup.settings";
 # include files
 INCLUDE_FILE="";
 EXCLUDE_FILE="";
-# backup folder initialzed check
-BACKUP_INIT_CHECK="";
+# backup folder initialzed verify
+BACKUP_INIT_FILE="";
+BACKUP_INIT_DATE="";
+# file with last compact date
+BACKUP_COMPACT_FILE="";
+# file with last check timestamp
+BACKUP_CHECK_FILE="";
 # one time backup prefix tag, if set will use <tag>.<prefix>-Y-M-DTh:m:s type backup prefix
 ONE_TIME_TAG="";
 DELETE_ONE_TIME_TAG="";
-# debug/verbose
+# check command prefix/glob
+CHECK_PREFIX="";
+# debug/verbose/other flags
 VERBOSE=0;
 LIST=0;
 DEBUG=0;
 DRYRUN=0;
 INFO=0;
+VERIFY=0;
 CHECK=0;
+CHECK_VERIFY_DATA=0;
+COMPACT=0;
 INIT=0;
 EXIT=0;
 PRINT=0;
@@ -68,11 +93,20 @@ REGEX="";
 REGEX_COMMENT="^[\ \t]*#";
 REGEX_GLOB='\*';
 REGEX_NUMERIC="^[0-9]{1,2}$";
-REGEX_ERROR="^Some part of the script failed with an error:";
+# port regex, but only approximately
+REGEX_PORT="^[0-9]{2,5}$";
+REGEX_ERROR="^Some part of the script failed with ERROR:";
 PRUNE_DEBUG="";
 INIT_REPOSITORY=0;
 FOLDER_OK=0;
 TMP_EXCLUDE_FILE="";
+# printf strings
+PRINTF_INFO_STRING="%-23s: %s\n";
+PRINTF_MASTER_BLOCK="=== [%-8s: %19s] ==[%s]====================================>\n";
+PRINTF_SUB_BLOCK="|-- [%-8s: %19s] --[%s]------------------------------------>\n";
+PRINTF_SUBEXT_BLOCK="|-- [%-8s: %s: %19s] --[%s]------------------------------------>\n";
+PRINTF_DB_SUB_BLOCK="|>- [%-8s: %s] ==[%s]=======================>\n";
+PRINTF_DB_RUN_TIME_SUB_BLOCK=">>- [%-8s: %s] ==[%s]==[Run time: %s]=======================>\n";
 # opt flags
 OPT_VERBOSE="";
 OPT_PROGRESS="";
@@ -80,6 +114,7 @@ OPT_LIST="";
 OPT_REMOTE="";
 OPT_LOG_FOLDER="";
 OPT_EXCLUDE="";
+OPT_CHECK_VERIFY_DATA="";
 # config variables (will be overwritten from .settings file)
 TARGET_USER="";
 TARGET_HOST="";
@@ -100,32 +135,53 @@ COMPRESSION_LEVEL="";
 SUB_COMPRESSION="";
 SUB_COMPRESSION_LEVEL="";
 # encryption settings
-DEFAULT_ENCRYPTION="none";
+DEFAULT_ENCRYPTION="keyfile";
 ENCRYPTION="";
-# force check always
-DEFAULT_FORCE_CHECK="false";
-FORCE_CHECK="";
+# force verify always
+DEFAULT_FORCE_VERIFY="false";
+FORCE_VERIFY="";
+FORCE_CHECK=""; # Deprecated name, use FORCE_VERIFY
+# compact
+DEFAULT_COMPACT_INTERVAL="1";
+LAST_COMPACT_DATE="";
+COMPACT_INTERVAL="";
+SUB_COMPACT_INTERVAL="";
+# check
+# default interval is none
+DEFAULT_CHECK_INTERVAL="";
+LAST_CHECK_DATE="";
+CHECK_INTERVAL="";
+SUB_CHECK_INTERVAL="";
+# backup set names
 BACKUP_SET="";
 SUB_BACKUP_SET="";
 # for database backup only
 DATABASE_FULL_DUMP="";
 DATABASE_USER="";
+DATABASE_USE_SUDO="";
+DATABASE_SUDO_USER="";
+DATABASE_PORT="";
+DATABASE_HOST="";
 # only for mysql old config file
 MYSQL_DB_CONFIG="";
 MYSQL_DB_CONFIG_PARAM="";
 # gitea module
 GIT_USER="";
-GITEA_TMP="";
+GITEA_WORKING_DIR="";
+GITEA_TEMP_DIR="";
 GITEA_BIN="";
 GITEA_CONFIG="";
+GITEA_EXPORT_TYPE="";
 # zabbix module
 ZABBIX_DUMP_BIN="";
 ZABBIX_CONFIG="";
 ZABBIX_DATABASE="";
 ZABBIX_UNKNOWN_TABLES="";
+ZABBIX_DB_PORT="";
 OPT_ZABBIX_DUMP="";
 OPT_ZABBIX_CONFIG="";
 OPT_ZABBIX_UNKNOWN_TABLES="";
+OPT_ZABBIX_DB_PORT="";
 # default keep 7 days, 4 weeks, 6 months, 1 year
 # if set 0, ignore/off
 # note that for last/hourly it is needed to create a different
@@ -165,9 +221,13 @@ function usage()
 	-T <tag>: create one time stand alone backup prefixed with tag name
 	-D <tag backup set>: remove a tagged backup set, full name must be given
 	-b <borg executable>: override default path
+	-Z: run compress after prune/backup. Only for borg 1.2 or newer
+	-C: run borg check if repository is ok
+	-y: in combination with -C: add --verify-data
+	-p <archive prefix|glob>: in combinatio with -C: only check archives with prefix or glob
 	-P: print list of archives created
-	-C: check if repository exists, if not abort
-	-E: exit after check
+	-V: verify if repository exists, if not abort
+	-e: exit after verify
 	-I: init repository (must be run first)
 	-i: print out only info
 	-l: list files during backup
@@ -183,7 +243,7 @@ function usage()
 }

 # set options
-while getopts ":c:L:T:D:b:vldniCEIPh" opt; do
+while getopts ":c:L:T:D:b:p:vldniCVeIPyZh" opt; do
 	case "${opt}" in
 		c|config)
 			BASE_FOLDER=${OPTARG};
@@ -200,18 +260,35 @@ while getopts ":c:L:T:D:b:vldniCEIPh" opt; do
 		b|borg)
 			OPT_BORG_EXECUTEABLE=${OPTARG};
 			;;
+		Z|Compact)
+			# will run compact alone
+			COMPACT=1;
+			;;
 		C|Check)
-			# will check if repo is there and abort if not
+			# will run borg check
+			# alt modes --repository-only, --archives-only,
+			# add mode --verify-data
+			# note that --repair has to be called manually is it might damange backups
 			CHECK=1;
 			;;
-		E|Exit)
-			# exit after check
+		y|Verify-Data)
+			CHECK_VERIFY_DATA=1;
+			;;
+		p|prefix-glob)
+			CHECK_PREFIX=${OPTARG};
+			;;
+		V|Verify)
+			# will verify if repo is there and abort if not
+			VERIFY=1;
+			;;
+		e|exit)
+			# exit after verify or init (default off)
 			EXIT=1;
 			;;
 		I|Init)
 			# will check if there is a repo and init it
 			# previoous this was default
-			CHECK=1;
+			VERIFY=1;
 			INIT=1;
 			;;
 		P|Print)
@@ -266,41 +343,57 @@ if [ ! -w "${BASE_FOLDER}" ]; then
 fi;

 # info -i && -C/-I cannot be run together
-if [ ${CHECK} -eq 1 ] || [ ${INIT} -eq 1 ] && [ ${INFO} -eq 1 ]; then
-	echo "Cannot have -i info option and -C check or -I initialize option at the same time";
+if [ ${VERIFY} -eq 1 ] || [ ${INIT} -eq 1 ] && [ ${INFO} -eq 1 ]; then
+	echo "Cannot have -i info option and -V verify or -I initialize option at the same time";
 	exit 1;
 fi;
 # print -P cannot be run with -i/-C/-I together
-if [ ${PRINT} -eq 1 ] && ([ ${INIT} -eq 1 ] || [ ${CHECK} -eq 1 ] || [ ${INFO} -eq 1 ]); then
-	echo "Cannot have -P print option and -i info, -C check or -I initizalize option at the same time";
+if [ ${PRINT} -eq 1 ] && { [ ${INIT} -eq 1 ] || [ ${VERIFY} -eq 1 ] || [ ${INFO} -eq 1 ]; }; then
+	echo "Cannot have -P print option and -i info, -V verify or -I initizalize option at the same time";
 	exit 1;
 fi;
-# if tag is set, you can't have init, check, info, etc
-if [ ! -z "${ONE_TIME_TAG}" ] && ([ ${PRINT} -eq 1 ] || [ ${INIT} -eq 1 ] || [ ${CHECK} -eq 1 ] || [ ${INFO} -eq 1 ]); then
-	echo "Cannot have -T '${ONE_TIME_TAG}' option with -i info, -C check, -I initialize or -P print option at the same time";
+# if tag is set, you can't have init, verify, info, etc
+if [ -n "${ONE_TIME_TAG}" ] && { [ ${PRINT} -eq 1 ] || [ ${INIT} -eq 1 ] || [ ${VERIFY} -eq 1 ] || [ ${INFO} -eq 1 ]; }; then
+	echo "Cannot have -T '${ONE_TIME_TAG}' option with -i info, -V verify, -I initialize or -P print option at the same time";
 	exit 1;
 fi;
-# check only alphanumeric, no spaces, only underscore and dash
-if [ ! -z "${ONE_TIME_TAG}" ] && ! [[ "${ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+$ ]]; then
+# verify only alphanumeric, no spaces, only underscore and dash
+if [ -n "${ONE_TIME_TAG}" ] && ! [[ "${ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+$ ]]; then
 	echo "One time tag '${ONE_TIME_TAG}' must be alphanumeric with dashes and underscore only.";
 	exit 1;
-elif [ ! -z "${ONE_TIME_TAG}" ]; then
-	# all ok, attach . at the end
+elif [ -n "${ONE_TIME_TAG}" ]; then
+	# all ok, attach '.' at the end
 	ONE_TIME_TAG=${ONE_TIME_TAG}".";
 fi;
 # if -D, cannot be with -T, -i, -C, -I, -P
-if [ ! -z "${DELETE_ONE_TIME_TAG}" ] && ([ ! -z "${ONE_TIME_TAG}" ] || [ ${PRINT} -eq 1 ] || [ ${INIT} -eq 1 ] || [ ${CHECK} -eq 1 ] || [ ${INFO} -eq 1 ]); then
-	echo "Cannot have -D delete tag option with -T one time tag, -i info, -C check, -I initialize or -P print option at the same time";
+if [ -n "${DELETE_ONE_TIME_TAG}" ] && { [ -n "${ONE_TIME_TAG}" ] || [ ${PRINT} -eq 1 ] || [ ${INIT} -eq 1 ] || [ ${VERIFY} -eq 1 ] || [ ${INFO} -eq 1 ]; }; then
+	echo "Cannot have -D delete tag option with -T one time tag, -i info, -V verify, -I initialize or -P print option at the same time";
 	exit 1;
 fi;
 # -D also must be in valid backup set format
 # ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},(\*-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T\*$ ]]
-if [ ! -z "${DELETE_ONE_TIME_TAG}" ] && ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},([A-Za-z0-9_-]+-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}$ ]] && ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},(\*-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T\*$ ]]; then
-	echo "Delete one time tag '${DELETE_ONE_TIME_TAG}' is in an invalid format. Please check existing tags with -P option."
+if [ -n "${DELETE_ONE_TIME_TAG}" ] && ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},([A-Za-z0-9_-]+-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}$ ]] && ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},(\*-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T\*$ ]]; then
+	echo "Delete one time tag '${DELETE_ONE_TIME_TAG}' is in an invalid format."
+	echo "Please verify existing tags with -P option."
 	echo "For a globing be sure it is in the format of: TAG.MODULE,*-YYYY-MM-DDT*";
 	echo "Note the dash (-) after the first *, also time (T) is a globa (*) must."
 	exit 1;
 fi;
+# -y can't be set without -C
+if [ ${CHECK_VERIFY_DATA} -eq 1 ] && [ ${CHECK} -eq 0 ]; then
+	echo "-y (verify-data) cannot be run without -C (Check) option";
+	exit 1;
+fi;
+# -p can't be set without -C
+if [ -n "${CHECK_PREFIX}" ] && [ ${CHECK} -eq 0 ]; then
+	echo "-p (pattern|glob) for check cannot be run without -C (Check) options";
+	exit 1;
+fi;
+# can't have -e if VERIFY or INIT is not set
+if [ ${EXIT} -eq 1 ] && [ ${VERIFY} -eq 0 ] && [ ${INIT} -eq 0 ]; then
+	echo "-e (exit) can only be used with -V (Verify) and -I (Init)";
+	exit 1;
+fi;

 # verbose & progress
 if [ ${VERBOSE} -eq 1 ]; then
@@ -317,31 +410,35 @@ if [ ${DRYRUN} -eq 1 ]; then
 else
 	DRY_RUN_STATS="-s";
 fi;
+if [ ${CHECK_VERIFY_DATA} -eq 1 ] && [ ${CHECK} -eq 1 ]; then
+	OPT_CHECK_VERIFY_DATA="--verify-data";
+fi;

 # read config file
 . "${BASE_FOLDER}${SETTINGS_FILE}";

 # if OPTION SET overrides ALL others
-if [ ! -z "${OPT_BORG_EXECUTEABLE}" ]; then
+if [ -n "${OPT_BORG_EXECUTEABLE}" ]; then
 	BORG_COMMAND="${OPT_BORG_EXECUTEABLE}";
 	if [ ! -f "${BORG_COMMAND}" ]; then
 		echo "borg command not found with option -b: ${BORG_COMMAND}";
 		exit;
 	fi;
 # if in setting file, use this
-elif [ ! -z "${BORG_EXECUTEABLE}" ]; then
+elif [ -n "${BORG_EXECUTEABLE}" ]; then
 	BORG_COMMAND="${BORG_EXECUTEABLE}";
 	if [ ! -f "${BORG_COMMAND}" ]; then
 		echo "borg command not found with setting: ${BORG_COMMAND}";
 		exit;
 	fi;
 elif ! command -v borg &> /dev/null; then
-	echo "borg backup seems not to be installed, please check paths";
+# elif [ -z ($command -v borg) ]; then
+	echo "borg backup seems not to be installed, please verify paths";
 	exit;
 fi;
-# check that this is a borg executable, no detail check
-_BORG_COMMAND_CHECK=$(${BORG_COMMAND} -V | grep "borg");
-if [[ "${_BORG_COMMAND_CHECK}" =~ ${REGEX_ERROR} ]]; then
+# verify that this is a borg executable, no detail check
+_BORG_COMMAND_VERIFY=$(${BORG_COMMAND} -V | grep "borg");
+if [[ "${_BORG_COMMAND_VERIFY}" =~ ${REGEX_ERROR} ]]; then
 	echo "Cannot extract borg info from command, is this a valid borg executable?: ${BORG_COMMAND}";
 	exit;
 fi;
@@ -360,8 +457,19 @@ fi;
 if [ -z "${ENCRYPTION}" ]; then
 	ENCRYPTION="${DEFAULT_ENCRYPTION}";
 fi;
-if [ -z "${FORCE_CHECK}" ]; then
-	FORCE_CHECK="${DEFAULT_FORCE_CHECK}";
+# check interval override
+if [ -z "${COMPACT_INTERVAL}" ]; then
+	COMPACT_INTERVAL="${DEFAULT_COMPACT_INTERVAL}";
+fi;
+if [ -z "${CHECK_INTERVAL}" ]; then
+	CHECK_INTERVAL="${DEFAULT_CHECK_INTERVAL}";
+fi;
+# deprecated name FORCE_CHECK, use FORCE_VERIFY instead
+if [ -n "${FORCE_CHECK}" ]; then
+	FORCE_VERIFY="${FORCE_CHECK}";
+fi;
+if [ -z "${FORCE_VERIFY}" ]; then
+	FORCE_VERIFY="${DEFAULT_FORCE_VERIFY}";
 fi;
 if [ -z "${KEEP_LAST}" ]; then
 	KEEP_LAST="${DEFAULT_KEEP_LAST}";
@@ -388,40 +496,48 @@ SETTINGS_FILE_SUB=$(echo "${SETTINGS_FILE}" | sed -e "s/\.settings/\.${MODULE,,}
 if [ -f "${BASE_FOLDER}${SETTINGS_FILE_SUB}" ]; then
 	. "${BASE_FOLDER}${SETTINGS_FILE_SUB}";
 	# if SUB_ set override master
-	if [ ! -z "${SUB_BACKUP_FILE}" ]; then
+	if [ -n "${SUB_BACKUP_FILE}" ]; then
 		BACKUP_FILE=${SUB_BACKUP_FILE}
 	fi;
 	# if sub backup set it set, override current
-	if [ ! -z "${SUB_BACKUP_SET}" ]; then
+	if [ -n "${SUB_BACKUP_SET}" ]; then
 		BACKUP_SET=${SUB_BACKUP_SET};
 	fi;
 	# ovrride compression
-	if [ ! -z "${SUB_COMPRESSION}" ]; then
+	if [ -n "${SUB_COMPRESSION}" ]; then
 		COMPRESSION=${SUB_COMPRESSION};
 	fi;
-	if [ ! -z "${SUB_COMPRESSION_LEVEL}" ]; then
+	if [ -n "${SUB_COMPRESSION_LEVEL}" ]; then
 		COMPRESSION_LEVEL=${SUB_COMPRESSION_LEVEL};
 	fi;
+	# compact interval override
+	if [ -n "${SUB_COMPACT_INTERVAL}" ]; then
+		COMPACT_INTERVAL="${SUB_COMPACT_INTERVAL}";
+	fi;
+	# override check interval
+	if [ -n "${SUB_CHECK_INTERVAL}" ]; then
+		CHECK_INTERVAL="${SUB_CHECK_INTERVAL}";
+	fi;
 	# check override for keep time
-	if [ ! -z "${SUB_KEEP_LAST}" ]; then
+	if [ -n "${SUB_KEEP_LAST}" ]; then
 		KEEP_LAST=${SUB_KEEP_LAST};
 	fi;
-	if [ ! -z "${SUB_KEEP_HOURS}" ]; then
+	if [ -n "${SUB_KEEP_HOURS}" ]; then
 		KEEP_HOURS=${SUB_KEEP_HOURS};
 	fi;
-	if [ ! -z "${SUB_KEEP_DAYS}" ]; then
+	if [ -n "${SUB_KEEP_DAYS}" ]; then
 		KEEP_DAYS=${SUB_KEEP_DAYS};
 	fi;
-	if [ ! -z "${SUB_KEEP_WEEKS}" ]; then
+	if [ -n "${SUB_KEEP_WEEKS}" ]; then
 		KEEP_WEEKS=${SUB_KEEP_WEEKS};
 	fi;
-	if [ ! -z "${SUB_KEEP_MONTHS}" ]; then
+	if [ -n "${SUB_KEEP_MONTHS}" ]; then
 		KEEP_MONTHS=${SUB_KEEP_MONTHS};
 	fi;
-	if [ ! -z "${SUB_KEEP_YEARS}" ]; then
+	if [ -n "${SUB_KEEP_YEARS}" ]; then
 		KEEP_YEARS=${SUB_KEEP_YEARS};
 	fi;
-	if [ ! -z "${SUB_KEEP_WITHIN}" ]; then
+	if [ -n "${SUB_KEEP_WITHIN}" ]; then
 		KEEP_WITHIN=${SUB_KEEP_WITHIN};
 	fi;
 fi;
@@ -464,9 +580,12 @@ if [[ ${TARGET_FOLDER} =~ ^~\/ ]]; then
 	exit 1;
 fi

+# COMPACT_INTERVAL must be a number from -1 to 365
+# CHECK_INTERVAL must be a number from -1 to 365
+
 # log file set and check
 # option folder overrides all other folders
-if [ ! -z "${OPT_LOG_FOLDER}" ]; then
+if [ -n "${OPT_LOG_FOLDER}" ]; then
 	LOG_FOLDER="${OPT_LOG_FOLDER}";
 fi;
 # if empty folder set to default folder
@@ -494,11 +613,26 @@ fi;
 # if ENCRYPTION is empty or not in the valid list fall back to none
 # NOTE This is currently set in default and doesn't need to be set on empty
 # only ivalid should be checked
-#if [ -z "${ENCRYPTION}" ]; then
-#	ENCRYPTION="none";
-#else
-	# TODO check for invalid encryption string
-#fi;
+if
+	[ "${ENCRYPTION}" = "authenticated" ] ||
+	[ "${ENCRYPTION}" = "repokey" ] ||
+	[ "${ENCRYPTION}" = "authenticated-blake2" ] ||
+	[ "${ENCRYPTION}" = "repokey-blake2" ] ;
+then
+	# if "authenticated" or "repokey" a password must be set
+	if [[ ! -v BORG_PASSPHRASE ]] && [[ ! -v BORG_PASSCOMMAND ]] && [[ ! -v BORG_PASSPHRASE_FD ]]; then
+		echo "Encryption method '${ENCRYPTION}' requires a BORG_PASSPHRASE, BORG_PASSCOMMAND or BORG_PASSPHRASE_FD to be set.";
+		exit 1;
+	fi;
+elif [ "${ENCRYPTION}" = "keyfile" ] || [ "${ENCRYPTION}" = "keyfile-blake2" ]; then
+	# if no password, set empty password
+	if [[ ! -v BORG_PASSPHRASE ]] && [[ ! -v BORG_PASSCOMMAND ]] && [[ ! -v BORG_PASSPHRASE_FD ]]; then
+		export BORG_PASSPHRASE="";
+	fi;
+elif [ "${ENCRYPTION}" != "none" ]; then
+	echo "Encryption method '${ENCRYPTION}' is not valid.";
+	exit 1;
+fi;

 ## FUNCTIONS

@@ -513,30 +647,30 @@ function convert_time
 {
 	timestamp=${1};
 	# round to four digits for ms
-	timestamp=$(printf "%1.4f" $timestamp);
+	timestamp=$(printf "%1.4f" "$timestamp");
 	# get the ms part and remove any leading 0
-	ms=$(echo ${timestamp} | cut -d "." -f 2 | sed -e 's/^0*//');
-	timestamp=$(echo ${timestamp} | cut -d "." -f 1);
+	ms=$(echo "${timestamp}" | cut -d "." -f 2 | sed -e 's/^0*//');
+	timestamp=$(echo "${timestamp}" | cut -d "." -f 1);
 	timegroups=(86400 3600 60 1); # day, hour, min, sec
 	timenames=("d" "h" "m" "s"); # day, hour, min, sec
 	output=( );
 	time_string=;
-	for timeslice in ${timegroups[@]}; do
+	for timeslice in "${timegroups[@]}"; do
 		# floor for the division, push to output
 		output[${#output[*]}]=$(awk "BEGIN {printf \"%d\", ${timestamp}/${timeslice}}");
 		timestamp=$(awk "BEGIN {printf \"%d\", ${timestamp}%${timeslice}}");
 	done;

 	for ((i=0; i<${#output[@]}; i++)); do
-		if [ ${output[$i]} -gt 0 ] || [ ! -z "$time_string" ]; then
-			if [ ! -z "${time_string}" ]; then
+		if [ "${output[$i]}" -gt 0 ] || [ -n "$time_string" ]; then
+			if [ -n "${time_string}" ]; then
 				time_string=${time_string}" ";
 			fi;
 			time_string=${time_string}${output[$i]}${timenames[$i]};
 		fi;
 	done;
-	if [ ! -z ${ms} ] && [ "${ms}" != "nan" ] && [ ${ms} -gt 0 ]; then
-		time_string=${time_string}" "${ms}"ms";
+	if [ -n "${ms}" ] && [ "${ms}" != "nan" ] && [ "${ms}" -gt 0 ]; then
+		time_string="${time_string} ${ms}ms";
 	fi;
 	# just in case the time is 0
 	if [ -z "${time_string}" ]; then
--- a/borg.backup.functions.verify.sh
+++ b/borg.backup.functions.verify.sh
@@ -0,0 +1,464 @@
+#!/usr/bin/env bash
+
+# allow variables in printf format string
+# shellcheck disable=SC2059
+
+if [ -z "${MODULE}" ]; then
+	echo "Script cannot be run on its own";
+	exit 1;
+fi;
+
+# start time in seconds
+START=$(date +'%s');
+# set init date, or today if not file is set
+BACKUP_INIT_DATE='';
+if [ -f "${BASE_FOLDER}${BACKUP_INIT_FILE}" ]; then
+	BACKUP_INIT_DATE=$(printf '%(%c)T' "$(cat "${BASE_FOLDER}${BACKUP_INIT_FILE}" 2>/dev/null)");
+fi;
+# start logging from here
+exec &> >(tee -a "${LOG}");
+printf "${PRINTF_MASTER_BLOCK}" "START" "$(date +'%F %T')" "${MODULE}";
+# show info for version always
+printf "${PRINTF_INFO_STRING}" "Script version" "${VERSION}";
+# show type
+printf "${PRINTF_INFO_STRING}" "Backup module" "${MODULE}";
+printf "${PRINTF_INFO_STRING}" "Module version" "${MODULE_VERSION}";
+# borg version
+printf "${PRINTF_INFO_STRING}" "Borg version" "${BORG_VERSION}";
+# host name
+printf "${PRINTF_INFO_STRING}" "Hostname" "${HOSTNAME}";
+# show base folder always
+printf "${PRINTF_INFO_STRING}" "Base folder" "${BASE_FOLDER}";
+# Module init date (when init file was writen)
+printf "${PRINTF_INFO_STRING}" "Module init date" "${BACKUP_INIT_DATE}";
+# print last compact date if positive integer
+# only if borg > 1.2
+if [ "$(version "$BORG_VERSION")" -ge "$(version "1.2.0")" ]; then
+	if [ "${COMPACT_INTERVAL##*[!0-9]*}" ]; then
+		printf "${PRINTF_INFO_STRING}" "Module compact interval" "${COMPACT_INTERVAL}";
+		if [ -f "${BASE_FOLDER}${BACKUP_COMPACT_FILE}" ]; then
+			LAST_COMPACT_DATE=$(cat "${BASE_FOLDER}${BACKUP_COMPACT_FILE}" 2>/dev/null);
+			printf "${PRINTF_INFO_STRING}" "Module last compact" \
+				"$(printf '%(%c)T' "${LAST_COMPACT_DATE}") ($(convert_time $(($(date +%s) - LAST_COMPACT_DATE))) ago)";
+		else
+			printf "${PRINTF_INFO_STRING}" "Module last compact" "No compact run yet"
+		fi;
+	fi;
+fi;
+# print last check date if positive integer
+if [ "${CHECK_INTERVAL##*[!0-9]*}" ]; then
+	printf "${PRINTF_INFO_STRING}" "Module check interval" "${CHECK_INTERVAL}";
+	# get last check date
+	if [ -f "${BASE_FOLDER}${BACKUP_CHECK_FILE}" ]; then
+		LAST_CHECK_DATE=$(cat "${BASE_FOLDER}${BACKUP_CHECK_FILE}" 2>/dev/null);
+		printf "${PRINTF_INFO_STRING}" "Module last check" \
+			"$(printf '%(%c)T' "${LAST_CHECK_DATE}") ($(convert_time $(($(date +%s) - LAST_CHECK_DATE))) ago)";
+	else
+		printf "${PRINTF_INFO_STRING}" "Module last check" "No check run yet";
+	fi;
+fi;
+
+# if force verify is true set VERIFY to 1 unless INFO is 1
+# Needs bash 4.0 at lesat for this
+if [ "${FORCE_VERIFY,,}" = "true" ] && [ "${INFO}" -eq 0 ]; then
+	VERIFY=1;
+	if [ "${DEBUG}" -eq 1 ]; then
+		echo "Force repository verify";
+	fi;
+fi;
+
+# remote borg path
+if [ -n "${TARGET_BORG_PATH}" ]; then
+	if [[ "${TARGET_BORG_PATH}" =~ \ |\' ]]; then
+		echo "Space found in ${TARGET_BORG_PATH}. Aborting";
+		echo "There are issues with passing on paths with spaces"
+		echo "as parameters"
+		exit;
+	fi;
+	OPT_REMOTE="--remote-path="$(printf "%q" "${TARGET_BORG_PATH}");
+fi;
+
+if [ -z "${TARGET_FOLDER}" ]; then
+	echo "[! $(date +'%F %T')] No target folder has been set yet";
+	exit 1;
+else
+	# There are big issues with TARGET FOLDERS with spaces
+	# we should abort anything with this
+	if [[ "${TARGET_FOLDER}" =~ \ |\' ]]; then
+		echo "Space found in ${TARGET_FOLDER}. Aborting";
+		echo "There is some problem with passing paths with spaces as";
+		echo "repository base folder"
+		exit;
+	fi;
+
+	# This does not care for multiple trailing or leading slashes
+	# it just makes sure we have at least one set
+	# for if we have a single slash, remove it
+	TARGET_FOLDER=${TARGET_FOLDER%/}
+	TARGET_FOLDER=${TARGET_FOLDER#/}
+	# and add slash front and back and escape the path
+	TARGET_FOLDER=$(printf "%q" "/${TARGET_FOLDER}/");
+fi;
+
+# if we have user/host then we build the ssh command
+TARGET_SERVER='';
+# allow host only (if full setup in .ssh/config)
+# user@host OR ssh://user@host:port/ IF TARGET_PORT is set
+# user/host/port
+if [ -n "${TARGET_USER}" ] && [ -n "${TARGET_HOST}" ] && [ -n "${TARGET_PORT}" ]; then
+	TARGET_SERVER="ssh://${TARGET_USER}@${TARGET_HOST}:${TARGET_PORT}/";
+# host/port
+elif [ -n "${TARGET_HOST}" ] && [ -n "${TARGET_PORT}" ]; then
+	TARGET_SERVER="ssh://${TARGET_HOST}:${TARGET_PORT}/";
+# user/host
+elif [ -n "${TARGET_USER}" ] && [ -n "${TARGET_HOST}" ]; then
+	TARGET_SERVER="${TARGET_USER}@${TARGET_HOST}:";
+# host
+elif [ -n "${TARGET_HOST}" ]; then
+	TARGET_SERVER="${TARGET_HOST}:";
+fi;
+# we dont allow special characters, so we don't need to special escape it
+REPOSITORY="${TARGET_SERVER}${TARGET_FOLDER}${BACKUP_FILE}";
+printf "${PRINTF_INFO_STRING}" "Repository" "${REPOSITORY}";
+
+# check if given compression name and level are valid
+OPT_COMPRESSION='';
+if [ -n "${COMPRESSION}" ]; then
+	# valid compression
+	if [ "${COMPRESSION}" = "lz4" ] || [ "${COMPRESSION}" = "zlib" ] || [ "${COMPRESSION}" = "lzma" ] || [ "${COMPRESSION}" = "zstd" ]; then
+		OPT_COMPRESSION="-C=${COMPRESSION}";
+		# if COMPRESSION_LEVEL, check it is a valid regex
+		# for zlib, zstd, lzma
+		if [ -n "${COMPRESSION_LEVEL}" ] && { [ "${COMPRESSION}" = "zlib" ] || [ "${COMPRESSION}" = "lzma" ] || [ "${COMPRESSION}" = "zstd" ]; }; then
+			MIN_COMPRESSION=0;
+			MAX_COMPRESSION=0;
+			case "${COMPRESSION}" in
+				zlib|lzma)
+					MIN_COMPRESSION=0;
+					MAX_COMPRESSION=9;
+					;;
+				zstd)
+					MIN_COMPRESSION=1;
+					MAX_COMPRESSION=22;
+					;;
+				*)
+					MIN_COMPRESSION=0;
+					MAX_COMPRESSION=0;
+					;;
+			esac;
+			# if [ "${COMPRESSION}" = "zlib" ] || [ "${COMPRESSION}" = "lzma" ]
+			# 	MIN_COMPRESSION=0;
+			# 	MAX_COMPRESSION=9;
+			# elif [ "${COMPRESSION}" = "zstd" ]; then
+			# 	MIN_COMPRESSION=1;
+			# 	MAX_COMPRESSION=22;
+			# fi;
+			error_message="[! $(date +'%F %T')] Compression level for ${COMPRESSION} needs to be a numeric value between ${MIN_COMPRESSION} and ${MAX_COMPRESSION}: ${COMPRESSION_LEVEL}";
+			if ! [[ "${COMPRESSION_LEVEL}" =~ ${REGEX_NUMERIC} ]]; then
+				echo "${error_message}";
+				exit 1;
+			elif [ "${COMPRESSION_LEVEL}" -lt "${MIN_COMPRESSION}" ] || [ "${COMPRESSION_LEVEL}" -gt "${MAX_COMPRESSION}" ]; then
+				echo "${error_message}";
+				exit 1;
+			else
+				OPT_COMPRESSION=${OPT_COMPRESSION}","${COMPRESSION_LEVEL};
+			fi;
+		fi;
+	else
+		echo "[! $(date +'%F %T')] Compress setting need to be lz4, zstd, zlib or lzma. Or empty for no compression: ${COMPRESSION}";
+		exit 1;
+	fi;
+fi;
+
+# home folder, needs to be set if there is eg a HOME=/ in the crontab
+if [ ! -w "${HOME}" ] || [ "${HOME}" = '/' ]; then
+	HOME=$(eval echo "$(whoami)");
+fi;
+
+# keep optionfs (for files)
+KEEP_OPTIONS=();
+# keep info string (for files)
+KEEP_INFO="";
+# override standard keep for tagged backups
+if [ -n "${ONE_TIME_TAG}" ]; then
+	BACKUP_SET="{now:%Y-%m-%dT%H:%M:%S}";
+	# set empty to avoid problems
+	KEEP_OPTIONS=("");
+else
+	# build options and info string,
+	# also flag BACKUP_SET check if hourly is set
+	BACKUP_SET_VERIFY=0;
+	if [ "${KEEP_LAST}" -gt 0 ]; then
+		KEEP_OPTIONS+=("--keep-last=${KEEP_LAST}");
+		KEEP_INFO="${KEEP_INFO}, last: ${KEEP_LAST}";
+	fi;
+	if [ "${KEEP_HOURS}" -gt 0 ]; then
+		KEEP_OPTIONS+=("--keep-hourly=${KEEP_HOURS}");
+		KEEP_INFO="${KEEP_INFO}, hourly: ${KEEP_HOURS}";
+		BACKUP_SET_VERIFY=1;
+	fi;
+	if [ "${KEEP_DAYS}" -gt 0 ]; then
+		KEEP_OPTIONS+=("--keep-daily=${KEEP_DAYS}");
+		KEEP_INFO="${KEEP_INFO}, daily: ${KEEP_DAYS}";
+	fi;
+	if [ "${KEEP_WEEKS}" -gt 0 ]; then
+		KEEP_OPTIONS+=("--keep-weekly=${KEEP_WEEKS}");
+		KEEP_INFO="${KEEP_INFO}, weekly: ${KEEP_WEEKS}";
+	fi;
+	if [ "${KEEP_MONTHS}" -gt 0 ]; then
+		KEEP_OPTIONS+=("--keep-monthly=${KEEP_MONTHS}");
+		KEEP_INFO="${KEEP_INFO}, monthly: ${KEEP_MONTHS}";
+	fi;
+	if [ "${KEEP_YEARS}" -gt 0 ]; then
+		KEEP_OPTIONS+=("--keep-yearly=${KEEP_YEARS}");
+		KEEP_INFO="${KEEP_INFO}, yearly: ${KEEP_YEARS}";
+	fi;
+	if [ -n "${KEEP_WITHIN}" ]; then
+		# check for invalid string. can only be number + H|d|w|m|y
+		if [[ "${KEEP_WITHIN}" =~ ^[0-9]+[Hdwmy]{1}$ ]]; then
+			KEEP_OPTIONS+=("--keep-within=${KEEP_WITHIN}");
+			KEEP_INFO="${KEEP_INFO}, within: ${KEEP_WITHIN}";
+			if [[ "${KEEP_WITHIN}" == *"H"* ]]; then
+				BACKUP_SET_VERIFY=1;
+			fi;
+		else
+			echo "[! $(date +'%F %T')] KEEP_WITHIN has invalid string.";
+			exit 1;
+		fi;
+	fi;
+	# abort if KEEP_OPTIONS is empty
+	if [ "${#KEEP_OPTIONS[@]}" -eq "0" ]; then
+		echo "[! $(date +'%F %T')] It seems no KEEP_* entries where set in a valid format.";
+		exit 1;
+	fi;
+	# set BACKUP_SET if empty, set to Year-month-day
+	if [ -z "${BACKUP_SET}" ]; then
+		BACKUP_SET="{now:%Y-%m-%d}";
+	fi;
+	# backup set check, and there is no hour entry (%H) in the archive string
+	# we add T%H:%M:%S in this case, before the last }
+	if [ ${BACKUP_SET_VERIFY} -eq 1 ] && [[ "${BACKUP_SET}" != *"%H"* ]]; then
+		BACKUP_SET=$(echo "${BACKUP_SET}" | sed -e "s/}/T%H:%M:%S}/");
+	fi;
+fi;
+
+# check if we have lock file, check pid in lock file, if no matching pid found
+# running remove lock file
+if [ -f "${BASE_FOLDER}${BACKUP_LOCK_FILE}" ]; then
+	LOCK_PID=$(cat "${BASE_FOLDER}${BACKUP_LOCK_FILE}" 2>/dev/null);
+	# check if lock file pid has an active program attached to it
+	if [ -f "/proc/${LOCK_PID}/cmdline" ]; then
+		echo "Script is already running on PID: ${$}";
+		. "${DIR}/borg.backup.functions.close.sh" 1;
+		exit 1;
+	else
+		echo "[#] Clean up stale lock file for PID: ${LOCK_PID}";
+		rm "${BASE_FOLDER}${BACKUP_LOCK_FILE}";
+	fi;
+fi;
+echo "${$}" > "${BASE_FOLDER}${BACKUP_LOCK_FILE}";
+
+# for folders list split set to "#" and keep the old setting as is
+_IFS=${IFS};
+IFS="#";
+# turn off for non file
+if [ "${MODULE}" != "file" ]; then
+	IFS=${_IFS};
+fi;
+
+# borg call, replace ##...## parts during run
+# used in all modules, except 'file'
+_BORG_CALL="${BORG_COMMAND} create ${OPT_REMOTE} -v ${OPT_LIST} ${OPT_PROGRESS} ${OPT_COMPRESSION} -s --stdin-name ##FILENAME## ${REPOSITORY}::##BACKUP_SET## -";
+_BORG_PRUNE="${BORG_COMMAND} prune ${OPT_REMOTE} -v --list ${OPT_PROGRESS} ${DRY_RUN_STATS} -P ##BACKUP_SET_PREFIX## ${KEEP_OPTIONS[*]} ${REPOSITORY}";
+
+# general borg settings
+# set base path to config directory to keep cache/config separated
+export BORG_BASE_DIR="${BASE_FOLDER}";
+# ignore non encrypted access
+export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="${_BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK}";
+# ignore moved repo access
+export BORG_RELOCATED_REPO_ACCESS_IS_OK="${_BORG_RELOCATED_REPO_ACCESS_IS_OK}";
+# and for debug print that tout
+if [ "${DEBUG}" -eq 1 ]; then
+	echo "export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=${_BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK};";
+	echo "export BORG_RELOCATED_REPO_ACCESS_IS_OK=${_BORG_RELOCATED_REPO_ACCESS_IS_OK};";
+	echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";";
+fi;
+# prepare debug commands only
+COMMAND_EXPORT="export BORG_BASE_DIR=\"${BASE_FOLDER}\";"
+COMMAND_INFO="${COMMAND_EXPORT}${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY}";
+# if the is not there, call init to create it
+# if this is user@host, we need to use ssh command to verify if the file is there
+# else a normal verify is ok
+# unless explicit given, verify is skipped
+# MARK: VERIFY / INFO
+if [ "${VERIFY}" -eq 1 ] || [ "${INIT}" -eq 1 ]; then
+	printf "${PRINTF_SUB_BLOCK}" "VERIFY" "$(date +'%F %T')" "${MODULE}";
+	if [ "${DEBUG}" -eq 1 ]; then
+		echo "${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY} 2>&1	";
+	fi;
+	# use borg info and verify if it returns "Repository ID:" in the first line
+	REPO_VERIFY=$(${BORG_COMMAND} info ${OPT_REMOTE} "${REPOSITORY}" 2>&1);
+	__last_error=$?;
+	# on any error in verify command force new INIT
+	if [[ $__last_error -ne 0 ]]; then
+		echo "[!] Repository verify error: ${REPO_VERIFY}";
+		INIT_REPOSITORY=1;
+	fi;
+	# if verrify but no init and repo is there but init file is missing set it
+	if [ "${VERIFY}" -eq 1 ] && [ "${INIT}" -eq 0 ] && [ "${INIT_REPOSITORY}" -eq 0 ] &&
+		[ ! -f "${BASE_FOLDER}${BACKUP_INIT_FILE}" ]; then
+		# write init file
+		echo "[!] Add missing init verify file";
+		date +%s > "${BASE_FOLDER}${BACKUP_INIT_FILE}";
+	fi;
+	# end if verified but repository is not here
+	if [ "${VERIFY}" -eq 1 ] && [ "${INIT}" -eq 0 ] && [ "${INIT_REPOSITORY}" -eq 1 ]; then
+		echo "[! $(date +'%F %T')] No repository. Please run with -I flag to initialze repository";
+		. "${DIR}/borg.backup.functions.close.sh" 1;
+		exit 1;
+	fi;
+	if [ "${EXIT}" -eq 1 ] && [ "${VERIFY}" -eq 1 ] && [ "${INIT}" -eq 0 ]; then
+		echo "Repository exists";
+		echo "For more information run:"
+		echo "${COMMAND_INFO}";
+		. "${DIR}/borg.backup.functions.close.sh";
+		exit;
+	fi;
+fi;
+# MARK: INIT
+if [ "${INIT}" -eq 1 ] && [ "${INIT_REPOSITORY}" -eq 1 ]; then
+
+	printf "${PRINTF_SUB_BLOCK}" "INIT" "$(date +'%F %T')" "${MODULE}";
+	if [ "${DEBUG}" -eq 1 ] || [ "${DRYRUN}" -eq 1 ]; then
+		echo "${BORG_COMMAND} init ${OPT_REMOTE} -e ${ENCRYPTION} ${OPT_VERBOSE} ${REPOSITORY}";
+		echo "${BORG_COMMAND} key export ${REPOSITORY}";
+		echo "${BORG_COMMAND} key export --paper ${REPOSITORY}";
+	fi
+	if [ "${DRYRUN}" -eq 0 ]; then
+		# should trap and exit properly here
+		${BORG_COMMAND} init ${OPT_REMOTE} -e "${ENCRYPTION}" ${OPT_VERBOSE} "${REPOSITORY}";
+		# show the key file
+		if [ "${ENCRYPTION}" = "keyfile" ]; then
+			echo "--- [ENCRYPTION KEY] --[START]-------------------------------------------------->";
+			echo "Store the key and password in a safe place";
+			echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";borg key export [--paper] ${REPOSITORY}";
+			echo "----[BORG KEY] -------------------------------->";
+			${BORG_COMMAND} key export "${REPOSITORY}";
+			echo "----[BORG KEY:paper] -------------------------->";
+			${BORG_COMMAND} key export --paper "${REPOSITORY}";
+			echo "--- [ENCRYPTION KEY] --[END  ]-------------------------------------------------->";
+		fi;
+		# write init file
+		date +%s > "${BASE_FOLDER}${BACKUP_INIT_FILE}";
+		echo "Repository initialized";
+		echo "For more information run:"
+		echo "${COMMAND_INFO}";
+	fi
+	. "${DIR}/borg.backup.functions.close.sh";
+	# exit after init
+	exit;
+elif [ "${INIT}" -eq 1 ] && [ "${INIT_REPOSITORY}" -eq 0 ]; then
+	echo "[!] ($(date +'%F %T')) Repository already initialized";
+	echo "For more information run:"
+	echo "${COMMAND_INFO}";
+	. "${DIR}/borg.backup.functions.close.sh" 1;
+	exit 1;
+fi;
+
+# verify for init file
+if [ ! -f "${BASE_FOLDER}${BACKUP_INIT_FILE}" ]; then
+	echo "[!] ($(date +'%F %T')) It seems the repository has never been initialized."
+	echo "Please run -I to initialize or if already initialzed run with -C for init update."
+	. "${DIR}/borg.backup.functions.close.sh" 1;
+	exit 1;
+fi;
+
+# MARK: LIST / PRINT
+# PRINT OUT current data, only do this if REPO exists
+if [ "${PRINT}" -eq 1 ]; then
+	printf "${PRINTF_SUB_BLOCK}" "PRINT" "$(date +'%F %T')" "${MODULE}";
+	FORMAT="{archive:<45} {comment:6} {start} - {end} [{id}] ({username}@{hostname}){NL}"
+	# show command on debug or dry run
+	if [ "${DEBUG}" -eq 1 ] || [ "${DRYRUN}" -eq 1 ]; then
+		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} list ${OPT_REMOTE} --format ${FORMAT} ${REPOSITORY}";
+	fi;
+	# run info command if not a dry drun
+	if [ "${DRYRUN}" -eq 0 ]; then
+		${BORG_COMMAND} list ${OPT_REMOTE} --format "${FORMAT}" "${REPOSITORY}" ;
+	fi;
+	if [ "${VERBOSE}" -eq 1 ]; then
+		echo "";
+		echo "Base command info:"
+		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} [COMMAND] ${OPT_REMOTE} ${REPOSITORY}::[BACKUP] [PATH]";
+		echo "Replace [COMMAND] with list for listing or extract for restoring backup data."
+		echo "Replace [BACKUP] with archive name."
+		echo "If no [PATH] is given then all files will be restored."
+		echo "Before extracting -n (dry run) is recommended to use."
+		echo "If archive size is needed the info command with archive name has to be used."
+		echo "When listing files in an archive set (::SET) the --format command can be used."
+		echo "Example: \"{mode} {user:6} {group:6} {size:8d} {csize:8d} {dsize:8d} {dcsize:8d} {mtime} {path}{extra} [{health}]{NL}\""
+	else
+		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} [COMMAND] ${OPT_REMOTE} [FORMAT] ${REPOSITORY}::[BACKUP] [PATH]";
+	fi;
+	. "${DIR}/borg.backup.functions.close.sh";
+	exit;
+fi;
+
+# run borg compact command and exit
+if [ "${COMPACT}" -eq 1 ]; then
+	. "${DIR}/borg.backup.functions.compact.sh";
+	. "${DIR}/borg.backup.functions.close.sh";
+	exit;
+fi;
+
+# run borg check command and exit
+if [ "${CHECK}" -eq 1 ]; then
+	. "${DIR}/borg.backup.functions.check.sh";
+	. "${DIR}/borg.backup.functions.close.sh";
+	exit;
+fi;
+
+# DELETE ONE TIME TAG
+if [ -n "${DELETE_ONE_TIME_TAG}" ]; then
+	printf "${PRINTF_SUB_BLOCK}" "DELETE" "$(date +'%F %T')" "${MODULE}";
+	# if a "*" is inside we don't do ONE archive, but globbing via -a option
+	DELETE_ARCHIVE=""
+	OPT_GLOB="";
+	# this is more or less for debug only
+	if [[ "${DELETE_ONE_TIME_TAG}" =~ $REGEX_GLOB ]]; then
+		OPT_GLOB="-a '${DELETE_ONE_TIME_TAG}'"
+	else
+		DELETE_ARCHIVE="::"${DELETE_ONE_TIME_TAG};
+	fi
+	# if this is borg <1.2 OPT_LIST does not work
+	if [ "$(version "$BORG_VERSION")" -lt "$(version "1.2.0")" ]; then
+		OPT_LIST="";
+	fi;
+	# if exists, delete and exit
+	# show command on debug or dry run
+	if [ "${DEBUG}" -eq 1 ]; then
+		echo "${BORG_COMMAND} delete ${OPT_REMOTE} ${OPT_LIST} -s ${OPT_GLOB} ${REPOSITORY}${DELETE_ARCHIVE}";
+	fi;
+	# run delete command if not a dry drun
+	# NOTE seems to be glob is not working if wrapped into another variable
+	if [[ "${DELETE_ONE_TIME_TAG}" =~ $REGEX_GLOB ]]; then
+		${BORG_COMMAND} delete ${OPT_REMOTE} ${OPT_LIST} ${DRY_RUN_STATS} -a "${DELETE_ONE_TIME_TAG}" ${REPOSITORY};
+	else
+		${BORG_COMMAND} delete ${OPT_REMOTE} ${OPT_LIST} ${DRY_RUN_STATS} ${REPOSITORY}${DELETE_ARCHIVE};
+	fi;
+	# if not a dry run, compact repository after delete
+	# not that compact only works on borg 1.2
+	if [ "$(version "$BORG_VERSION")" -ge "$(version "1.2.0")" ]; then
+		if [ "${DRYRUN}" -eq 0 ]; then
+			${BORG_COMMAND} compact ${OPT_REMOTE} "${REPOSITORY}";
+		fi;
+		if [ "${DEBUG}" -eq 1 ]; then
+			echo "${BORG_COMMAND} compact ${OPT_REMOTE} ${REPOSITORY}";
+		fi;
+	fi;
+	. "${DIR}/borg.backup.functions.close.sh";
+	exit;
+fi;
+
+# __END__
--- a/borg.backup.gitea.settings-default
+++ b/borg.backup.gitea.settings-default
@@ -2,10 +2,8 @@

 # rename to borg.backup.gitea.settings to use

-# override settings in borg.backup.settings with SUB_ prefix
-# valid for BACKUP_FILE, BACKUP_SET, COMPRESSION*, KEEP_*
-
 GIT_USER="";
-GITEA_TMP="";
+GITEA_WORKING_DIR="";
+GITEA_TEMP_DIR="";
 GITEA_BIN="";
 GITEA_CONFIG="";
--- a/borg.backup.gitea.sh
+++ b/borg.backup.gitea.sh
@@ -1,18 +1,27 @@
 #!/usr/bin/env bash

+# allow variables in printf format string
+# shellcheck disable=SC2059
+
+# Backup gitea database, all git folders and gitea settings
+
 MODULE="gitea"
-MODULE_VERSION="1.1.1";
+MODULE_VERSION="1.2.2";

 DIR="${BASH_SOURCE%/*}"
 if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 # init system
 . "${DIR}/borg.backup.functions.init.sh";

-# init check file
-BACKUP_INIT_CHECK="borg.backup.gitea.init";
+# init verify, compact and check file
+BACKUP_INIT_FILE="borg.backup.${MODULE}.init";
+BACKUP_COMPACT_FILE="borg.backup.${MODULE}.compact";
+BACKUP_CHECK_FILE="borg.backup.${MODULE}.check";
+# lock file
+BACKUP_LOCK_FILE="borg.backup.${MODULE}.lock";

-# check valid data
-. "${DIR}/borg.backup.functions.check.sh";
+# verify valid data
+. "${DIR}/borg.backup.functions.verify.sh";
 # if info print info and then abort run
 . "${DIR}/borg.backup.functions.info.sh";

@@ -23,9 +32,13 @@ if [ -z "${GIT_USER}" ]; then
 	GIT_USER="git";
 fi;
 # set GITEA_* if not set
-if [ -z "${GITEA_TMP}" ]; then
+if [ -z "${GITEA_WORKING_DIR}" ]; then
 	# run gitea backup (user mktemp?)
-	GITEA_TMP="/tmp/gitea/";
+	GITEA_WORKING_DIR="/var/tmp/gitea/";
+fi;
+# general temp folder for temporary data storage, this is not working output folder
+if [ -z "${GITEA_TEMP_DIR}" ]; then
+	GITEA_TEMP_DIR="/var/tmp";
 fi;
 if [ -z "${GITEA_BIN}" ]; then
 	GITEA_BIN="/usr/local/bin/gitea";
@@ -33,6 +46,10 @@ fi;
 if [ -z "${GITEA_CONFIG}" ]; then
 	GITEA_CONFIG="/etc/gitea/app.ini"
 fi;
+# This one is not advertised in the config file as it is not recommended to change
+if [ -z "${GITEA_EXPORT_TYPE}" ]; then
+	GITEA_EXPORT_TYPE="zip";
+fi;
 if [ ! -f "${GITEA_BIN}" ]; then
 	echo "[! $(date +'%F %T')] Cannot find gitea binary";
 	. "${DIR}/borg.backup.functions.close.sh" 1;
@@ -43,6 +60,16 @@ if [ ! -f "${GITEA_CONFIG}" ]; then
 	. "${DIR}/borg.backup.functions.close.sh" 1;
 	exit 1;
 fi;
+# some basic checks with abort
+if [ ! -d "${GITEA_TEMP_DIR}" ]; then
+	echo "Temp directory does not exist: ${GITEA_TEMP_DIR}";
+	exit;
+fi;
+# we should check GITEA_EXPORT_TYPE too at some point for an allow list
+# At the moment warn if not zip
+if [ "${GITEA_EXPORT_TYPE}" != "zip" ]; then
+	echo "[!!!!] The gitea export type has been changed from 'zip' to '${GITEA_EXPORT_TYPE}'. This can either break or make exports take very ling";
+fi;
 # Filename
 FILENAME="gitea.backup.zip";
 # backup set and prefix
@@ -52,9 +79,9 @@ BACKUP_SET_NAME="${ONE_TIME_TAG}${BACKUP_SET_PREFIX}${BACKUP_SET}";
 # borg call
 BORG_CALL=$(echo "${_BORG_CALL}" | sed -e "s/##FILENAME##/${FILENAME}/" | sed -e "s/##BACKUP_SET##/${BACKUP_SET_NAME}/");
 BORG_PRUNE=$(echo "${_BORG_PRUNE}" | sed -e "s/##BACKUP_SET_PREFIX##/${BACKUP_SET_PREFIX}/");
-echo "--- [git data and database: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+printf "${PRINTF_SUB_BLOCK}" "BACKUP: git data and database" "$(date +'%F %T')" "${MODULE}";
 if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
-	echo "sudo -u ${GIT_USER} ${GITEA_BIN} dump -c ${GITEA_CONFIG} -w ${GITEA_TMP} -L -f - | ${BORG_CALL}";
+	echo "sudo -u ${GIT_USER} ${GITEA_BIN} dump -c ${GITEA_CONFIG} -w ${GITEA_WORKING_DIR} -t ${GITEA_TEMP_DIR} --type ${GITEA_EXPORT_TYPE} -L -f - | ${BORG_CALL}";
 	if [ -z "${ONE_TIME_TAG}" ]; then
 		echo "${BORG_PRUNE}";
 	fi;
@@ -64,21 +91,23 @@ if [ ${DRYRUN} -eq 0 ]; then
 		# below was an old workaround
 		#export USER="${LOGNAME}" # workaround for broken gitea EUID check
 		# make sure temp folder is there and is set as git. user
-		if [ ! -d "${GITEA_TMP}" ]; then
-			mkdir -p "${GITEA_TMP}";
+		if [ ! -d "${GITEA_WORKING_DIR}" ]; then
+			mkdir -p "${GITEA_WORKING_DIR}";
 		fi;
-		chown -R ${GIT_USER}. "${GITEA_TMP}";
+		chown -R ${GIT_USER}: "${GITEA_WORKING_DIR}";
 		# this needs to be run in a folder that can be stat by git user
-		cd "${GITEA_TMP}";
-		sudo -u ${GIT_USER} ${GITEA_BIN} dump -c ${GITEA_CONFIG} -w ${GITEA_TMP} -L -f - | ${BORG_CALL};
-	) | sed 's/\x1B\[[0-9;]\{1,\}[A-Za-z]//g' # remove all ESC strings
+		cd "${GITEA_WORKING_DIR}" || exit 1;
+		sudo -u ${GIT_USER} ${GITEA_BIN} dump -c ${GITEA_CONFIG} -w ${GITEA_WORKING_DIR} -t ${GITEA_TEMP_DIR} --type ${GITEA_EXPORT_TYPE} -L -f - | ${BORG_CALL};
+	) 2>&1 | sed 's/\x1B\[[0-9;]\{1,\}[A-Za-z]//g' # remove all ESC strings
 fi;
 if [ -z "${ONE_TIME_TAG}" ]; then
-	echo "--- [PRUNE : $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+	printf "${PRINTF_SUB_BLOCK}" "PRUNE" "$(date +'%F %T')" "${MODULE}";
 	echo "Prune repository with keep${KEEP_INFO:1}";
 	${BORG_PRUNE};
 	# if this is borg version >1.2 we need to run compact after prune
-	. "${DIR}/borg.backup.functions.compact.sh";
+	. "${DIR}/borg.backup.functions.compact.sh" "auto";
+	# check in auto mode
+	. "${DIR}/borg.backup.functions.check.sh" "auto";
 fi;

 . "${DIR}/borg.backup.functions.close.sh";
--- a/borg.backup.mysql.sh
+++ b/borg.backup.mysql.sh
@@ -1,16 +1,16 @@
 #!/usr/bin/env bash

+# allow variables in printf format string
+# shellcheck disable=SC2059
+
 # Backup MySQL/MariaDB
 # default is per table dump, can be set to one full dump
 # config override set in borg.backup.mysql.settings
 # if run as mysql user, be sure user is in the backup group

-# Run -I first to initialize repository
-# There are no automatic repository checks unless -C is given
-
 # set last edit date + time
 MODULE="mysql"
-MODULE_VERSION="1.1.0";
+MODULE_VERSION="1.1.5";

 DIR="${BASH_SOURCE%/*}"
 if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
@@ -18,14 +18,18 @@ if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 . "${DIR}/borg.backup.functions.init.sh";

 # include and exclude file
-INCLUDE_FILE="borg.backup.mysql.include";
-EXCLUDE_FILE="borg.backup.mysql.exclude";
-SCHEMA_ONLY_FILE="borg.backup.mysql.schema-only";
-# init check file
-BACKUP_INIT_CHECK="borg.backup.mysql.init";
+INCLUDE_FILE="borg.backup.${MODULE}.include";
+EXCLUDE_FILE="borg.backup.${MODULE}.exclude";
+SCHEMA_ONLY_FILE="borg.backup.${MODULE}.schema-only";
+# init verify, compact and check file
+BACKUP_INIT_FILE="borg.backup.${MODULE}.init";
+BACKUP_COMPACT_FILE="borg.backup.${MODULE}.compact";
+BACKUP_CHECK_FILE="borg.backup.${MODULE}.check";
+# lock file
+BACKUP_LOCK_FILE="borg.backup.${MODULE}.lock";

-# check valid data
-. "${DIR}/borg.backup.functions.check.sh";
+# verify valid data
+. "${DIR}/borg.backup.functions.verify.sh";
 # if info print info and then abort run
 . "${DIR}/borg.backup.functions.info.sh";

@@ -50,10 +54,10 @@ if [ ! -f "${MYSQL_CMD}" ]; then
 	. "${DIR}/borg.backup.functions.close.sh" 1;
 	exit 1;
 fi;
-# check that the user can actually do, else abort here
+# verify that the user can actually do, else abort here
 # note: this is the only way to not error
-_MYSQL_CHECK=$(mysqladmin ${MYSQL_DB_CONFIG_PARAM} ping 2>&1);
-_MYSQL_OK=$(echo "${_MYSQL_CHECK}" | grep "is alive");
+_MYSQL_VERIFY=$(mysqladmin ${MYSQL_DB_CONFIG_PARAM} ping 2>&1);
+_MYSQL_OK=$(echo "${_MYSQL_VERIFY}" | grep "is alive");
 if [ -z "${_MYSQL_OK}" ]; then
 	echo "[! $(date +'%F %T')] Current user has no access right to mysql database";
 	. "${DIR}/borg.backup.functions.close.sh" 1;
@@ -61,10 +65,11 @@ if [ -z "${_MYSQL_OK}" ]; then
 fi;
 # below is for file name only
 # set DB_VERSION (Distrib n.n.n-type)
-# NEW: mysql  Ver 15.1 Distrib 10.5.12-MariaDB, for debian-linux-gnu (x86_64) using  EditLine wrapper
-# OLD: mysql  Ver 14.14 Distrib 5.7.35, for Linux (x86_64) using  EditLine wrapper
+# OLD:     mysql  Ver 14.14 Distrib 5.7.35, for Linux (x86_64) using  EditLine wrapper
+# NEW:     mysql  Ver 15.1 Distrib 10.5.12-MariaDB, for debian-linux-gnu (x86_64) using  EditLine wrapper
+# VARIANT: mysql from 11.8.3-MariaDB, client 15.2 for debian-linux-gnu (x86_64) using  EditLine wrapper
 _DB_VERSION_TYPE=$("${MYSQL_CMD}" --version);
-_DB_VERSION=$(echo "${_DB_VERSION_TYPE}" | sed 's/.*Distrib \([0-9]\{1,\}\.[0-9]\{1,\}\)\.[0-9]\{1,\}.*/\1/');
+_DB_VERSION=$(echo "${_DB_VERSION_TYPE}" | sed 's/.*\(Distrib\|from\) \([0-9]\{1,\}\.[0-9]\{1,\}\)\.[0-9]\{1,\}.*/\2/');
 DB_VERSION=$(echo "${_DB_VERSION}" | cut -d " " -f 1);
 # temporary until correct type detection is set
 DB_TYPE="mysql";
@@ -85,14 +90,15 @@ EVENTDB="mysql"
 EVENTS="--events"

 # ALL IN ONE FILE or PER DATABASE FLAG
-if [ ! -z "${DATABASE_FULL_DUMP}" ]; then
+if [ -n "${DATABASE_FULL_DUMP}" ]; then
 	SCHEMA_ONLY='';
 	schema_flag='data';
 	if [ "${DATABASE_FULL_DUMP}" = "schema" ]; then
 		SCHEMA_ONLY='--no-data';
 		schema_flag='schema';
 	fi;
-	echo "---[BACKUP: all databases: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+	LOCAL_START=$(date +'%s');
+	printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "all databases" "$(date +'%F %T')" "${MODULE}";
 	# We only do a full backup and not per table backup here
 	# Filename
 	FILENAME="all-${schema_flag}-${DB_TYPE}_${DB_VERSION}_${DB_HOST}_${DB_PORT}.sql"
@@ -119,15 +125,18 @@ if [ ! -z "${DATABASE_FULL_DUMP}" ]; then
 		fi;
 	fi;
 	if [ -z "${ONE_TIME_TAG}" ]; then
-		echo "--- [PRUNE : all databases: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+		printf "${PRINTF_SUBEXT_BLOCK}" "PRUNE" "all databases" "$(date +'%F %T')" "${MODULE}";
 		echo "Prune repository with keep${KEEP_INFO:1}";
 		${BORG_PRUNE};
 	fi;
+	DURATION=$(( $(date +'%s') - LOCAL_START ));
+	printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "all databases" "${MODULE}" "$(convert_time ${DURATION})";
 else
 	${MYSQL_CMD} ${MYSQL_DB_CONFIG_PARAM} -B -N -e "show databases" |
-	while read db; do
-		echo "========[DB: ${db}]========================[${MODULE}]====================================>";
-		echo "--- [BACKUP: ${db}: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+	while read -r db; do
+		LOCAL_START=$(date +'%s');
+		printf "${PRINTF_DB_SUB_BLOCK}" "DB" "${db}" "${MODULE}";
+		printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "${db}" "$(date +'%F %T')" "${MODULE}";
 		# exclude checks
 		include=0;
 		if [ -s "${BASE_FOLDER}${INCLUDE_FILE}" ]; then
@@ -142,7 +151,7 @@ else
 		fi;
 		exclude=0;
 		if [ -f "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
-			while read excl_db; do
+			while read -r excl_db; do
 				if [ "${db}" = "${excl_db}" ]; then
 					exclude=1;
 					break;
@@ -172,7 +181,7 @@ else
 			SCHEMA_ONLY=''; # empty for all
 			schema_flag='data'; # or data
 			if [ -s "${BASE_FOLDER}${SCHEMA_ONLY_FILE}" ]; then
-				while read schema_db; do
+				while read -r schema_db; do
 					if [ "${db}" = "${schema_db}" ]; then
 						SCHEMA_ONLY='--no-data';
 						schema_flag='schema';
@@ -205,19 +214,23 @@ else
 				fi;
 			fi;
 			if [ -z "${ONE_TIME_TAG}" ]; then
-				echo "--- [PRUNE : ${db}: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+				printf "${PRINTF_SUBEXT_BLOCK}" "PRUNE" "${db}" "$(date +'%F %T')" "${MODULE}";
 				echo "Prune repository prefixed ${BACKUP_SET_PREFIX} with keep${KEEP_INFO:1}";
 				${BORG_PRUNE};
 			fi;
 		else
 			echo "- [E] ${db}";
 		fi;
+		DURATION=$(( $(date +'%s') - LOCAL_START ));
+		printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "${db}" "${MODULE}" "$(convert_time ${DURATION})";
 	done;
 fi;
 # run compact at the end if not a dry run
 if [ -z "${ONE_TIME_TAG}" ]; then
 	# if this is borg version >1.2 we need to run compact after prune
-	. "${DIR}/borg.backup.functions.compact.sh";
+	. "${DIR}/borg.backup.functions.compact.sh" "auto";
+	# check in auto mode
+	. "${DIR}/borg.backup.functions.check.sh" "auto";
 fi;

 . "${DIR}/borg.backup.functions.close.sh";
--- a/borg.backup.pgsql.settings-default
+++ b/borg.backup.pgsql.settings-default
@@ -9,5 +9,15 @@
 # note that with this databases that have been dropped need to be pruned manually
 # if 'schema' word is used, only schema data is dumped
 DATABASE_FULL_DUMP="";
-# override default postgres user
+# override default postgres user and sudo user for all postgres commands
+# All commands must be run as the postgres admin user
 DATABASE_USER="";
+# disable sudo usage by setting to "0", default is to use sudo
+DATABASE_USE_SUDO="";
+# set the sudo user, if not set postgres or DATABASE_USER is used
+DATABASE_SUDO_USER="";
+# override port
+DATABASE_PORT="";
+# override database host, if set to local, localhost or 127.0.0.1 it will use sockets to connect
+# for socket connection ident or sudo has to be used
+DATABASE_HOST="";
--- a/borg.backup.pgsql.sh
+++ b/borg.backup.pgsql.sh
@@ -1,44 +1,77 @@
 #!/usr/bin/env bash

+# allow variables in printf format string
+# shellcheck disable=SC2059
+
 # Backup PostgreSQL
 # default is per table dump, can be set to one full dump
 # config override set in borg.backup.pgsql.settings
 # if run as postgres user, be sure user is in the backup group

-# Run -I first to initialize repository
-# There are no automatic repository checks unless -C is given
-
 # set last edit date + time
 MODULE="pgsql"
-MODULE_VERSION="1.1.1";
+MODULE_VERSION="1.3.0";


 DIR="${BASH_SOURCE%/*}"
 if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
+DEFAULT_DATABASE_USE_SUDO="1";
+DEFAULT_DATABASE_USER="postgres";
 # init system
 . "${DIR}/borg.backup.functions.init.sh";

 # include and exclude file
-INCLUDE_FILE="borg.backup.pgsql.include";
-EXCLUDE_FILE="borg.backup.pgsql.exclude";
-SCHEMA_ONLY_FILE="borg.backup.pgsql.schema-only";
-# init check file
-BACKUP_INIT_CHECK="borg.backup.pgsql.init";
+INCLUDE_FILE="borg.backup.${MODULE}.include";
+EXCLUDE_FILE="borg.backup.${MODULE}.exclude";
+SCHEMA_ONLY_FILE="borg.backup.${MODULE}.schema-only";
+DATA_ONLY_FILE="borg.backup.${MODULE}.data-only";
+# init verify, compact and check file
+BACKUP_INIT_FILE="borg.backup.${MODULE}.init";
+BACKUP_COMPACT_FILE="borg.backup.${MODULE}.compact";
+BACKUP_CHECK_FILE="borg.backup.${MODULE}.check";
+# lock file
+BACKUP_LOCK_FILE="borg.backup.${MODULE}.lock";

-# check valid data
-. "${DIR}/borg.backup.functions.check.sh";
+# verify valid data
+. "${DIR}/borg.backup.functions.verify.sh";
 # if info print info and then abort run
 . "${DIR}/borg.backup.functions.info.sh";

-if [ ! -z "${DATABASE_USER}" ]; then
-	DB_USER=${DATABASE_USER};
+# set db and sudo user
+if [ -n "${DATABASE_USER}" ]; then
+	DB_USER="${DATABASE_USER}";
 else
-	DB_USER='postgres';
+	DB_USER="${DEFAULT_DATABASE_USER}";
 fi;
+if [ -n "${DATABASE_SUDO_USER}" ]; then
+	DB_SUDO_USER="${DATABASE_SUDO_USER}";
+else
+	DB_SUDO_USER="${DB_USER}";
+fi;
+# set flag if we should use sudo
+if [ -n "${DATABASE_USE_SUDO}" ]; then
+	USE_SUDO="${DATABASE_USE_SUDO}";
+else
+	USE_SUDO="${DEFAULT_DATABASE_USE_SUDO}";
+fi;
+# sudo command, or none if not requested
+SUDO_COMMAND_A=("sudo" "-u" "${DB_SUDO_USER}");
+if [ "${USE_SUDO}" -eq "0" ]; then
+	SUDO_COMMAND_A=()
+fi;
+
 # get current pgsql version first
 # if first part <10 then user full, else only first part
 # eg 9.4 -> 9.4, 12.5 -> 12
-PG_VERSION=$(pgv=$(psql -U ${DB_USER} -d template1 -t -A -F "," -X -q -c 'select version();' | sed -e 's/^PostgreSQL \([0-9]\{1,\}\.[0-9]\{1,\}\).*/\1/'); if [[ $(echo "${pgv}" | cut -d "." -f 1) -ge 10 ]]; then echo "${pgv}" | cut -d "." -f 1; else echo "${pgv}" | cut -d "." -f 1,2; fi );
+PG_VERSION_CMD=("${SUDO_COMMAND_A[@]}" "psql" "-U" "${DB_USER}" "-d" "template1" "-t" "-A" "-F" "," "-X" "-q" "-c" "select version();");
+PG_VERSION=$(
+	pgv=$("${PG_VERSION_CMD[@]}" | sed -e 's/^PostgreSQL \([0-9]\{1,\}\.[0-9]\{1,\}\).*/\1/');
+	if [[ $(echo "${pgv}" | cut -d "." -f 1) -ge 10 ]]; then
+		echo "${pgv}" | cut -d "." -f 1;
+	else
+		echo "${pgv}" | cut -d "." -f 1,2;
+	fi;
+);
 _PATH_PG_VERSION=${PG_VERSION};
 _backup_error=$?;
 if [ $_backup_error -ne 0 ] || [ -z "${PG_VERSION}" ]; then
@@ -48,76 +81,111 @@ if [ $_backup_error -ne 0 ] || [ -z "${PG_VERSION}" ]; then
 fi;

 # path set per Distribution type and current running DB version
+# Debian/Ubuntu: PG_BASE_PATH='/usr/lib/postgresql/';
 # Redhat: PG_BASE_PATH='/usr/pgsql-';
-# AWS 1: PG_BASE_PATH='/usr/lib64/pgsql';
-# Debian: PG_BASE_PATH='/usr/lib/postgresql/';
-PG_BASE_PATH='/usr/lib/postgresql/';
-if [ ! -f "${PG_BASE_PATH}${_PATH_PG_VERSION}/bin/psql" ]; then
-	PG_BASE_PATH='/usr/pgsql-';
-	if [ ! -f "${PG_BASE_PATH}${_PATH_PG_VERSION}/bin/psql" ]; then
-		PG_BASE_PATH='/usr/lib64/pgsql';
-		_PATH_PG_VERSION=$(echo "${PG_VERSION}" | sed -e 's/\.//');
-		if [ ! -f "${PG_BASE_PATH}${_PATH_PG_VERSION}/bin/psql" ]; then
-			echo "[! $(date +'%F %T')] PostgreSQL not found in any paths";
-			. "${DIR}/borg.backup.functions.close.sh" 1;
-			exit 1;
-		fi;
+PG_BASE_PATH_LIST=("/usr/lib/postgresql" "/usr/lib64/pgsql");
+PG_BASE_PATH="";
+for path in "${PG_BASE_PATH_LIST[@]}"; do
+	if [ -f "${path}/${_PATH_PG_VERSION}/bin/psql" ]; then
+		PG_BASE_PATH="${path}/";
+		break;
 	fi;
+done;
+if [ -z "${PG_BASE_PATH}" ]; then
+	echo "[! $(date +'%F %T')] PostgreSQL not found in any paths";
+		. "${DIR}/borg.backup.functions.close.sh" 1;
+	exit 1;
 fi;
-PG_PATH=${PG_BASE_PATH}${_PATH_PG_VERSION}'/bin/';
-PG_PSQL=${PG_PATH}'psql';
-PG_DUMP=${PG_PATH}'pg_dump';
-PG_DUMPALL=${PG_PATH}'pg_dumpall';
+PG_PATH="${PG_BASE_PATH}${_PATH_PG_VERSION}/bin/";
+PG_PSQL=("${PG_PATH}psql");
+PG_DUMP=("${PG_PATH}pg_dump");
+PG_DUMPALL=("${PG_PATH}pg_dumpall");
+PG_ERROR=0
 # check that command are here
-if [ ! -f "${PG_PSQL}" ]; then
-	echo "[! $(date +'%F %T')] psql binary not found in ${PG_PATH}";
-	. "${DIR}/borg.backup.functions.close.sh" 1;
-	exit 1;
+if [ ! -f "${PG_PSQL[0]}" ]; then
+	echo "[!] ($(date +'%F %T')) psql binary not found in ${PG_PATH}";
+	PG_ERROR=1;
 fi;
-if [ ! -f "${PG_DUMP}" ]; then
-	echo "[! $(date +'%F %T')] pg_dump binary not found in ${PG_PATH}";
-	. "${DIR}/borg.backup.functions.close.sh" 1;
-	exit 1;
+if [ ! -f "${PG_DUMP[0]}" ]; then
+	echo "[!] ($(date +'%F %T')) pg_dump binary not found in ${PG_PATH}";
+	PG_ERROR=1;
 fi;
-if [ ! -f "${PG_DUMPALL}" ]; then
-	echo "[! $(date +'%F %T')] pg_dumpall binary not found in ${PG_PATH}";
+if [ ! -f "${PG_DUMPALL[0]}" ]; then
+	echo "[!] ($(date +'%F %T')) pg_dumpall binary not found in ${PG_PATH}";
+	PG_ERROR=1;
+fi;
+if [ ${PG_ERROR} -ne 0 ]; then
 	. "${DIR}/borg.backup.functions.close.sh" 1;
-	exit 1;
+	exit ${PG_ERROR};
+fi;
+# prefix with sudo, if sudo command is requested
+if [ "${USE_SUDO}" -ne "0" ]; then
+	PG_PSQL=("${SUDO_COMMAND_A[@]}" "${PG_PSQL[@]}");
+	PG_DUMP=("${SUDO_COMMAND_A[@]}" "${PG_DUMP[@]}");
+	PG_DUMPALL=("${SUDO_COMMAND_A[@]}" "${PG_DUMPALL[@]}");
 fi;

 DB_VERSION=${PG_VERSION};
-# TODO override port/host info
-DB_PORT='5432';
-DB_HOST='local'; # or <host>
-CONN_DB_HOST=''; # -h <host>
-CONN_DB_PORT=''; # -p <port>
+# override for port or host name
+if [ -n "${DATABASE_PORT}" ] && [[ "${DATABASE_PORT}" =~ ${REGEX_PORT} ]]; then
+	DB_PORT="${DATABASE_PORT}";
+else
+	DB_PORT="5432";
+fi;
+if [ -n "${DATABASE_HOST}" ]; then
+	DB_HOST="${DATABASE_HOST}";
+else
+	DB_HOST="local";
+fi;
+# use socket for local
+if [ "${DB_HOST}" = "local" ] || [ "${DB_HOST}" = "localhost" ] || [ "${DB_HOST}" = "127.0.0.1" ]; then
+	DB_HOST='local';
+	CONN_DB_HOST=();
+	CONN_DB_PORT=();
+else
+	CONN_DB_HOST=("-p" "${DB_HOST}"); # -h <host>
+	CONN_DB_PORT=("-h" "${DB_HOSTNAME}"); # -p <port>
+fi;
+# now add user, con, host to all commands
+PG_PSQL=("${PG_PSQL[@]}" "-U" "${DB_USER}" "${CONN_DB_HOST[@]}" "${CONN_DB_PORT[@]}");
+PG_DUMP=("${PG_DUMP[@]}" "-U" "${DB_USER}" "${CONN_DB_HOST[@]}" "${CONN_DB_PORT[@]}");
+PG_DUMPALL=("${PG_DUMPALL[@]}" "-U" "${DB_USER}" "${CONN_DB_HOST[@]}" "${CONN_DB_PORT[@]}");

 # ALL IN ONE FILE or PER DATABASE FLAG
-if [ ! -z "${DATABASE_FULL_DUMP}" ]; then
+if [ -n "${DATABASE_FULL_DUMP}" ]; then
 	SCHEMA_ONLY='';
 	schema_flag='data';
 	if [ "${DATABASE_FULL_DUMP}" = "schema" ]; then
 		SCHEMA_ONLY='-s';
 		schema_flag='schema';
 	fi;
-	echo "--- [BACKUP: all databases: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+	LOCAL_START=$(date +'%s');
+	printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "all databases" "$(date +'%F %T')" "${MODULE}";
 	# Filename
-	FILENAME-"all.${DB_USER}.NONE.${schema_flag}-${DB_VERSION}_${DB_HOST}_${DB_PORT}.c.sql"
+	FILENAME="all.${DB_USER}.NONE.${schema_flag}-${DB_VERSION}_${DB_HOST}_${DB_PORT}.c.sql"
 	# backup set:
 	BACKUP_SET_PREFIX="${MODULE},all-";
 	BACKUP_SET_NAME="${ONE_TIME_TAG}${BACKUP_SET_PREFIX}${schema_flag}-${BACKUP_SET}";
 	# borg call
-	BORG_CALL=$(echo "${_BORG_CALL}" | sed -e "s/##FILENAME##/${FILENAME}/" | sed -e "s/##BACKUP_SET##/${BACKUP_SET_NAME}/");
-	BORG_PRUNE=$(echo "${_BORG_PRUNE}" | sed -e "s/##BACKUP_SET_PREFIX##/${BACKUP_SET_PREFIX}/");
+	BORG_CALL=$(
+		echo "${_BORG_CALL}" |
+		sed -e "s/##FILENAME##/${FILENAME}/" |
+		sed -e "s/##BACKUP_SET##/${BACKUP_SET_NAME}/"
+	);
+	BORG_PRUNE=$(
+		echo "${_BORG_PRUNE}" |
+		sed -e "s/##BACKUP_SET_PREFIX##/${BACKUP_SET_PREFIX}/"
+	);
+	PG_DUMPALL_CMD=("${PG_DUMPALL[@]}" "${SCHEMA_ONLY}" "-c");
 	if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
 		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";";
-		echo "${PG_DUMPALL} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} ${SCHEMA_ONLY} -c | ${BORG_CALL}";
+		echo "${PG_DUMPALL_CMD[*]} | ${BORG_CALL}";
 		if [ -z "${ONE_TIME_TAG}" ]; then
 			echo "${BORG_PRUNE}";
 		fi;
 	fi;
 	if [ ${DRYRUN} -eq 0 ]; then
-		$(${PG_DUMPALL} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} ${SCHEMA_ONLY} -c | ${BORG_CALL});
+		"${PG_DUMPALL_CMD[@]}" | ${BORG_CALL};
 		_backup_error=$?;
 		if [ $_backup_error -ne 0 ]; then
 			echo "[! $(date +'%F %T')] Backup creation failed for full dump with error code: ${_backup_error}";
@@ -126,32 +194,43 @@ if [ ! -z "${DATABASE_FULL_DUMP}" ]; then
 		fi;
 	fi;
 	if [ -z "${ONE_TIME_TAG}" ]; then
-		echo "--- [PRUNE : all databases: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+		printf "${PRINTF_SUBEXT_BLOCK}" "PRUNE" "all databases" "$(date +'%F %T')" "${MODULE}";
 		echo "Prune repository with keep${KEEP_INFO:1}";
 		${BORG_PRUNE};
 	fi;
+	DURATION=$(( $(date +'%s') - LOCAL_START ));
+	printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "all databases" "${MODULE}" "$(convert_time ${DURATION})";
 else
 	# dump globals first
 	db="pg_globals";
 	schema_flag="data";
-	echo "--- [BACKUP: ${db}: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+	LOCAL_START=$(date +'%s');
+	printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "${db}" "$(date +'%F %T')" "${MODULE}";
 	# Filename
 	FILENAME="${db}.${DB_USER}.NONE.${schema_flag}-${DB_VERSION}_${DB_HOST}_${DB_PORT}.c.sql"
 	# backup set:
 	BACKUP_SET_PREFIX="${MODULE},${db}-";
 	BACKUP_SET_NAME="${ONE_TIME_TAG}${BACKUP_SET_PREFIX}${schema_flag}-${BACKUP_SET}";
 	# borg call
-	BORG_CALL=$(echo "${_BORG_CALL}" | sed -e "s/##FILENAME##/${FILENAME}/" | sed -e "s/##BACKUP_SET##/${BACKUP_SET_NAME}/");
-	BORG_PRUNE=$(echo "${_BORG_PRUNE}" | sed -e "s/##BACKUP_SET_PREFIX##/${BACKUP_SET_PREFIX}/");
+	BORG_CALL=$(
+		echo "${_BORG_CALL}" |
+		sed -e "s/##FILENAME##/${FILENAME}/" |
+		sed -e "s/##BACKUP_SET##/${BACKUP_SET_NAME}/"
+	);
+	BORG_PRUNE=$(
+		echo "${_BORG_PRUNE}" |
+		sed -e "s/##BACKUP_SET_PREFIX##/${BACKUP_SET_PREFIX}/"
+	);
+	PG_DUMPALL_CMD=("${PG_DUMPALL[@]}" "--globals-only");
 	if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
 		echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";";
-		echo "${PG_DUMPALL} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} --globals-only | ${BORG_CALL}";
+		echo "${PG_DUMPALL_CMD[*]} | ${BORG_CALL}";
 		if [ -z "${ONE_TIME_TAG}" ]; then
 			echo "${BORG_PRUNE}";
 		fi;
 	fi;
 	if [ ${DRYRUN} -eq 0 ]; then
-		${PG_DUMPALL} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} --globals-only | ${BORG_CALL};
+		"${PG_DUMPALL_CMD[@]}" | ${BORG_CALL};
 		_backup_error=$?;
 		if [ $_backup_error -ne 0 ]; then
 			echo "[! $(date +'%F %T')] Backup creation failed for ${db} dump with error code: ${_backup_error}";
@@ -160,22 +239,29 @@ else
 		fi;
 	fi;
 	if [ -z "${ONE_TIME_TAG}" ]; then
-		echo "--- [PRUNE : ${db}: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+		printf "${PRINTF_SUBEXT_BLOCK}" "PRUNE" "${db}" "$(date +'%F %T')" "${MODULE}";
 		echo "Prune repository with keep${KEEP_INFO:1}";
 		${BORG_PRUNE};
 	fi;
+	DURATION=$(( $(date +'%s') - LOCAL_START ));
+	printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "${db}" "${MODULE}" "$(convert_time ${DURATION})";

 	# get list of tables
-	for owner_db in $(${PG_PSQL} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} -d template1 -t -A -F "," -X -q -c "SELECT pg_catalog.pg_get_userbyid(datdba) AS owner, datname, pg_catalog.pg_encoding_to_char(encoding) AS encoding FROM pg_catalog.pg_database WHERE datname "\!"~ 'template(0|1)' ORDER BY datname;"); do
+	GET_DB_CMD=(
+		"${PG_PSQL[@]}" "-d" "template1" "-t" "-A" "-F" "," "-X" "-q" "-c"
+		"SELECT pg_catalog.pg_get_userbyid(datdba) AS owner, datname, pg_catalog.pg_encoding_to_char(encoding) AS encoding FROM pg_catalog.pg_database WHERE datname !~ 'template(0|1)' ORDER BY datname;"
+	);
+	for owner_db in $("${GET_DB_CMD[@]}"); do
+		LOCAL_START=$(date +'%s');
 		# get the user who owns the DB too
-		owner=$(echo ${owner_db} | cut -d "," -f 1);
-		db=$(echo ${owner_db} | cut -d "," -f 2);
-		encoding=$(echo ${owner_db} | cut -d "," -f 3);
-		echo "========[DB: ${db}]========================[${MODULE}]====================================>";
-		echo "--- [BACKUP: ${db}: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+		owner=$(echo "${owner_db}" | cut -d "," -f 1);
+		db=$(echo "${owner_db}" | cut -d "," -f 2);
+		encoding=$(echo "${owner_db}" | cut -d "," -f 3);
+		printf "${PRINTF_DB_SUB_BLOCK}" "DB" "${db}" "${MODULE}";
+		printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "${db}" "$(date +'%F %T')" "${MODULE}";
 		include=0;
 		if [ -s "${BASE_FOLDER}${INCLUDE_FILE}" ]; then
-			while read incl_db; do
+			while read -r incl_db; do
 				if [ "${db}" = "${incl_db}" ]; then
 					include=1;
 					break;
@@ -186,7 +272,7 @@ else
 		fi;
 		exclude=0;
 		if [ -f "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
-			while read excl_db; do
+			while read -r excl_db; do
 				if [ "${db}" = "${excl_db}" ]; then
 					exclude=1;
 					break;
@@ -194,19 +280,40 @@ else
 			done<"${BASE_FOLDER}${EXCLUDE_FILE}";
 		fi;
 		if [ ${include} -eq 1 ] && [ ${exclude} -eq 0 ]; then
+			PG_DUMP_CMD=("${PG_DUMP[@]}" "-c" "--format=c");
 			# set dump type
-			SCHEMA_ONLY=''; # empty for all
-			schema_flag='data'; # or data
+			schema_flag=''; # schema or data
+			# schema exclude over data exclude, can't have both
 			if [ -s "${BASE_FOLDER}${SCHEMA_ONLY_FILE}" ]; then
-				while read schema_db; do
+				# default is data dump
+				schema_flag='data';
+				while read -r schema_db; do
 					if [ "${db}" = "${schema_db}" ]; then
-						SCHEMA_ONLY='-s';
 						schema_flag='schema';
 						# skip out
 						break;
 					fi;
 				done<"${BASE_FOLDER}${SCHEMA_ONLY_FILE}";
+			elif [ -s "${BASE_FOLDER}${DATA_ONLY_FILE}" ]; then
+				# default to schema, unless in data list
+				schema_flag='schema';
+				while read -r data_db; do
+					if [ "${db}" = "${data_db}" ]; then
+						schema_flag='data';
+						# skip out
+						break;
+					fi;
+				done<"${BASE_FOLDER}${DATA_ONLY_FILE}";
 			fi;
+			# if nothing is set, default to data
+			if [ -z "${schema_flag}" ]; then
+				schema_flag="data";
+			fi;
+			# set schema flag if schmea only is requested
+			if [ $schema_flag = "schema" ]; then
+				PG_DUMP_CMD+=("-s");
+			fi;
+			PG_DUMP_CMD+=("${db}");
 			# Filename
 			# Database.User.Encoding.pgsql|data|schema-Version_Host_Port_YearMonthDay_HourMinute_Counter.Fromat(c).sql
 			FILENAME="${db}.${owner}.${encoding}.${schema_flag}-${DB_VERSION}_${DB_HOST}_${DB_PORT}.c.sql"
@@ -215,18 +322,25 @@ else
 			# backup set:
 			BACKUP_SET_NAME="${ONE_TIME_TAG}${BACKUP_SET_PREFIX}${schema_flag}-${BACKUP_SET}";
 			# borg call
-			BORG_CALL=$(echo "${_BORG_CALL}" | sed -e "s/##FILENAME##/${FILENAME}/" | sed -e "s/##BACKUP_SET##/${BACKUP_SET_NAME}/");
+			BORG_CALL=$(
+				echo "${_BORG_CALL}" |
+				sed -e "s/##FILENAME##/${FILENAME}/" |
+				sed -e "s/##BACKUP_SET##/${BACKUP_SET_NAME}/"
+			);
 			# borg prune
-			BORG_PRUNE=$(echo "${_BORG_PRUNE}" | sed -e "s/##BACKUP_SET_PREFIX##/${BACKUP_SET_PREFIX}/");
+			BORG_PRUNE=$(
+				echo "${_BORG_PRUNE}" |
+				sed -e "s/##BACKUP_SET_PREFIX##/${BACKUP_SET_PREFIX}/"
+			);
 			if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
 				echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";";
-				echo "${PG_DUMP} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} -c ${SCHEMA_ONLY} --format=c ${db} | ${BORG_CALL}";
+				echo "${PG_DUMP_CMD[*]} | ${BORG_CALL}";
 				if [ -z "${ONE_TIME_TAG}" ]; then
 					echo "${BORG_PRUNE}";
 				fi;
 			fi;
 			if [ ${DRYRUN} -eq 0 ]; then
-				${PG_DUMP} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} -c ${SCHEMA_ONLY} --format=c ${db} | ${BORG_CALL};
+				"${PG_DUMP_CMD[@]}" | ${BORG_CALL};
 				_backup_error=$?;
 				if [ $_backup_error -ne 0 ]; then
 					echo "[! $(date +'%F %T')] Backup creation failed for ${db} dump with error code: ${_backup_error}";
@@ -235,19 +349,23 @@ else
 				fi;
 			fi;
 			if [ -z "${ONE_TIME_TAG}" ]; then
-				echo "--- [PRUNE : ${db}: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+				printf "${PRINTF_SUBEXT_BLOCK}" "PRUNE" "${db}" "$(date +'%F %T')" "${MODULE}";
 				echo "Prune repository prefixed ${BACKUP_SET_PREFIX} with keep${KEEP_INFO:1}";
 				${BORG_PRUNE};
 			fi;
 		else
 			echo "- [E] ${db}";
 		fi;
+		DURATION=$(( $(date +'%s') - LOCAL_START ));
+		printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "${db}" "${MODULE}" "$(convert_time ${DURATION})";
 	done;
 fi;
 # run compact at the end if not a dry run
 if [ -z "${ONE_TIME_TAG}" ]; then
 	# if this is borg version >1.2 we need to run compact after prune
-	. "${DIR}/borg.backup.functions.compact.sh";
+	. "${DIR}/borg.backup.functions.compact.sh" "auto";
+	# check in auto mode
+	. "${DIR}/borg.backup.functions.check.sh" "auto";
 fi;

 . "${DIR}/borg.backup.functions.close.sh";
--- a/borg.backup.settings-default
+++ b/borg.backup.settings-default
@@ -13,28 +13,43 @@ TARGET_PORT="";
 TARGET_BORG_PATH="";
 # folder where the backup folder will be created
 TARGET_FOLDER="";
-# the backup file (folder) for this host $(hostname), must end with .borg
+# the backup file (folder) for this backup, avoid dynamic variables in here, must end with .borg
 BACKUP_FILE="";
-# compression settings (empty for none, lz4, zstd, zlib, lzma)
-# level, if empty then default, else number between 0 and 9, or 1 to 22 for zstd
-# default is zstd, 3
+# compression settings (none, lz4, zstd, zlib, lzma)
+# empty is default zstd
 COMPRESSION="";
+# compression level, if empty then default 3, else number between 0 and 9, or 1 to 22 for zstd
 COMPRESSION_LEVEL="";
 # encryption settings:
 # SHA-256: 'none', 'authenticated', 'repokey', 'keyfile'
 # BLAKE2b: 'authenticated-blake2', 'repokey-blake2', 'keyfile-blake2'
-# Note: none or empty does not encrypt
-# Blank passwords allowed for only key (if used, use keyfile)
+# Note: none does not encrypt and is not recommended
+# Default is keyfile
+# Blank passwords allowed for only keyfile, else passwords have to be set via BORG_PASSPHRASE or BORG_PASSCOMMAND
 # See: http://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically
-ENCRYPTION="";
-# force repository check, default is off, set to true for check
-FORCE_CHECK="";
-# default is %Y-%m-%d
+ENCRYPTION=""
+# force repository verify, default is off, set to true for verify on every run
+FORCE_VERIFY="";
+# compact interval, only if using borg 1.2 or higher
+# after how many days to run compress on data
+# default is 1 for run after each prune
+# set to 0 or negative to turn off
+# or any other value for every n days
+COMPACT_INTERVAL="";
+# check interval, if 0 or negative number, no check will ever run
+# if empty fall back to default set
+# if set to 1 then every time the script runs
+# any other value it means ever n days, eg 90 would be every 90 days
+CHECK_INTERVAL="";
+# default is {now:%Y-%m-%d}
 # todays date, if more than one per day add -%H%M for hour/minute
 # it can also be "{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S.%f}"
 BACKUP_SET="";
 # prune times, how many are kept in each time frame
+KEEP_LAST="";
+KEEP_HOURS="";
 KEEP_DAYS="";
 KEEP_WEEKS="";
 KEEP_MONTHS="";
 KEEP_YEARS="";
+KEEP_WITHIN="";
--- a/borg.backup.zabbix.settings-default
+++ b/borg.backup.zabbix.settings-default
@@ -15,3 +15,5 @@ ZABBIX_DATABASE="";
 ZABBIX_CONFIG="";
 # unknown tables, default ignore, is set to true, will force backup them
 ZABBIX_UNKNOWN_TABLES="";
+# overide zabbix db port read from zabbix config, if not set read from zabbix config
+ZABBIX_DB_PORT="";
--- a/borg.backup.zabbix.sh
+++ b/borg.backup.zabbix.sh
@@ -1,20 +1,27 @@
 #!/usr/bin/env bash

+# allow variables in printf format string
+# shellcheck disable=SC2059
+
 # Backup zabbix config and settings only

 MODULE="zabbix"
-MODULE_VERSION="1.1.1";
+MODULE_VERSION="1.1.4";

 DIR="${BASH_SOURCE%/*}"
 if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 # init system
 . "${DIR}/borg.backup.functions.init.sh";

-# init check file
-BACKUP_INIT_CHECK="borg.backup.zabbix.init";
+# init verify, compact and check file
+BACKUP_INIT_FILE="borg.backup.${MODULE}.init";
+BACKUP_COMPACT_FILE="borg.backup.${MODULE}.compact";
+BACKUP_CHECK_FILE="borg.backup.${MODULE}.check";
+# lock file
+BACKUP_LOCK_FILE="borg.backup.${MODULE}.lock";

-# check valid data
-. "${DIR}/borg.backup.functions.check.sh";
+# verify valid data
+. "${DIR}/borg.backup.functions.verify.sh";
 # if info print info and then abort run
 . "${DIR}/borg.backup.functions.info.sh";

@@ -22,7 +29,7 @@ BACKUP_INIT_CHECK="borg.backup.zabbix.init";
 if [ -z "${ZABBIX_DUMP_BIN}" ]; then
 	ZABBIX_DUMP_BIN="/usr/local/bin/zabbix-dump";
 fi;
-if [ ! -z "${ZABBIX_CONFIG}" ] && [ ! -f "${ZABBIX_CONFIG}" ]; then
+if [ -n "${ZABBIX_CONFIG}" ] && [ ! -f "${ZABBIX_CONFIG}" ]; then
 	echo "[! $(date +'%F %T')] Cannot find zabbix config: ${ZABBIX_CONFIG}";
 	. "${DIR}/borg.backup.functions.close.sh" 1;
 	exit 1;
@@ -33,6 +40,10 @@ fi;
 if [ "${ZABBIX_DATABASE}" = "psql" ]; then
 	OPT_ZABBIX_DUMP="-C";
 fi;
+OPT_ZABBIX_DB_PORT="";
+if [ -n "${ZABBIX_DB_PORT}" ]; then
+	OPT_ZABBIX_DB_PORT="-P ${ZABBIX_DB_PORT}";
+fi;
 if [ "${ZABBIX_DATABASE}" != "psql" ] && [ "${ZABBIX_DATABASE}" != "mysql" ]; then
 	echo "[! $(date +'%F %T')] Zabbix dump must have database set to either psql or mysql";
 	. "${DIR}/borg.backup.functions.close.sh" 1;
@@ -44,7 +55,7 @@ if [ ! -f "${ZABBIX_DUMP_BIN}" ]; then
 	exit 1;
 fi;
 # -i (ignore)/ -f (backup)
-if [ ! -z "${ZABBIX_UNKNOWN_TABLES}" ]; then
+if [ -n "${ZABBIX_UNKNOWN_TABLES}" ]; then
 	OPT_ZABBIX_UNKNOWN_TABLES="-f";
 else
 	OPT_ZABBIX_UNKNOWN_TABLES="-i";
@@ -64,21 +75,23 @@ if [ -z "${BACKUP_SET_PREFIX}" ]; then
 	BORG_PRUNE=$(echo "${BORG_PRUNE}" | sed -e 's/-P //');
 fi;

-echo "--- [BACKUP: zabbix settings: $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+printf "${PRINTF_SUB_BLOCK}" "BACKUP: zabbix settings" "$(date +'%F %T')" "${MODULE}";
 if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
-	echo "${ZABBIX_DUMP_BIN} -t ${ZABBIX_DATABASE} ${OPT_ZABBIX_UNKNOWN_TABLES} ${OPT_ZABBIX_DUMP} ${OPT_ZABBIX_CONFIG} -o - | ${BORG_CALL}"
+	echo "${ZABBIX_DUMP_BIN} ${OPT_ZABBIX_DB_PORT} -t ${ZABBIX_DATABASE} ${OPT_ZABBIX_UNKNOWN_TABLES} ${OPT_ZABBIX_DUMP} ${OPT_ZABBIX_CONFIG} -o - | ${BORG_CALL}"
 	if [ -z "${ONE_TIME_TAG}" ]; then
 		echo "${BORG_PRUNE}";
 	fi;
 fi;
 if [ ${DRYRUN} -eq 0 ]; then
-	${ZABBIX_DUMP_BIN} -t ${ZABBIX_DATABASE} ${OPT_ZABBIX_UNKNOWN_TABLES} ${OPT_ZABBIX_DUMP} ${OPT_ZABBIX_CONFIG} -o - | ${BORG_CALL};
+	${ZABBIX_DUMP_BIN} ${OPT_ZABBIX_DB_PORT} -t ${ZABBIX_DATABASE} ${OPT_ZABBIX_UNKNOWN_TABLES} ${OPT_ZABBIX_DUMP} ${OPT_ZABBIX_CONFIG} -o - | ${BORG_CALL};
 fi;
 if [ -z "${ONE_TIME_TAG}" ]; then
-	echo "--- [PRUNE : $(date +'%F %T')] --[${MODULE}]------------------------------------>";
+	printf "${PRINTF_SUB_BLOCK}" "PRUNE" "$(date +'%F %T')" "${MODULE}";
 	${BORG_PRUNE};
 	# if this is borg version >1.2 we need to run compact after prune
-	. "${DIR}/borg.backup.functions.compact.sh";
+	. "${DIR}/borg.backup.functions.compact.sh" "auto";
+	# check in auto mode
+	. "${DIR}/borg.backup.functions.check.sh" "auto";
 fi;

 . "${DIR}/borg.backup.functions.close.sh";
--- a/borg.mount.sh
+++ b/borg.mount.sh
@@ -1,5 +1,8 @@
 #!/usr/bin/env bash

+echo "${0} Currently not checked";
+exit;
+
 set -e -u -o pipefail

 # mount this servers borg backup to a folder
@@ -13,7 +16,7 @@ SETTINGS_FILE="borg.backup.settings";
 # base mount path (default)
 MOUNT_PATH="/mnt/restore/";
 # backup path to borg storage
-ATTIC_BACKUP_FILE='';
+BORG_BACKUP_FILE='';
 # if we are mount or unmount (default is mount)
 UMOUNT=0;

@@ -30,7 +33,7 @@ function usage ()
 }

 # set options
-while getopts ":c:m:uf:h" opt do
+while getopts ":c:m:uf:h" opt; do
 	case "${opt}" in
 		c|config)
 			BASE_FOLDER=${OPTARG};
@@ -42,7 +45,7 @@ while getopts ":c:m:uf:h" opt do
 			UMOUNT=1;
 			;;
 		f|file)
-			ATTIC_BACKUP_FILE=${OPTARG};
+			BORG_BACKUP_FILE=${OPTARG};
 			;;
 		h|help)
 			usage;
@@ -69,7 +72,7 @@ fi;

 if [ ${UMOUNT} -eq 0 ]; then
 	TARGET_SERVER='';
-	if [ -z "${ATTIC_BACKUP_FILE}" ]; then
+	if [ -z "${BORG_BACKUP_FILE}" ]; then
 		if [ ! -f "${BASE_FOLDER}${SETTINGS_FILE}" ]; then
 			echo "Cannot find ${BASE_FOLDER}${SETTINGS_FILE}";
 			exit 0;
@@ -77,17 +80,17 @@ if [ ${UMOUNT} -eq 0 ]; then
 		. ${BASE_FOLDER}${SETTINGS_FILE}
 		# set the borg backup file base on the settings data
 		# if we have user/host then we build the ssh command
-		if [ ! -z "${TARGET_USER}" ] && [ ! -z "${TARGET_HOST}" ]; then
+		if [ -n "${TARGET_USER}" ] && [ -n "${TARGET_HOST}" ]; then
 			TARGET_SERVER=${TARGET_USER}"@"${TARGET_HOST}":";
 		fi;
 		REPOSITORY=${TARGET_SERVER}${TARGET_FOLDER}${BACKUP_FILE};
 	else
-		REPOSITORY=${ATTIC_BACKUP_FILE};
+		REPOSITORY=${BORG_BACKUP_FILE};
 	fi;

 	# check that the repostiory exists
 	REPOSITORY_OK=0;
-	if [ ! -z "${TARGET_SERVER}" ]; then
+	if [ -n "${TARGET_SERVER}" ]; then
 		# remove trailing : for this
 		TARGET_SERVER=${TARGET_SERVER/:};
 		# use ssh command to check remote existense
@@ -112,4 +115,4 @@ else
 	borg umount "${MOUNT_PATH}";
 fi;

-## END
+# __END__
Author	SHA1	Message	Date
Clemens Schwaighofer	03925562ff	Merge branch 'development' into refactor/TTD-3115/shellcheck-cleanup	2025-12-04 11:46:31 +09:00
Clemens Schwaighofer	924014688e	Update gitignore file with backup lock files	2025-12-04 09:57:54 +09:00
Clemens Schwaighofer	f9a58171cc	Bump version to v4.8.0	2025-12-04 09:35:08 +09:00
Clemens Schwaighofer	1b52af909a	Update PostgreSQL module to run as normal user with sudo for PostgreSQL also clean up all PostgreSQL command calls as array type calls clean up binary finder for postgresql installations and add port/host override options	2025-12-04 09:28:57 +09:00
Clemens Schwaighofer	32a75626bb	Version number fixes	2025-12-03 15:49:45 +09:00
Clemens Schwaighofer	12ff648504	PostgreSQL backup hat wrong table selection	2025-12-03 15:31:55 +09:00
Clemens Schwaighofer	69be76e34c	Default settings file update for BACKUP_FILE	2025-12-03 14:48:54 +09:00
Clemens Schwaighofer	4a95049d4f	Default config file info update	2025-12-03 14:12:24 +09:00
Clemens Schwaighofer	d0e96a82e0	Clean up PostgreSQL call, clean up varify error block, ERR trap fixes ERR trap is now returning last error so $? can bet used Verify catches $? error now for INIT repository check	2025-12-03 14:00:49 +09:00
Clemens Schwaighofer	e5c5df6013	Fix error trap with return error to STD ERR	2025-12-03 13:08:35 +09:00
Clemens Schwaighofer	6023fac636	Fix trap clanup that also trapped ERR	2025-12-03 13:05:11 +09:00
Clemens Schwaighofer	9e0db8bae9	Fix the error trap with exit to stop calling it again with itself	2025-12-03 13:03:46 +09:00
Clemens Schwaighofer	8a8de773df	Change borg repository verify error message collection	2025-12-03 12:47:49 +09:00
Clemens Schwaighofer	f41dd1b723	Change verify command error check	2025-12-03 12:45:56 +09:00
Clemens Schwaighofer	c5d88a64e1	Fix last command check for verify	2025-12-03 12:43:52 +09:00
Clemens Schwaighofer	943d1c551e	Stop trapping errors und unset enviromant vars On error just print error, but do not reset env vars, if this is done on verify error the init afterwards will store all settings in the wrong path	2025-12-03 12:38:01 +09:00
Clemens Schwaighofer	783bf2adb9	Merge branch 'development' into shellcheck-cleanup	2025-12-02 10:02:17 +09:00
Clemens Schwaighofer	45e1e29d22	Update readme file with info about encryption	2025-12-02 10:01:37 +09:00
Clemens Schwaighofer	dd2a85f5d6	Merge branch 'development' into shellcheck-cleanup	2025-12-01 18:16:01 +09:00
Clemens Schwaighofer	1f8d38720c	On info if verbose is set, print out the key too	2025-12-01 17:05:45 +09:00
Clemens Schwaighofer	7e4dc8d500	Clean up bash shell code and add proper ENCRYPTION support - check for invalid encryption setting - switch from none to keyfile type (without password) - error for no password for repokey or authentication type encryption - update init and print out the key file data	2025-12-01 16:53:50 +09:00
Clemens Schwaighofer	297c745df7	Borg MySQL backup: fix version detection for MariaDB variant This one is from Debian 13	2025-09-18 09:58:51 +09:00
Clemens Schwaighofer	2e5f0648f1	Merge branch 'development' into shellcheck-cleanup	2025-08-21 10:42:20 +09:00
Clemens Schwaighofer	a2bf6b3331	Readme update for zabbix backup settings change	2025-08-21 10:40:56 +09:00
Clemens Schwaighofer	acd5d369eb	Merge branch 'development' into shellcheck-cleanup	2025-08-20 22:41:54 +09:00
Clemens Schwaighofer	07896af1ae	Zabbix port override settings fix use OPT var prefix for options add missing init in functions init	2025-08-20 22:37:37 +09:00
Clemens Schwaighofer	8de971b922	Fix Zabbix db port override settings	2025-08-20 22:16:49 +09:00
Clemens Schwaighofer	e7699015c5	Allow override of zabbix db port	2025-08-20 22:14:19 +09:00
Clemens Schwaighofer	de9d5d3588	Zabbix backup set port to 5432 default to override internal set ports	2025-08-20 22:11:38 +09:00
Clemens Schwaighofer	8407defe4f	Starting shellcheck cleanup	2024-09-04 10:58:48 +09:00
Clemens Schwaighofer	28dd6cb911	Merge branch 'master' into development	2024-09-04 10:38:58 +09:00
Clemens Schwaighofer	ab47692848	borg backup default settings file sample update	2024-09-04 10:37:36 +09:00
Clemens Schwaighofer	4267cb5b32	Bug Fix in gitea backup for temp dir check	2024-09-04 10:37:16 +09:00
Clemens Schwaighofer	f24daba195	Add shellcheckrc	2024-09-03 12:40:39 +09:00
Clemens Schwaighofer	98890b87f5	gitea module fix chown call do not use ".", use ":" for username and group separator	2024-06-11 10:33:47 +09:00
Clemens Schwaighofer	a5bc90bb89	Text fix in PostgreSQL backup module	2024-06-10 09:52:06 +09:00
Clemens Schwaighofer	5af6a84d5f	Mysql/PostgreSQL module version string update	2024-06-07 15:08:49 +09:00
Clemens Schwaighofer	14d71f3053	Naming update for full update info text	2024-06-07 15:08:20 +09:00
Clemens Schwaighofer	24d4fb694f	Typo in postgresql module for full dump filename	2024-06-07 14:25:00 +09:00
Clemens Schwaighofer	f9fbbcc6de	Version fix for master script	2024-06-07 09:57:21 +09:00
Clemens Schwaighofer	64e876a9a4	Fix run time output template string	2024-06-07 09:56:51 +09:00
Clemens Schwaighofer	694e54ca9c	Missed out duration calculation in postgresql run time output	2024-06-07 09:16:52 +09:00
Clemens Schwaighofer	09bd6015ac	Version update for modules gitea, mysql, postgresql	2024-06-07 09:12:04 +09:00
Clemens Schwaighofer	5b8b0afec2	Fix not initialized new gitea variable names	2024-06-07 09:09:02 +09:00
Clemens Schwaighofer	0cc7e574fe	Borg backup update for gitea dump, mysql/postgresql info block Gitea dump has now settings for - temp folder (GITEA_TEMP_DIR) - new name for working dir (GITEA_WORKING_DIR) from old (GITEA_TMP) MySQL and PostgreSQL exports have a run time output per db dump set and not only at the end of the full run	2024-06-06 10:30:48 +09:00
Clemens Schwaighofer	beffbee20c	Add missing OPT_REMOTE to commands check, compact commands had OPT_REMOTE missing	2023-05-10 16:30:23 +09:00
Clemens Schwaighofer	72f3f86eb6	Test for different command lookup check	2022-11-22 09:36:30 +09:00
Clemens Schwaighofer	c0f9634442	add *.compact to git ignore file	2022-11-14 20:27:42 +09:00
Clemens Schwaighofer	0e379d6ce0	Bump borg file module version number	2022-08-30 10:00:05 +09:00
Clemens Schwaighofer	bba8be6a42	Exclude File check is if file is >0 and fix wrong check for tmp file Exclude file was only checked if it exists, even if it was empty it would start processing. In the exclude check there was also a wrong check for the tmp exclude file, it actually checked if the main exclude file was >0 instead of the tmp file.	2022-08-30 09:56:20 +09:00
Clemens Schwaighofer	ce3cdc7627	Readme update to fix Marktdown lint issues	2022-08-08 14:51:26 +09:00
Clemens Schwaighofer	67c0b00268	Initialize LAST COMPACT/CHECK date variable	2022-06-07 19:57:36 +09:00
Clemens Schwaighofer	abbcfa883f	Add compact auto run in intervals like check archive Same settings like check archive in days. Auto set to 1 to run after each prune. Can be controlled per module.	2022-06-06 14:58:36 +09:00
Clemens Schwaighofer	9739436589	Redirect STDERR output from borg backup to STDIN for ANSI code cleanup gitea dump prints out info/status messages to STDERR. The whole subprocess then returns this on STDERR and so is not parsed through the sed regex command. Redirect STDERR to STDIN	2022-05-31 09:54:56 +09:00
Clemens Schwaighofer	bf0364a0e7	Move lock file clean up to before any exit info is printed Just try to clean lock file always on exit	2022-04-25 08:57:01 +09:00
Clemens Schwaighofer	8e87503528	Add auto check functionality CHECK_INTERVAL or SUB_CHECK_INTERVAL in module settings. If set to 1, will check everytime. Ever other number is for n days after last check. FORCE_CHECK for check if repository has been setup os now renamed FORCE_VERIFY but FORCE_CHECK is currently still honored but deprecated all function shell scripts will abort if run on their own Print info on last check time if set in info block Internal updates: All separator lines are now printf strings for central control. All module used config/settings file names use $MODULE name Check functionality is in its own file	2022-04-25 06:49:57 +09:00
Clemens Schwaighofer	34277483e9	Borg check -p (progress) is only set if -v (verbose) flag is set Avoid any progress output if run from script	2022-04-18 16:00:41 +09:00
Clemens Schwaighofer	dffb7c6450	Fix borg check prefix settings	2022-04-18 15:58:03 +09:00
Clemens Schwaighofer	2ae05f5302	Add borg check and rename options Because we added borg check functionality, some of the Options have been renamed: -C -> -V -E -> -e (as it is a sub) -C: check -y: --verify-data -p: prefix or glob for check Internal variables with CHECK have been renamed or changed to VERIFY Borg -C without any extra parameters is equal to borg check. -y adds the --verify-data and -p is a mix of the -P and -a options. If there is a "*" in the option then -a will be used, else -P Note that repair command has to be run manually. Run -C with -v (verbose) to see the repair command structure. borg check can take a long time on very large repositories.	2022-04-18 15:41:28 +09:00
Clemens Schwaighofer	b5ead9a2e1	Show host name and module init date in the start info block	2022-04-01 11:16:38 +09:00
Clemens Schwaighofer	fa77876440	Update Version ID to 4.2.4	2022-03-31 11:09:11 +09:00
Clemens Schwaighofer	f128b7ebc4	Bug fix for no keep options set check	2022-03-31 11:01:02 +09:00
Clemens Schwaighofer	77977207c8	Change close parameter check to pure string type	2022-03-31 09:21:23 +09:00
Clemens Schwaighofer	c1f6bb443a	Fix for close with empty parameter	2022-03-31 09:18:23 +09:00
Clemens Schwaighofer	86b0fa122a	Add empty KEEP option for error with -T	2022-03-30 20:27:46 +09:00
Clemens Schwaighofer	32c320be27	Fix close call int compare	2022-03-30 14:48:52 +09:00
Clemens Schwaighofer	500ab01790	Bug fix for unset var in close call	2022-03-30 11:35:19 +09:00
Clemens Schwaighofer	ab58ab3ad0	Update postgresql module version	2022-03-30 09:48:37 +09:00
Clemens Schwaighofer	7767eb58df	Override sudo for postgresql in upgrade script	2022-03-30 09:32:57 +09:00
Clemens Schwaighofer	38f467de96	PostgreSQL backup add schema or data dump in either default direction	2022-03-29 11:21:06 +09:00
Clemens Schwaighofer	8f91690f6a	Upgrade script for zabbix fix if no zabbix prefix set at all	2022-03-29 10:53:59 +09:00
Clemens Schwaighofer	e860573e0c	Fix upgrade script gitea module rename check	2022-03-29 10:28:26 +09:00
Clemens Schwaighofer	f990e86949	update script gitea block fix for missing module name	2022-03-29 10:25:12 +09:00
Clemens Schwaighofer	c929987900	variable name wrong for module check in upgrade script	2022-03-29 10:20:52 +09:00
Clemens Schwaighofer	6cb941818c	Repository ID via config only works with local repos, drop that and keep repo ID in -i option only	2022-03-29 10:01:00 +09:00
Clemens Schwaighofer	edaf41f1af	Repository ID info, borg command call via variable The main info block gets Repository ID as info too (for cache/etc folders check) Missing borg command as variable for borg list command.	2022-03-29 09:57:12 +09:00
Clemens Schwaighofer	c7f2197614	add -E to sudo for upgrade script to keep enviroment variables for postgres run	2022-03-29 09:16:03 +09:00
Clemens Schwaighofer	88ea600e1d	Mising remote path option in update script	2022-03-29 08:55:19 +09:00
Clemens Schwaighofer	f396032728	Add remote override path for borg backup in update script	2022-03-29 08:53:37 +09:00
Clemens Schwaighofer	18cbcea2b0	Archive print page make archive space 45 chars wide	2022-03-29 07:01:31 +09:00
Clemens Schwaighofer	a0537a24d3	PostgreSQL sudo for upgrade script	2022-03-29 06:46:23 +09:00
Clemens Schwaighofer	89897eb676	Add base BORG export variables in upgrade script	2022-03-29 06:36:06 +09:00