Compare commits

...

38 Commits

Author SHA1 Message Date
Clemens Schwaighofer
32a75626bb Version number fixes 2025-12-03 15:49:45 +09:00
Clemens Schwaighofer
12ff648504 PostgreSQL backup hat wrong table selection 2025-12-03 15:31:55 +09:00
Clemens Schwaighofer
69be76e34c Default settings file update for BACKUP_FILE 2025-12-03 14:48:54 +09:00
Clemens Schwaighofer
4a95049d4f Default config file info update 2025-12-03 14:12:24 +09:00
Clemens Schwaighofer
d0e96a82e0 Clean up PostgreSQL call, clean up varify error block, ERR trap fixes
ERR trap is now returning last error so $? can bet used
Verify catches $? error now for INIT repository check
2025-12-03 14:00:49 +09:00
Clemens Schwaighofer
e5c5df6013 Fix error trap with return error to STD ERR 2025-12-03 13:08:35 +09:00
Clemens Schwaighofer
6023fac636 Fix trap clanup that also trapped ERR 2025-12-03 13:05:11 +09:00
Clemens Schwaighofer
9e0db8bae9 Fix the error trap with exit to stop calling it again with itself 2025-12-03 13:03:46 +09:00
Clemens Schwaighofer
8a8de773df Change borg repository verify error message collection 2025-12-03 12:47:49 +09:00
Clemens Schwaighofer
f41dd1b723 Change verify command error check 2025-12-03 12:45:56 +09:00
Clemens Schwaighofer
c5d88a64e1 Fix last command check for verify 2025-12-03 12:43:52 +09:00
Clemens Schwaighofer
943d1c551e Stop trapping errors und unset enviromant vars
On error just print error, but do not reset env vars, if this is done on verify error the init afterwards will store all settings in the wrong path
2025-12-03 12:38:01 +09:00
Clemens Schwaighofer
45e1e29d22 Update readme file with info about encryption 2025-12-02 10:01:37 +09:00
Clemens Schwaighofer
1f8d38720c On info if verbose is set, print out the key too 2025-12-01 17:05:45 +09:00
Clemens Schwaighofer
7e4dc8d500 Clean up bash shell code and add proper ENCRYPTION support
- check for invalid encryption setting
- switch from none to keyfile type (without password)
- error for no password for repokey or authentication type encryption
- update init and print out the key file data
2025-12-01 16:53:50 +09:00
Clemens Schwaighofer
297c745df7 Borg MySQL backup: fix version detection for MariaDB variant
This one is from Debian 13
2025-09-18 09:58:51 +09:00
Clemens Schwaighofer
a2bf6b3331 Readme update for zabbix backup settings change 2025-08-21 10:40:56 +09:00
Clemens Schwaighofer
07896af1ae Zabbix port override settings fix
use OPT var prefix for options
add missing init in functions init
2025-08-20 22:37:37 +09:00
Clemens Schwaighofer
8de971b922 Fix Zabbix db port override settings 2025-08-20 22:16:49 +09:00
Clemens Schwaighofer
e7699015c5 Allow override of zabbix db port 2025-08-20 22:14:19 +09:00
Clemens Schwaighofer
28dd6cb911 Merge branch 'master' into development 2024-09-04 10:38:58 +09:00
Clemens Schwaighofer
ab47692848 borg backup default settings file sample update 2024-09-04 10:37:36 +09:00
Clemens Schwaighofer
4267cb5b32 Bug Fix in gitea backup for temp dir check 2024-09-04 10:37:16 +09:00
Clemens Schwaighofer
f24daba195 Add shellcheckrc 2024-09-03 12:40:39 +09:00
Clemens Schwaighofer
98890b87f5 gitea module fix chown call
do not use ".", use ":" for username and group separator
2024-06-11 10:33:47 +09:00
Clemens Schwaighofer
a5bc90bb89 Text fix in PostgreSQL backup module 2024-06-10 09:52:06 +09:00
Clemens Schwaighofer
5af6a84d5f Mysql/PostgreSQL module version string update 2024-06-07 15:08:49 +09:00
Clemens Schwaighofer
14d71f3053 Naming update for full update info text 2024-06-07 15:08:20 +09:00
Clemens Schwaighofer
24d4fb694f Typo in postgresql module for full dump filename 2024-06-07 14:25:00 +09:00
Clemens Schwaighofer
f9fbbcc6de Version fix for master script 2024-06-07 09:57:21 +09:00
Clemens Schwaighofer
64e876a9a4 Fix run time output template string 2024-06-07 09:56:51 +09:00
Clemens Schwaighofer
694e54ca9c Missed out duration calculation in postgresql run time output 2024-06-07 09:16:52 +09:00
Clemens Schwaighofer
09bd6015ac Version update for modules gitea, mysql, postgresql 2024-06-07 09:12:04 +09:00
Clemens Schwaighofer
5b8b0afec2 Fix not initialized new gitea variable names 2024-06-07 09:09:02 +09:00
Clemens Schwaighofer
0cc7e574fe Borg backup update for gitea dump, mysql/postgresql info block
Gitea dump has now settings for
- temp folder (GITEA_TEMP_DIR)
- new name for working dir (GITEA_WORKING_DIR) from old (GITEA_TMP)

MySQL and PostgreSQL exports have a run time output per db dump set and not only at the end of the full run
2024-06-06 10:30:48 +09:00
Clemens Schwaighofer
beffbee20c Add missing OPT_REMOTE to commands
check, compact commands had OPT_REMOTE missing
2023-05-10 16:30:23 +09:00
Clemens Schwaighofer
72f3f86eb6 Test for different command lookup check 2022-11-22 09:36:30 +09:00
Clemens Schwaighofer
c0f9634442 add *.compact to git ignore file 2022-11-14 20:27:42 +09:00
19 changed files with 355 additions and 228 deletions

1
.gitignore vendored
View File

@@ -6,3 +6,4 @@ borg.backup.*.include
borg.backup.*.exclude
borg.backup.*.schema-only
borg.backup.*.init
borg.backup.*.compact

2
.shellcheckrc Normal file
View File

@@ -0,0 +1,2 @@
shell=bash
external-sources=true

View File

@@ -141,7 +141,7 @@ All below have default values if not set in the main settings file
* COMPRESSION: zstd
* COMPRESSION_LEVEL: 3
* ENCRYPTION: none
* ENCRYPTION: keyfile (with no password)
* FORCE_VERIFY: false
* COMPACT_INTERVAL: 1
* CHECK_INTERVAL: none
@@ -220,10 +220,10 @@ The postgres user must be added to the backup group for this, so that the basic
### PostgreSQL Config variables
Variable | Default | Description
| Variable | Default | Description |
| - | - | - |
DATABASE_FULL_DUMP | | if empty, dump per databse, if set dump all in one file, if set to schema dump only schema
DATABASE_USER | | overide username to connect to database
| DATABASE_FULL_DUMP | | if empty, dump per databse, if set dump all in one file, if set to schema dump only schema |
| DATABASE_USER | | overide username to connect to database |
### PostgreSQL Control files
@@ -239,10 +239,10 @@ If non root ident authentication run is used, be sure that the `mysql` user is i
### MySQL Config variables
Variable | Default | Description
| Variable | Default | Description |
| - | - | - |
DATABASE_FULL_DUMP | | if empty, dump per databse, if set dump all in one file, if set to schema dump only schema
MYSQL_DB_CONFIG | | override file for connection. In modern mariaDB installations it is rcommended to run the script as root or mysql user and use the ident authentication instead.
| DATABASE_FULL_DUMP | | if empty, dump per databse, if set dump all in one file, if set to schema dump only schema |
| MYSQL_DB_CONFIG | | override file for connection. In modern mariaDB installations it is rcommended to run the script as root or mysql user and use the ident authentication instead. |
### MySQLControl files
@@ -259,12 +259,13 @@ This user is neede to create the temporary dump folder and access for the git fi
### gitea Config Variables
Variable | Default | Description
| Variable | Default | Description |
| - | - | - |
GIT_USER | git | The user that runs gitea |
GITEA_TMP | /tmp/gitea/ | Where the temporary dump files from the backup are stored, as user git |
GITEA_BIN | /usr/local/bin/gitea | Where the gitea binary is located |
GITEA_CONFIG | /etc/gitea/app.ini | The configuration file for gitea |
| GIT_USER | git | The user that runs gitea |
| GITEA_WORKING_DIR | /var/tmp/gitea/ | Where the temporary dump files from the backup are stored, as user git |
| GITEA_TEMP_DIR | /var/tmp/ | General temporary folder |
| GITEA_BIN | /usr/local/bin/gitea | Where the gitea binary is located |
| GITEA_CONFIG | /etc/gitea/app.ini | The configuration file for gitea |
### gitea Control files
@@ -272,14 +273,17 @@ There are no control files for gitea backup
## zabbix config backup settings
The `zabbix-dump` dump script must be installed from: <https://github.com/gullevek/zabbix-backup>
### zabbix Config Variables
Variable | Default | Description
| Variable | Default | Description |
| - | - | - |
ZABBIX_DUMP | /usr/local/bin/zabbix-dump |
ZABBIX_DATABASE | '' | Must be set as either psql or mysql
ZABBIX_CONFIG | '' | if not set uses default location
ZABBIX_UNKNOWN_TABLES | '' | if set, changed to -f (force)
| ZABBIX_DUMP | /usr/local/bin/zabbix-dump | |
| ZABBIX_DATABASE | '' | Must be set as either psql or mysql |
| ZABBIX_CONFIG | '' | if not set uses default location |
| ZABBIX_UNKNOWN_TABLES | '' | if set, changed to -f (force) |
| ZABBIX_DB_PORT | '' | if set, sets -P parameter on zabbix backup script to override auto get from zabbix server config |
### zabbix Control files

View File

@@ -66,7 +66,7 @@ if [ ! -f "${BASE_FOLDER}${SETTINGS_FILE}" ]; then
fi;
. "${BASE_FOLDER}${SETTINGS_FILE}";
if [ ! -z "${TARGET_BORG_PATH}" ]; then
if [ -n "${TARGET_BORG_PATH}" ]; then
OPT_REMOTE="--remote-path="$(printf "%q" "${TARGET_BORG_PATH}");
fi;
export BORG_BASE_DIR="${BASE_FOLDER}";
@@ -82,16 +82,16 @@ for MODULE in ${MODULE_LIST}; do
TARGET_FOLDER=${TARGET_FOLDER#/}
# and add slash front and back and escape the path
TARGET_FOLDER=$(printf "%q" "/${TARGET_FOLDER}/");
if [ ! -z "${TARGET_USER}" ] && [ ! -z "${TARGET_HOST}" ] && [ ! -z "${TARGET_PORT}" ]; then
if [ -n "${TARGET_USER}" ] && [ -n "${TARGET_HOST}" ] && [ -n "${TARGET_PORT}" ]; then
TARGET_SERVER="ssh://${TARGET_USER}@${TARGET_HOST}:${TARGET_PORT}/";
# host/port
elif [ ! -z "${TARGET_HOST}" ] && [ ! -z "${TARGET_PORT}" ]; then
elif [ -n "${TARGET_HOST}" ] && [ -n "${TARGET_PORT}" ]; then
TARGET_SERVER="ssh://${TARGET_HOST}:${TARGET_PORT}/";
# user/host
elif [ ! -z "${TARGET_USER}" ] && [ ! -z "${TARGET_HOST}" ]; then
elif [ -n "${TARGET_USER}" ] && [ -n "${TARGET_HOST}" ]; then
TARGET_SERVER="${TARGET_USER}@${TARGET_HOST}:";
# host
elif [ ! -z "${TARGET_HOST}" ]; then
elif [ -n "${TARGET_HOST}" ]; then
TARGET_SERVER="${TARGET_HOST}:";
fi;
# we dont allow special characters, so we don't need to special escape it
@@ -112,14 +112,14 @@ for MODULE in ${MODULE_LIST}; do
if [ "${MODULE}" = "gitea" ]; then
# if just date, add gitea,
# else rename
if [ ! -z "${i##gitea*}" ]; then
if [ -n "${i##gitea*}" ]; then
target_name="${MODULE},${i}";
else
target_name=$(echo $i | sed -e 's/gitea-/gitea,/');
fi;
elif [ "${MODULE}" = "zabbix" ]; then
# if zabbix is missing, prefix
if [ ! -z "${i##zabbix*}" ]; then
if [ -n "${i##zabbix*}" ]; then
target_name="${MODULE},${i}";
else
target_name=$(echo $i | sed -e 's/zabbix-settings-/zabbix,settings-/');

View File

@@ -1,5 +1,8 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
# Plain file backup
# set last edit date + time
@@ -35,7 +38,7 @@ FOLDERS=();
# this if for debug output with quoted folders
FOLDERS_Q=();
# include list
while read include_folder; do
while read -r include_folder; do
# strip any leading spaces from that folder
include_folder=$(echo "${include_folder}" | sed -e 's/^[ \t]*//');
# check that those folders exist, warn on error,
@@ -45,7 +48,7 @@ while read include_folder; do
echo "# [C] Comment: '${include_folder}'";
else
# skip if it is empty
if [ ! -z "${include_folder}" ]; then
if [ -n "${include_folder}" ]; then
# if this is a glob, do a double check that the base folder actually exists (?)
if [[ "${include_folder}" =~ $REGEX_GLOB ]]; then
# if this is */ then allow it
@@ -99,7 +102,7 @@ if [ -s "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
# remove non valid ones and warn
#TMP_EXCLUDE_FILE=$(mktemp --tmpdir ${EXCLUDE_FILE}.XXXXXXXX); # non mac
TMP_EXCLUDE_FILE=$(mktemp "${TEMPDIR}${EXCLUDE_FILE}".XXXXXXXX);
while read exclude_folder; do
while read -r exclude_folder; do
# strip any leading spaces from that folder
exclude_folder=$(echo "${exclude_folder}" | sed -e 's/^[ \t]*//');
# folder or any type of file is ok
@@ -108,10 +111,10 @@ if [ -s "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
echo "# [C] Comment: '${exclude_folder}'";
else
# skip if it is empty
if [ ! -z "${exclude_folder}" ]; then
if [ -n "${exclude_folder}" ]; then
# if it DOES NOT start with a / we assume free folder and add as is
if [[ "${exclude_folder}" != /* ]]; then
echo "${exclude_folder}" >> ${TMP_EXCLUDE_FILE};
echo "${exclude_folder}" >> "${TMP_EXCLUDE_FILE}";
echo "+ [E] General exclude: '${exclude_folder}'";
# if this is a glob, do a double check that the base folder actually exists (?)
elif [[ "${exclude_folder}" =~ $REGEX_GLOB ]]; then
@@ -121,7 +124,7 @@ if [ -s "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
if [ ! -d "${_exclude_folder}" ]; then
echo "- [E] Exclude folder with glob '${exclude_folder}' does not exist or is not accessable";
else
echo "${exclude_folder}" >> ${TMP_EXCLUDE_FILE};
echo "${exclude_folder}" >> "${TMP_EXCLUDE_FILE}";
echo "+ [E] Exclude folder with glob '${exclude_folder}'";
fi;
# do a warning for a possible invalid folder
@@ -129,7 +132,7 @@ if [ -s "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
elif [ ! -d "${exclude_folder}" ] && [ ! -e "${exclude_folder}" ]; then
echo "- [E] Exclude folder or file '${exclude_folder}' does not exist or is not accessable";
else
echo "${exclude_folder}" >> ${TMP_EXCLUDE_FILE};
echo "${exclude_folder}" >> "${TMP_EXCLUDE_FILE}";
# if it is a folder, remove the last / or the symlink check will not work
if [ -d "${exclude_folder}" ]; then
_exclude_folder=${exclude_folder%/*};

View File

@@ -1,5 +1,8 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
if [ -z "${MODULE}" ]; then
echo "Script cannot be run on its own";
exit 1;
@@ -18,13 +21,13 @@ if [ $# -ge 1 ] && [ "$1" = "auto" ]; then
# get current date timestmap
CURRENT_DATE=$(date +%s);
# if =1 always ok
if [ ${CHECK_INTERVAL} -eq 1 ]; then
if [ "${CHECK_INTERVAL}" -eq 1 ]; then
RUN_CHECK=1;
# always add verify data for automatic check
OPT_CHECK_VERIFY_DATA="--verify-data";
# set new check time here
echo ${CURRENT_DATE} > "${BASE_FOLDER}${BACKUP_CHECK_FILE}";
elif [ ${CHECK_INTERVAL} -gt 1 ]; then
echo "${CURRENT_DATE}" > "${BASE_FOLDER}${BACKUP_CHECK_FILE}";
elif [ "${CHECK_INTERVAL}" -gt 1 ]; then
# else load last timestamp and check if today - last time stamp > days
if [ -z "${LAST_CHECK_DATE}" ]; then
LAST_CHECK_DATE=$(cat "${BASE_FOLDER}${BACKUP_CHECK_FILE}" 2>/dev/null | sed -e 's/ //g');
@@ -34,19 +37,19 @@ if [ $# -ge 1 ] && [ "$1" = "auto" ]; then
LAST_CHECK_DATE=0;
fi;
# if the difference greate than check date, run. CHECK INTERVAL is in days
if [ $(($CURRENT_DATE-$LAST_CHECK_DATE)) -ge $((${CHECK_INTERVAL}*86400)) ]; then
if [ $((CURRENT_DATE - LAST_CHECK_DATE)) -ge $((CHECK_INTERVAL * 86400)) ]; then
RUN_CHECK=1;
# always add verify data for automatic check
OPT_CHECK_VERIFY_DATA="--verify-data";
# set new check time here
echo ${CURRENT_DATE} > "${BASE_FOLDER}${BACKUP_CHECK_FILE}";
echo "${CURRENT_DATE}" > "${BASE_FOLDER}${BACKUP_CHECK_FILE}";
fi;
fi;
elif [ ${CHECK} -eq 1 ]; then
elif [ "${CHECK}" -eq 1 ]; then
RUN_CHECK=1;
fi;
if [ ${RUN_CHECK} -eq 1 ]; then
if [ "${RUN_CHECK}" -eq 1 ]; then
# run borg check command
IFS=${_IFS};
printf "${PRINTF_SUB_BLOCK}" "CHECK" "$(date +'%F %T')" "${MODULE}";
@@ -54,28 +57,28 @@ if [ ${RUN_CHECK} -eq 1 ]; then
OPT_GLOB="";
if [[ "${CHECK_PREFIX}" =~ $REGEX_GLOB ]]; then
OPT_GLOB="-a '${CHECK_PREFIX}'"
elif [ ! -z "${CHECK_PREFIX}" ]; then
elif [ -n "${CHECK_PREFIX}" ]; then
OPT_GLOB="-P ${CHECK_PREFIX}";
fi;
# debug/dryrun
if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} check ${OPT_PROGRESS} ${OPT_CHECK_VERIFY_DATA} ${OPT_GLOB} ${REPOSITORY}";
if [ "${DEBUG}" -eq 1 ] || [ "${DRYRUN}" -eq 1 ]; then
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} check ${OPT_REMOTE} ${OPT_PROGRESS} ${OPT_CHECK_VERIFY_DATA} ${OPT_GLOB} ${REPOSITORY}";
fi;
# run info command if not a dry drun
if [ ${DRYRUN} -eq 0 ]; then
if [ "${DRYRUN}" -eq 0 ]; then
# if glob add glob command directly
if [[ "${CHECK_PREFIX}" =~ $REGEX_GLOB ]]; then
${BORG_COMMAND} check ${OPT_PROGRESS} ${OPT_CHECK_VERIFY_DATA} -a "${CHECK_PREFIX}" ${REPOSITORY};
${BORG_COMMAND} check ${OPT_REMOTE} ${OPT_PROGRESS} ${OPT_CHECK_VERIFY_DATA} -a "${CHECK_PREFIX}" "${REPOSITORY}";
else
${BORG_COMMAND} check ${OPT_PROGRESS} ${OPT_CHECK_VERIFY_DATA} ${OPT_GLOB} ${REPOSITORY};
${BORG_COMMAND} check ${OPT_REMOTE} ${OPT_PROGRESS} ${OPT_CHECK_VERIFY_DATA} ${OPT_GLOB} "${REPOSITORY}";
fi;
fi;
# print additional info for use --repair command
# but only for manual checks
if [ ${VERBOSE} -eq 1 ] && [ ${CHECK} -eq 1 ]; then
if [ "${VERBOSE}" -eq 1 ] && [ "${CHECK}" -eq 1 ]; then
echo "";
echo "In case of needed repair: "
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} check ${OPT_PROGRESS} --repair ${OPT_GLOB} ${REPOSITORY}";
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} check ${OPT_REMOTE} ${OPT_PROGRESS} --repair ${OPT_GLOB} ${REPOSITORY}";
echo "Before running repair, a copy from the backup should be made because repair might damage a backup"
fi;
fi;

View File

@@ -1,5 +1,8 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
if [ -z "${MODULE}" ]; then
echo "Script cannot be run on its own";
exit 1;
@@ -16,7 +19,7 @@ if [ $# -ge 1 ] && [ "$1" = "1" ]; then
printf "${PRINTF_MASTER_BLOCK}" "ERROR" "$(date +'%F %T')" "${MODULE}";
else
# running time calculation
DURATION=$[ $(date +'%s')-$START ];
DURATION=$(( $(date +'%s') - START ));
echo "=== [Run time: $(convert_time ${DURATION})]";
printf "${PRINTF_MASTER_BLOCK}" "END" "$(date +'%F %T')" "${MODULE}";
fi;

View File

@@ -1,5 +1,8 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
if [ -z "${MODULE}" ]; then
echo "Script cannot be run on its own";
exit 1;
@@ -7,7 +10,7 @@ fi;
# compact (only if BORG COMPACT is set)
# only for borg 1.2
if [ $(version $BORG_VERSION) -ge $(version "1.2.0") ]; then
if [ "$(version "$BORG_VERSION")" -ge "$(version "1.2.0")" ]; then
RUN_COMPACT=0;
if [ $# -ge 1 ] && [ "$1" = "auto" ]; then
# strip any spaces and convert to int
@@ -18,11 +21,11 @@ if [ $(version $BORG_VERSION) -ge $(version "1.2.0") ]; then
fi;
# get current date timestmap
CURRENT_DATE=$(date +%s);
if [ ${COMPACT_INTERVAL} -eq 1 ]; then
if [ "${COMPACT_INTERVAL}" -eq 1 ]; then
RUN_COMPACT=1;
# set new compact time here
echo ${CURRENT_DATE} > "${BASE_FOLDER}${BACKUP_COMPACT_FILE}";
elif [ ${COMPACT_INTERVAL} -gt 1 ]; then
echo "${CURRENT_DATE}" > "${BASE_FOLDER}${BACKUP_COMPACT_FILE}";
elif [ "${COMPACT_INTERVAL}" -gt 1 ]; then
# else load last timestamp and check if today - last time stamp > days
if [ -z "${LAST_COMPACT_DATE}" ]; then
LAST_COMPACT_DATE=$(cat "${BASE_FOLDER}${BACKUP_COMPACT_FILE}" 2>/dev/null | sed -e 's/ //g');
@@ -32,25 +35,25 @@ if [ $(version $BORG_VERSION) -ge $(version "1.2.0") ]; then
LAST_COMPACT_DATE=0;
fi;
# if the difference greate than compact date, run. COMPACT INTERVAL is in days
if [ $(($CURRENT_DATE-$LAST_COMPACT_DATE)) -ge $((${COMPACT_INTERVAL}*86400)) ]; then
if [ $((CURRENT_DATE - LAST_COMPACT_DATE)) -ge $((COMPACT_INTERVAL * 86400)) ]; then
RUN_COMPACT=1;
# set new compact time here
echo ${CURRENT_DATE} > "${BASE_FOLDER}${BACKUP_COMPACT_FILE}";
echo "${CURRENT_DATE}" > "${BASE_FOLDER}${BACKUP_COMPACT_FILE}";
fi;
fi;
elif [ ${COMPACT} -eq 1 ]; then
elif [ "${COMPACT}" -eq 1 ]; then
RUN_COMPACT=1;
fi;
if [ ${RUN_COMPACT} -eq 1 ]; then
if [ "${RUN_COMPACT}" -eq 1 ]; then
# reset to normal IFS, so command works here
IFS=${_IFS};
printf "${PRINTF_SUB_BLOCK}" "COMPACT" "$(date +'%F %T')" "${MODULE}";
BORG_COMPACT="${BORG_COMMAND} compact -v ${OPT_PROGRESS} ${REPOSITORY}";
if [ ${DEBUG} -eq 1 ]; then
BORG_COMPACT="${BORG_COMMAND} compact ${OPT_REMOTE} -v ${OPT_PROGRESS} ${REPOSITORY}";
if [ "${DEBUG}" -eq 1 ]; then
echo "${BORG_COMPACT}";
fi;
if [ ${DRYRUN} -eq 0 ]; then
if [ "${DRYRUN}" -eq 0 ]; then
${BORG_COMPACT};
fi;
fi;

View File

@@ -1,25 +1,33 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
if [ -z "${MODULE}" ]; then
echo "Script cannot be run on its own";
exit 1;
fi;
if [ ${INFO} -eq 1 ]; then
if [ "${INFO}" -eq 1 ]; then
printf "${PRINTF_SUB_BLOCK}" "INFO" "$(date +'%F %T')" "${MODULE}";
# show command on debug or dry run
if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
if [ "${DEBUG}" -eq 1 ] || [ "${DRYRUN}" -eq 1 ]; then
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY}";
fi;
# run info command if not a dry drun
if [ ${DRYRUN} -eq 0 ]; then
${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY};
if [ "${DRYRUN}" -eq 0 ]; then
${BORG_COMMAND} info ${OPT_REMOTE} "${REPOSITORY}";
if [ "${VERBOSE}" -eq 1 ]; then
# print key information
echo "------------------------------------------------------------------------------";
${BORG_COMMAND} key export "${REPOSITORY}";
fi;
fi;
if [ "${MODULE}" = "files" ]; then
if [ $FOLDER_OK -eq 1 ]; then
if [ "${FOLDER_OK}" -eq 1 ]; then
echo "--- [Run command]:";
#IFS="#";
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${COMMAND} "${FOLDERS_Q[*]};
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${COMMAND} ${FOLDERS_Q[*]}";
else
echo "[!] No folders where set for the backup";
fi;

View File

@@ -5,16 +5,24 @@ if [ -z "${MODULE}" ]; then
exit 1;
fi;
# E: inherit trap ERR
# T: DEBUG and RETURN traps are inherited
# u: unset variables ere error
set -ETu #-e -o pipefail
trap cleanup SIGINT SIGTERM ERR
trap _catch ERR
trap _cleanup SIGINT SIGTERM
cleanup() {
_cleanup() {
# script cleanup here
echo "Some part of the script failed with an error: $? @LINE: $(caller)";
echo "Script abort: $? @LINE: $(caller)";
# unset exported vars
unset BORG_BASE_DIR BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK BORG_RELOCATED_REPO_ACCESS_IS_OK;
# end trap
trap - SIGINT SIGTERM ERR
trap - SIGINT SIGTERM
}
_catch() {
local last_exit_code=$?;
echo "Some part of the script failed with ERROR: $last_exit_code @COMMAND: '$BASH_COMMAND' @LINE: $(caller)" >&2;
}
# on exit unset any exported var
trap "unset BORG_BASE_DIR BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK BORG_RELOCATED_REPO_ACCESS_IS_OK" EXIT;
@@ -24,7 +32,7 @@ function version {
}
# version for all general files
VERSION="4.4.0";
VERSION="4.7.2";
# borg version and borg comamnd
BORG_VERSION="";
@@ -85,7 +93,7 @@ REGEX="";
REGEX_COMMENT="^[\ \t]*#";
REGEX_GLOB='\*';
REGEX_NUMERIC="^[0-9]{1,2}$";
REGEX_ERROR="^Some part of the script failed with an error:";
REGEX_ERROR="^Some part of the script failed with ERROR:";
PRUNE_DEBUG="";
INIT_REPOSITORY=0;
FOLDER_OK=0;
@@ -95,7 +103,8 @@ PRINTF_INFO_STRING="%-23s: %s\n";
PRINTF_MASTER_BLOCK="=== [%-8s: %19s] ==[%s]====================================>\n";
PRINTF_SUB_BLOCK="|-- [%-8s: %19s] --[%s]------------------------------------>\n";
PRINTF_SUBEXT_BLOCK="|-- [%-8s: %s: %19s] --[%s]------------------------------------>\n";
PRINTF_DB_SUB_BLOCK=">>- [%-8s: %s] =======================[%s]====================================>\n";
PRINTF_DB_SUB_BLOCK="|>- [%-8s: %s] ==[%s]=======================>\n";
PRINTF_DB_RUN_TIME_SUB_BLOCK=">>- [%-8s: %s] ==[%s]==[Run time: %s]=======================>\n";
# opt flags
OPT_VERBOSE="";
OPT_PROGRESS="";
@@ -124,7 +133,7 @@ COMPRESSION_LEVEL="";
SUB_COMPRESSION="";
SUB_COMPRESSION_LEVEL="";
# encryption settings
DEFAULT_ENCRYPTION="none";
DEFAULT_ENCRYPTION="keyfile";
ENCRYPTION="";
# force verify always
DEFAULT_FORCE_VERIFY="false";
@@ -152,17 +161,21 @@ MYSQL_DB_CONFIG="";
MYSQL_DB_CONFIG_PARAM="";
# gitea module
GIT_USER="";
GITEA_TMP="";
GITEA_WORKING_DIR="";
GITEA_TEMP_DIR="";
GITEA_BIN="";
GITEA_CONFIG="";
GITEA_EXPORT_TYPE="";
# zabbix module
ZABBIX_DUMP_BIN="";
ZABBIX_CONFIG="";
ZABBIX_DATABASE="";
ZABBIX_UNKNOWN_TABLES="";
ZABBIX_DB_PORT="";
OPT_ZABBIX_DUMP="";
OPT_ZABBIX_CONFIG="";
OPT_ZABBIX_UNKNOWN_TABLES="";
OPT_ZABBIX_DB_PORT="";
# default keep 7 days, 4 weeks, 6 months, 1 year
# if set 0, ignore/off
# note that for last/hourly it is needed to create a different
@@ -329,32 +342,32 @@ if [ ${VERIFY} -eq 1 ] || [ ${INIT} -eq 1 ] && [ ${INFO} -eq 1 ]; then
exit 1;
fi;
# print -P cannot be run with -i/-C/-I together
if [ ${PRINT} -eq 1 ] && ([ ${INIT} -eq 1 ] || [ ${VERIFY} -eq 1 ] || [ ${INFO} -eq 1 ]); then
if [ ${PRINT} -eq 1 ] && { [ ${INIT} -eq 1 ] || [ ${VERIFY} -eq 1 ] || [ ${INFO} -eq 1 ]; }; then
echo "Cannot have -P print option and -i info, -V verify or -I initizalize option at the same time";
exit 1;
fi;
# if tag is set, you can't have init, verify, info, etc
if [ ! -z "${ONE_TIME_TAG}" ] && ([ ${PRINT} -eq 1 ] || [ ${INIT} -eq 1 ] || [ ${VERIFY} -eq 1 ] || [ ${INFO} -eq 1 ]); then
if [ -n "${ONE_TIME_TAG}" ] && { [ ${PRINT} -eq 1 ] || [ ${INIT} -eq 1 ] || [ ${VERIFY} -eq 1 ] || [ ${INFO} -eq 1 ]; }; then
echo "Cannot have -T '${ONE_TIME_TAG}' option with -i info, -V verify, -I initialize or -P print option at the same time";
exit 1;
fi;
# verify only alphanumeric, no spaces, only underscore and dash
if [ ! -z "${ONE_TIME_TAG}" ] && ! [[ "${ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+$ ]]; then
if [ -n "${ONE_TIME_TAG}" ] && ! [[ "${ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+$ ]]; then
echo "One time tag '${ONE_TIME_TAG}' must be alphanumeric with dashes and underscore only.";
exit 1;
elif [ ! -z "${ONE_TIME_TAG}" ]; then
# all ok, attach . at the end
elif [ -n "${ONE_TIME_TAG}" ]; then
# all ok, attach '.' at the end
ONE_TIME_TAG=${ONE_TIME_TAG}".";
fi;
# if -D, cannot be with -T, -i, -C, -I, -P
if [ ! -z "${DELETE_ONE_TIME_TAG}" ] && ([ ! -z "${ONE_TIME_TAG}" ] || [ ${PRINT} -eq 1 ] || [ ${INIT} -eq 1 ] || [ ${VERIFY} -eq 1 ] || [ ${INFO} -eq 1 ]); then
if [ -n "${DELETE_ONE_TIME_TAG}" ] && { [ -n "${ONE_TIME_TAG}" ] || [ ${PRINT} -eq 1 ] || [ ${INIT} -eq 1 ] || [ ${VERIFY} -eq 1 ] || [ ${INFO} -eq 1 ]; }; then
echo "Cannot have -D delete tag option with -T one time tag, -i info, -V verify, -I initialize or -P print option at the same time";
exit 1;
fi;
# -D also must be in valid backup set format
# ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},(\*-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T\*$ ]]
if [ ! -z "${DELETE_ONE_TIME_TAG}" ] && ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},([A-Za-z0-9_-]+-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}$ ]] && ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},(\*-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T\*$ ]]; then
echo "Delete one time tag '${DELETE_ONE_TIME_TAG}' is in an invalid format. "
if [ -n "${DELETE_ONE_TIME_TAG}" ] && ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},([A-Za-z0-9_-]+-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}$ ]] && ! [[ "${DELETE_ONE_TIME_TAG}" =~ ^[A-Za-z0-9_-]+\.${MODULE},(\*-)?[0-9]{4}-[0-9]{2}-[0-9]{2}T\*$ ]]; then
echo "Delete one time tag '${DELETE_ONE_TIME_TAG}' is in an invalid format."
echo "Please verify existing tags with -P option."
echo "For a globing be sure it is in the format of: TAG.MODULE,*-YYYY-MM-DDT*";
echo "Note the dash (-) after the first *, also time (T) is a globa (*) must."
@@ -366,7 +379,7 @@ if [ ${CHECK_VERIFY_DATA} -eq 1 ] && [ ${CHECK} -eq 0 ]; then
exit 1;
fi;
# -p can't be set without -C
if [ ! -z "${CHECK_PREFIX}" ] && [ ${CHECK} -eq 0 ]; then
if [ -n "${CHECK_PREFIX}" ] && [ ${CHECK} -eq 0 ]; then
echo "-p (pattern|glob) for check cannot be run without -C (Check) options";
exit 1;
fi;
@@ -399,20 +412,21 @@ fi;
. "${BASE_FOLDER}${SETTINGS_FILE}";
# if OPTION SET overrides ALL others
if [ ! -z "${OPT_BORG_EXECUTEABLE}" ]; then
if [ -n "${OPT_BORG_EXECUTEABLE}" ]; then
BORG_COMMAND="${OPT_BORG_EXECUTEABLE}";
if [ ! -f "${BORG_COMMAND}" ]; then
echo "borg command not found with option -b: ${BORG_COMMAND}";
exit;
fi;
# if in setting file, use this
elif [ ! -z "${BORG_EXECUTEABLE}" ]; then
elif [ -n "${BORG_EXECUTEABLE}" ]; then
BORG_COMMAND="${BORG_EXECUTEABLE}";
if [ ! -f "${BORG_COMMAND}" ]; then
echo "borg command not found with setting: ${BORG_COMMAND}";
exit;
fi;
elif ! command -v borg &> /dev/null; then
# elif [ -z ($command -v borg) ]; then
echo "borg backup seems not to be installed, please verify paths";
exit;
fi;
@@ -445,7 +459,7 @@ if [ -z "${CHECK_INTERVAL}" ]; then
CHECK_INTERVAL="${DEFAULT_CHECK_INTERVAL}";
fi;
# deprecated name FORCE_CHECK, use FORCE_VERIFY instead
if [ ! -z "${FORCE_CHECK}" ]; then
if [ -n "${FORCE_CHECK}" ]; then
FORCE_VERIFY="${FORCE_CHECK}";
fi;
if [ -z "${FORCE_VERIFY}" ]; then
@@ -476,48 +490,48 @@ SETTINGS_FILE_SUB=$(echo "${SETTINGS_FILE}" | sed -e "s/\.settings/\.${MODULE,,}
if [ -f "${BASE_FOLDER}${SETTINGS_FILE_SUB}" ]; then
. "${BASE_FOLDER}${SETTINGS_FILE_SUB}";
# if SUB_ set override master
if [ ! -z "${SUB_BACKUP_FILE}" ]; then
if [ -n "${SUB_BACKUP_FILE}" ]; then
BACKUP_FILE=${SUB_BACKUP_FILE}
fi;
# if sub backup set it set, override current
if [ ! -z "${SUB_BACKUP_SET}" ]; then
if [ -n "${SUB_BACKUP_SET}" ]; then
BACKUP_SET=${SUB_BACKUP_SET};
fi;
# ovrride compression
if [ ! -z "${SUB_COMPRESSION}" ]; then
if [ -n "${SUB_COMPRESSION}" ]; then
COMPRESSION=${SUB_COMPRESSION};
fi;
if [ ! -z "${SUB_COMPRESSION_LEVEL}" ]; then
if [ -n "${SUB_COMPRESSION_LEVEL}" ]; then
COMPRESSION_LEVEL=${SUB_COMPRESSION_LEVEL};
fi;
# compact interval override
if [ ! -z "${SUB_COMPACT_INTERVAL}" ]; then
if [ -n "${SUB_COMPACT_INTERVAL}" ]; then
COMPACT_INTERVAL="${SUB_COMPACT_INTERVAL}";
fi;
# override check interval
if [ ! -z "${SUB_CHECK_INTERVAL}" ]; then
if [ -n "${SUB_CHECK_INTERVAL}" ]; then
CHECK_INTERVAL="${SUB_CHECK_INTERVAL}";
fi;
# check override for keep time
if [ ! -z "${SUB_KEEP_LAST}" ]; then
if [ -n "${SUB_KEEP_LAST}" ]; then
KEEP_LAST=${SUB_KEEP_LAST};
fi;
if [ ! -z "${SUB_KEEP_HOURS}" ]; then
if [ -n "${SUB_KEEP_HOURS}" ]; then
KEEP_HOURS=${SUB_KEEP_HOURS};
fi;
if [ ! -z "${SUB_KEEP_DAYS}" ]; then
if [ -n "${SUB_KEEP_DAYS}" ]; then
KEEP_DAYS=${SUB_KEEP_DAYS};
fi;
if [ ! -z "${SUB_KEEP_WEEKS}" ]; then
if [ -n "${SUB_KEEP_WEEKS}" ]; then
KEEP_WEEKS=${SUB_KEEP_WEEKS};
fi;
if [ ! -z "${SUB_KEEP_MONTHS}" ]; then
if [ -n "${SUB_KEEP_MONTHS}" ]; then
KEEP_MONTHS=${SUB_KEEP_MONTHS};
fi;
if [ ! -z "${SUB_KEEP_YEARS}" ]; then
if [ -n "${SUB_KEEP_YEARS}" ]; then
KEEP_YEARS=${SUB_KEEP_YEARS};
fi;
if [ ! -z "${SUB_KEEP_WITHIN}" ]; then
if [ -n "${SUB_KEEP_WITHIN}" ]; then
KEEP_WITHIN=${SUB_KEEP_WITHIN};
fi;
fi;
@@ -565,7 +579,7 @@ fi
# log file set and check
# option folder overrides all other folders
if [ ! -z "${OPT_LOG_FOLDER}" ]; then
if [ -n "${OPT_LOG_FOLDER}" ]; then
LOG_FOLDER="${OPT_LOG_FOLDER}";
fi;
# if empty folder set to default folder
@@ -593,11 +607,26 @@ fi;
# if ENCRYPTION is empty or not in the valid list fall back to none
# NOTE This is currently set in default and doesn't need to be set on empty
# only ivalid should be checked
#if [ -z "${ENCRYPTION}" ]; then
# ENCRYPTION="none";
#else
# TODO check for invalid encryption string
#fi;
if
[ "${ENCRYPTION}" = "authenticated" ] ||
[ "${ENCRYPTION}" = "repokey" ] ||
[ "${ENCRYPTION}" = "authenticated-blake2" ] ||
[ "${ENCRYPTION}" = "repokey-blake2" ] ;
then
# if "authenticated" or "repokey" a password must be set
if [[ ! -v BORG_PASSPHRASE ]] && [[ ! -v BORG_PASSCOMMAND ]] && [[ ! -v BORG_PASSPHRASE_FD ]]; then
echo "Encryption method '${ENCRYPTION}' requires a BORG_PASSPHRASE, BORG_PASSCOMMAND or BORG_PASSPHRASE_FD to be set.";
exit 1;
fi;
elif [ "${ENCRYPTION}" = "keyfile" ] || [ "${ENCRYPTION}" = "keyfile-blake2" ]; then
# if no password, set empty password
if [[ ! -v BORG_PASSPHRASE ]] && [[ ! -v BORG_PASSCOMMAND ]] && [[ ! -v BORG_PASSPHRASE_FD ]]; then
export BORG_PASSPHRASE="";
fi;
elif [ "${ENCRYPTION}" != "none" ]; then
echo "Encryption method '${ENCRYPTION}' is not valid.";
exit 1;
fi;
## FUNCTIONS
@@ -612,30 +641,30 @@ function convert_time
{
timestamp=${1};
# round to four digits for ms
timestamp=$(printf "%1.4f" $timestamp);
timestamp=$(printf "%1.4f" "$timestamp");
# get the ms part and remove any leading 0
ms=$(echo ${timestamp} | cut -d "." -f 2 | sed -e 's/^0*//');
timestamp=$(echo ${timestamp} | cut -d "." -f 1);
ms=$(echo "${timestamp}" | cut -d "." -f 2 | sed -e 's/^0*//');
timestamp=$(echo "${timestamp}" | cut -d "." -f 1);
timegroups=(86400 3600 60 1); # day, hour, min, sec
timenames=("d" "h" "m" "s"); # day, hour, min, sec
output=( );
time_string=;
for timeslice in ${timegroups[@]}; do
for timeslice in "${timegroups[@]}"; do
# floor for the division, push to output
output[${#output[*]}]=$(awk "BEGIN {printf \"%d\", ${timestamp}/${timeslice}}");
timestamp=$(awk "BEGIN {printf \"%d\", ${timestamp}%${timeslice}}");
done;
for ((i=0; i<${#output[@]}; i++)); do
if [ ${output[$i]} -gt 0 ] || [ ! -z "$time_string" ]; then
if [ ! -z "${time_string}" ]; then
if [ "${output[$i]}" -gt 0 ] || [ -n "$time_string" ]; then
if [ -n "${time_string}" ]; then
time_string=${time_string}" ";
fi;
time_string=${time_string}${output[$i]}${timenames[$i]};
fi;
done;
if [ ! -z ${ms} ] && [ "${ms}" != "nan" ] && [ ${ms} -gt 0 ]; then
time_string=${time_string}" "${ms}"ms";
if [ -n "${ms}" ] && [ "${ms}" != "nan" ] && [ "${ms}" -gt 0 ]; then
time_string="${time_string} ${ms}ms";
fi;
# just in case the time is 0
if [ -z "${time_string}" ]; then

View File

@@ -1,5 +1,8 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
if [ -z "${MODULE}" ]; then
echo "Script cannot be run on its own";
exit 1;
@@ -10,7 +13,7 @@ START=$(date +'%s');
# set init date, or today if not file is set
BACKUP_INIT_DATE='';
if [ -f "${BASE_FOLDER}${BACKUP_INIT_FILE}" ]; then
BACKUP_INIT_DATE=$(printf '%(%c)T' $(cat "${BASE_FOLDER}${BACKUP_INIT_FILE}" 2>/dev/null));
BACKUP_INIT_DATE=$(printf '%(%c)T' "$(cat "${BASE_FOLDER}${BACKUP_INIT_FILE}" 2>/dev/null)");
fi;
# start logging from here
exec &> >(tee -a "${LOG}");
@@ -30,13 +33,13 @@ printf "${PRINTF_INFO_STRING}" "Base folder" "${BASE_FOLDER}";
printf "${PRINTF_INFO_STRING}" "Module init date" "${BACKUP_INIT_DATE}";
# print last compact date if positive integer
# only if borg > 1.2
if [ $(version $BORG_VERSION) -ge $(version "1.2.0") ]; then
if [ "$(version "$BORG_VERSION")" -ge "$(version "1.2.0")" ]; then
if [ "${COMPACT_INTERVAL##*[!0-9]*}" ]; then
printf "${PRINTF_INFO_STRING}" "Module compact interval" "${COMPACT_INTERVAL}";
if [ -f "${BASE_FOLDER}${BACKUP_COMPACT_FILE}" ]; then
LAST_COMPACT_DATE=$(cat "${BASE_FOLDER}${BACKUP_COMPACT_FILE}" 2>/dev/null);
printf "${PRINTF_INFO_STRING}" "Module last compact" \
"$(printf '%(%c)T' ${LAST_COMPACT_DATE}) ($(convert_time $(($(date +%s)-${LAST_COMPACT_DATE}))) ago)";
"$(printf '%(%c)T' "${LAST_COMPACT_DATE}") ($(convert_time $(($(date +%s) - LAST_COMPACT_DATE))) ago)";
else
printf "${PRINTF_INFO_STRING}" "Module last compact" "No compact run yet"
fi;
@@ -49,7 +52,7 @@ if [ "${CHECK_INTERVAL##*[!0-9]*}" ]; then
if [ -f "${BASE_FOLDER}${BACKUP_CHECK_FILE}" ]; then
LAST_CHECK_DATE=$(cat "${BASE_FOLDER}${BACKUP_CHECK_FILE}" 2>/dev/null);
printf "${PRINTF_INFO_STRING}" "Module last check" \
"$(printf '%(%c)T' ${LAST_CHECK_DATE}) ($(convert_time $(($(date +%s)-${LAST_CHECK_DATE}))) ago)";
"$(printf '%(%c)T' "${LAST_CHECK_DATE}") ($(convert_time $(($(date +%s) - LAST_CHECK_DATE))) ago)";
else
printf "${PRINTF_INFO_STRING}" "Module last check" "No check run yet";
fi;
@@ -57,15 +60,15 @@ fi;
# if force verify is true set VERIFY to 1 unless INFO is 1
# Needs bash 4.0 at lesat for this
if [ "${FORCE_VERIFY,,}" = "true" ] && [ ${INFO} -eq 0 ]; then
if [ "${FORCE_VERIFY,,}" = "true" ] && [ "${INFO}" -eq 0 ]; then
VERIFY=1;
if [ ${DEBUG} -eq 1 ]; then
if [ "${DEBUG}" -eq 1 ]; then
echo "Force repository verify";
fi;
fi;
# remote borg path
if [ ! -z "${TARGET_BORG_PATH}" ]; then
if [ -n "${TARGET_BORG_PATH}" ]; then
if [[ "${TARGET_BORG_PATH}" =~ \ |\' ]]; then
echo "Space found in ${TARGET_BORG_PATH}. Aborting";
echo "There are issues with passing on paths with spaces"
@@ -102,16 +105,16 @@ TARGET_SERVER='';
# allow host only (if full setup in .ssh/config)
# user@host OR ssh://user@host:port/ IF TARGET_PORT is set
# user/host/port
if [ ! -z "${TARGET_USER}" ] && [ ! -z "${TARGET_HOST}" ] && [ ! -z "${TARGET_PORT}" ]; then
if [ -n "${TARGET_USER}" ] && [ -n "${TARGET_HOST}" ] && [ -n "${TARGET_PORT}" ]; then
TARGET_SERVER="ssh://${TARGET_USER}@${TARGET_HOST}:${TARGET_PORT}/";
# host/port
elif [ ! -z "${TARGET_HOST}" ] && [ ! -z "${TARGET_PORT}" ]; then
elif [ -n "${TARGET_HOST}" ] && [ -n "${TARGET_PORT}" ]; then
TARGET_SERVER="ssh://${TARGET_HOST}:${TARGET_PORT}/";
# user/host
elif [ ! -z "${TARGET_USER}" ] && [ ! -z "${TARGET_HOST}" ]; then
elif [ -n "${TARGET_USER}" ] && [ -n "${TARGET_HOST}" ]; then
TARGET_SERVER="${TARGET_USER}@${TARGET_HOST}:";
# host
elif [ ! -z "${TARGET_HOST}" ]; then
elif [ -n "${TARGET_HOST}" ]; then
TARGET_SERVER="${TARGET_HOST}:";
fi;
# we dont allow special characters, so we don't need to special escape it
@@ -120,13 +123,13 @@ printf "${PRINTF_INFO_STRING}" "Repository" "${REPOSITORY}";
# check if given compression name and level are valid
OPT_COMPRESSION='';
if [ ! -z "${COMPRESSION}" ]; then
if [ -n "${COMPRESSION}" ]; then
# valid compression
if [ "${COMPRESSION}" = "lz4" ] || [ "${COMPRESSION}" = "zlib" ] || [ "${COMPRESSION}" = "lzma" ] || [ "${COMPRESSION}" = "zstd" ]; then
OPT_COMPRESSION="-C=${COMPRESSION}";
# if COMPRESSION_LEVEL, check it is a valid regex
# for zlib, zstd, lzma
if [ ! -z "${COMPRESSION_LEVEL}" ] && ([ "${COMPRESSION}" = "zlib" ] || [ "${COMPRESSION}" = "lzma" ] || [ "${COMPRESSION}" = "zstd" ]); then
if [ -n "${COMPRESSION_LEVEL}" ] && { [ "${COMPRESSION}" = "zlib" ] || [ "${COMPRESSION}" = "lzma" ] || [ "${COMPRESSION}" = "zstd" ]; }; then
MIN_COMPRESSION=0;
MAX_COMPRESSION=0;
case "${COMPRESSION}" in
@@ -152,10 +155,10 @@ if [ ! -z "${COMPRESSION}" ]; then
# fi;
error_message="[! $(date +'%F %T')] Compression level for ${COMPRESSION} needs to be a numeric value between ${MIN_COMPRESSION} and ${MAX_COMPRESSION}: ${COMPRESSION_LEVEL}";
if ! [[ "${COMPRESSION_LEVEL}" =~ ${REGEX_NUMERIC} ]]; then
echo ${error_message};
echo "${error_message}";
exit 1;
elif [ ${COMPRESSION_LEVEL} -lt ${MIN_COMPRESSION} ] || [ ${COMPRESSION_LEVEL} -gt ${MAX_COMPRESSION} ]; then
echo ${error_message};
elif [ "${COMPRESSION_LEVEL}" -lt "${MIN_COMPRESSION}" ] || [ "${COMPRESSION_LEVEL}" -gt "${MAX_COMPRESSION}" ]; then
echo "${error_message}";
exit 1;
else
OPT_COMPRESSION=${OPT_COMPRESSION}","${COMPRESSION_LEVEL};
@@ -177,7 +180,7 @@ KEEP_OPTIONS=();
# keep info string (for files)
KEEP_INFO="";
# override standard keep for tagged backups
if [ ! -z "${ONE_TIME_TAG}" ]; then
if [ -n "${ONE_TIME_TAG}" ]; then
BACKUP_SET="{now:%Y-%m-%dT%H:%M:%S}";
# set empty to avoid problems
KEEP_OPTIONS=("");
@@ -185,32 +188,32 @@ else
# build options and info string,
# also flag BACKUP_SET check if hourly is set
BACKUP_SET_VERIFY=0;
if [ ${KEEP_LAST} -gt 0 ]; then
if [ "${KEEP_LAST}" -gt 0 ]; then
KEEP_OPTIONS+=("--keep-last=${KEEP_LAST}");
KEEP_INFO="${KEEP_INFO}, last: ${KEEP_LAST}";
fi;
if [ ${KEEP_HOURS} -gt 0 ]; then
if [ "${KEEP_HOURS}" -gt 0 ]; then
KEEP_OPTIONS+=("--keep-hourly=${KEEP_HOURS}");
KEEP_INFO="${KEEP_INFO}, hourly: ${KEEP_HOURS}";
BACKUP_SET_VERIFY=1;
fi;
if [ ${KEEP_DAYS} -gt 0 ]; then
if [ "${KEEP_DAYS}" -gt 0 ]; then
KEEP_OPTIONS+=("--keep-daily=${KEEP_DAYS}");
KEEP_INFO="${KEEP_INFO}, daily: ${KEEP_DAYS}";
fi;
if [ ${KEEP_WEEKS} -gt 0 ]; then
if [ "${KEEP_WEEKS}" -gt 0 ]; then
KEEP_OPTIONS+=("--keep-weekly=${KEEP_WEEKS}");
KEEP_INFO="${KEEP_INFO}, weekly: ${KEEP_WEEKS}";
fi;
if [ ${KEEP_MONTHS} -gt 0 ]; then
if [ "${KEEP_MONTHS}" -gt 0 ]; then
KEEP_OPTIONS+=("--keep-monthly=${KEEP_MONTHS}");
KEEP_INFO="${KEEP_INFO}, monthly: ${KEEP_MONTHS}";
fi;
if [ ${KEEP_YEARS} -gt 0 ]; then
if [ "${KEEP_YEARS}" -gt 0 ]; then
KEEP_OPTIONS+=("--keep-yearly=${KEEP_YEARS}");
KEEP_INFO="${KEEP_INFO}, yearly: ${KEEP_YEARS}";
fi;
if [ ! -z "${KEEP_WITHIN}" ]; then
if [ -n "${KEEP_WITHIN}" ]; then
# check for invalid string. can only be number + H|d|w|m|y
if [[ "${KEEP_WITHIN}" =~ ^[0-9]+[Hdwmy]{1}$ ]]; then
KEEP_OPTIONS+=("--keep-within=${KEEP_WITHIN}");
@@ -244,7 +247,7 @@ fi;
if [ -f "${BASE_FOLDER}${BACKUP_LOCK_FILE}" ]; then
LOCK_PID=$(cat "${BASE_FOLDER}${BACKUP_LOCK_FILE}" 2>/dev/null);
# check if lock file pid has an active program attached to it
if [ -f /proc/${LOCK_PID}/cmdline ]; then
if [ -f "/proc/${LOCK_PID}/cmdline" ]; then
echo "Script is already running on PID: ${$}";
. "${DIR}/borg.backup.functions.close.sh" 1;
exit 1;
@@ -272,11 +275,11 @@ _BORG_PRUNE="${BORG_COMMAND} prune ${OPT_REMOTE} -v --list ${OPT_PROGRESS} ${DRY
# set base path to config directory to keep cache/config separated
export BORG_BASE_DIR="${BASE_FOLDER}";
# ignore non encrypted access
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=${_BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK};
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="${_BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK}";
# ignore moved repo access
export BORG_RELOCATED_REPO_ACCESS_IS_OK=${_BORG_RELOCATED_REPO_ACCESS_IS_OK};
export BORG_RELOCATED_REPO_ACCESS_IS_OK="${_BORG_RELOCATED_REPO_ACCESS_IS_OK}";
# and for debug print that tout
if [ ${DEBUG} -eq 1 ]; then
if [ "${DEBUG}" -eq 1 ]; then
echo "export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=${_BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK};";
echo "export BORG_RELOCATED_REPO_ACCESS_IS_OK=${_BORG_RELOCATED_REPO_ACCESS_IS_OK};";
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";";
@@ -288,36 +291,34 @@ COMMAND_INFO="${COMMAND_EXPORT}${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY}"
# if this is user@host, we need to use ssh command to verify if the file is there
# else a normal verify is ok
# unless explicit given, verify is skipped
if [ ${VERIFY} -eq 1 ] || [ ${INIT} -eq 1 ]; then
# MARK: VERIFY / INFO
if [ "${VERIFY}" -eq 1 ] || [ "${INIT}" -eq 1 ]; then
printf "${PRINTF_SUB_BLOCK}" "VERIFY" "$(date +'%F %T')" "${MODULE}";
if [ ! -z "${TARGET_SERVER}" ]; then
if [ ${DEBUG} -eq 1 ]; then
echo "${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY} 2>&1|grep \"Repository ID:\"";
fi;
# use borg info and verify if it returns "Repository ID:" in the first line
REPO_VERIFY=$(${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY} 2>&1|grep "Repository ID:");
# this is currently a hack to work round the error code in borg info
# this checks if REPO_VERIFY holds this error message and then starts init
if [[ -z "${REPO_VERIFY}" ]] || [[ "${REPO_VERIFY}" =~ ${REGEX_ERROR} ]]; then
INIT_REPOSITORY=1;
fi;
elif [ ! -d "${REPOSITORY}" ]; then
if [ "${DEBUG}" -eq 1 ]; then
echo "${BORG_COMMAND} info ${OPT_REMOTE} ${REPOSITORY} 2>&1 ";
fi;
# use borg info and verify if it returns "Repository ID:" in the first line
REPO_VERIFY=$(${BORG_COMMAND} info ${OPT_REMOTE} "${REPOSITORY}" 2>&1);
__last_error=$?;
# on any error in verify command force new INIT
if [[ $__last_error -ne 0 ]]; then
echo "[!] Repository verify error: ${REPO_VERIFY}";
INIT_REPOSITORY=1;
fi;
# if verrify but no init and repo is there but init file is missing set it
if [ ${VERIFY} -eq 1 ] && [ ${INIT} -eq 0 ] && [ ${INIT_REPOSITORY} -eq 0 ] &&
if [ "${VERIFY}" -eq 1 ] && [ "${INIT}" -eq 0 ] && [ "${INIT_REPOSITORY}" -eq 0 ] &&
[ ! -f "${BASE_FOLDER}${BACKUP_INIT_FILE}" ]; then
# write init file
echo "[!] Add missing init verify file";
echo "$(date +%s)" > "${BASE_FOLDER}${BACKUP_INIT_FILE}";
date +%s > "${BASE_FOLDER}${BACKUP_INIT_FILE}";
fi;
# end if verified but repository is not here
if [ ${VERIFY} -eq 1 ] && [ ${INIT} -eq 0 ] && [ ${INIT_REPOSITORY} -eq 1 ]; then
if [ "${VERIFY}" -eq 1 ] && [ "${INIT}" -eq 0 ] && [ "${INIT_REPOSITORY}" -eq 1 ]; then
echo "[! $(date +'%F %T')] No repository. Please run with -I flag to initialze repository";
. "${DIR}/borg.backup.functions.close.sh" 1;
exit 1;
fi;
if [ ${EXIT} -eq 1 ] && [ ${VERIFY} -eq 1 ] && [ ${INIT} -eq 0 ]; then
if [ "${EXIT}" -eq 1 ] && [ "${VERIFY}" -eq 1 ] && [ "${INIT}" -eq 0 ]; then
echo "Repository exists";
echo "For more information run:"
echo "${COMMAND_INFO}";
@@ -325,16 +326,31 @@ if [ ${VERIFY} -eq 1 ] || [ ${INIT} -eq 1 ]; then
exit;
fi;
fi;
if [ ${INIT} -eq 1 ] && [ ${INIT_REPOSITORY} -eq 1 ]; then
# MARK: INIT
if [ "${INIT}" -eq 1 ] && [ "${INIT_REPOSITORY}" -eq 1 ]; then
printf "${PRINTF_SUB_BLOCK}" "INIT" "$(date +'%F %T')" "${MODULE}";
if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
if [ "${DEBUG}" -eq 1 ] || [ "${DRYRUN}" -eq 1 ]; then
echo "${BORG_COMMAND} init ${OPT_REMOTE} -e ${ENCRYPTION} ${OPT_VERBOSE} ${REPOSITORY}";
echo "${BORG_COMMAND} key export ${REPOSITORY}";
echo "${BORG_COMMAND} key export --paper ${REPOSITORY}";
fi
if [ ${DRYRUN} -eq 0 ]; then
if [ "${DRYRUN}" -eq 0 ]; then
# should trap and exit properly here
${BORG_COMMAND} init ${OPT_REMOTE} -e ${ENCRYPTION} ${OPT_VERBOSE} ${REPOSITORY};
${BORG_COMMAND} init ${OPT_REMOTE} -e "${ENCRYPTION}" ${OPT_VERBOSE} "${REPOSITORY}";
# show the key file
if [ "${ENCRYPTION}" = "keyfile" ]; then
echo "--- [ENCRYPTION KEY] --[START]-------------------------------------------------->";
echo "Store the key and password in a safe place";
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";borg key export [--paper] ${REPOSITORY}";
echo "----[BORG KEY] -------------------------------->";
${BORG_COMMAND} key export "${REPOSITORY}";
echo "----[BORG KEY:paper] -------------------------->";
${BORG_COMMAND} key export --paper "${REPOSITORY}";
echo "--- [ENCRYPTION KEY] --[END ]-------------------------------------------------->";
fi;
# write init file
echo "$(date +%s)" > "${BASE_FOLDER}${BACKUP_INIT_FILE}";
date +%s > "${BASE_FOLDER}${BACKUP_INIT_FILE}";
echo "Repository initialized";
echo "For more information run:"
echo "${COMMAND_INFO}";
@@ -342,8 +358,8 @@ if [ ${INIT} -eq 1 ] && [ ${INIT_REPOSITORY} -eq 1 ]; then
. "${DIR}/borg.backup.functions.close.sh";
# exit after init
exit;
elif [ ${INIT} -eq 1 ] && [ ${INIT_REPOSITORY} -eq 0 ]; then
echo "[! $(date +'%F %T')] Repository already initialized";
elif [ "${INIT}" -eq 1 ] && [ "${INIT_REPOSITORY}" -eq 0 ]; then
echo "[!] ($(date +'%F %T')) Repository already initialized";
echo "For more information run:"
echo "${COMMAND_INFO}";
. "${DIR}/borg.backup.functions.close.sh" 1;
@@ -352,25 +368,26 @@ fi;
# verify for init file
if [ ! -f "${BASE_FOLDER}${BACKUP_INIT_FILE}" ]; then
echo "[! $(date +'%F %T')] It seems the repository has never been initialized."
echo "[!] ($(date +'%F %T')) It seems the repository has never been initialized."
echo "Please run -I to initialize or if already initialzed run with -C for init update."
. "${DIR}/borg.backup.functions.close.sh" 1;
exit 1;
fi;
# MARK: LIST / PRINT
# PRINT OUT current data, only do this if REPO exists
if [ ${PRINT} -eq 1 ]; then
if [ "${PRINT}" -eq 1 ]; then
printf "${PRINTF_SUB_BLOCK}" "PRINT" "$(date +'%F %T')" "${MODULE}";
FORMAT="{archive:<45} {comment:6} {start} - {end} [{id}] ({username}@{hostname}){NL}"
# show command on debug or dry run
if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
if [ "${DEBUG}" -eq 1 ] || [ "${DRYRUN}" -eq 1 ]; then
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} list ${OPT_REMOTE} --format ${FORMAT} ${REPOSITORY}";
fi;
# run info command if not a dry drun
if [ ${DRYRUN} -eq 0 ]; then
${BORG_COMMAND} list ${OPT_REMOTE} --format "${FORMAT}" ${REPOSITORY} ;
if [ "${DRYRUN}" -eq 0 ]; then
${BORG_COMMAND} list ${OPT_REMOTE} --format "${FORMAT}" "${REPOSITORY}" ;
fi;
if [ ${VERBOSE} -eq 1 ]; then
if [ "${VERBOSE}" -eq 1 ]; then
echo "";
echo "Base command info:"
echo "export BORG_BASE_DIR=\"${BASE_FOLDER}\";${BORG_COMMAND} [COMMAND] ${OPT_REMOTE} ${REPOSITORY}::[BACKUP] [PATH]";
@@ -389,21 +406,21 @@ if [ ${PRINT} -eq 1 ]; then
fi;
# run borg compact command and exit
if [ ${COMPACT} -eq 1 ]; then
if [ "${COMPACT}" -eq 1 ]; then
. "${DIR}/borg.backup.functions.compact.sh";
. "${DIR}/borg.backup.functions.close.sh";
exit;
fi;
# run borg check command and exit
if [ ${CHECK} -eq 1 ]; then
if [ "${CHECK}" -eq 1 ]; then
. "${DIR}/borg.backup.functions.check.sh";
. "${DIR}/borg.backup.functions.close.sh";
exit;
fi;
# DELETE ONE TIME TAG
if [ ! -z "${DELETE_ONE_TIME_TAG}" ]; then
if [ -n "${DELETE_ONE_TIME_TAG}" ]; then
printf "${PRINTF_SUB_BLOCK}" "DELETE" "$(date +'%F %T')" "${MODULE}";
# if a "*" is inside we don't do ONE archive, but globbing via -a option
DELETE_ARCHIVE=""
@@ -415,12 +432,12 @@ if [ ! -z "${DELETE_ONE_TIME_TAG}" ]; then
DELETE_ARCHIVE="::"${DELETE_ONE_TIME_TAG};
fi
# if this is borg <1.2 OPT_LIST does not work
if [ $(version $BORG_VERSION) -lt $(version "1.2.0") ]; then
if [ "$(version "$BORG_VERSION")" -lt "$(version "1.2.0")" ]; then
OPT_LIST="";
fi;
# if exists, delete and exit
# show command on debug or dry run
if [ ${DEBUG} -eq 1 ]; then
if [ "${DEBUG}" -eq 1 ]; then
echo "${BORG_COMMAND} delete ${OPT_REMOTE} ${OPT_LIST} -s ${OPT_GLOB} ${REPOSITORY}${DELETE_ARCHIVE}";
fi;
# run delete command if not a dry drun
@@ -432,12 +449,12 @@ if [ ! -z "${DELETE_ONE_TIME_TAG}" ]; then
fi;
# if not a dry run, compact repository after delete
# not that compact only works on borg 1.2
if [ $(version $BORG_VERSION) -ge $(version "1.2.0") ]; then
if [ ${DRYRUN} -eq 0 ]; then
${BORG_COMMAND} compact ${REPOSITORY};
if [ "$(version "$BORG_VERSION")" -ge "$(version "1.2.0")" ]; then
if [ "${DRYRUN}" -eq 0 ]; then
${BORG_COMMAND} compact ${OPT_REMOTE} "${REPOSITORY}";
fi;
if [ ${DEBUG} -eq 1 ]; then
echo "${BORG_COMMAND} compact ${REPOSITORY}";
if [ "${DEBUG}" -eq 1 ]; then
echo "${BORG_COMMAND} compact ${OPT_REMOTE} ${REPOSITORY}";
fi;
fi;
. "${DIR}/borg.backup.functions.close.sh";

View File

@@ -2,10 +2,8 @@
# rename to borg.backup.gitea.settings to use
# override settings in borg.backup.settings with SUB_ prefix
# valid for BACKUP_FILE, BACKUP_SET, COMPRESSION*, KEEP_*
GIT_USER="";
GITEA_TMP="";
GITEA_WORKING_DIR="";
GITEA_TEMP_DIR="";
GITEA_BIN="";
GITEA_CONFIG="";

View File

@@ -1,9 +1,12 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
# Backup gitea database, all git folders and gitea settings
MODULE="gitea"
MODULE_VERSION="1.1.4";
MODULE_VERSION="1.2.2";
DIR="${BASH_SOURCE%/*}"
if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
@@ -29,9 +32,13 @@ if [ -z "${GIT_USER}" ]; then
GIT_USER="git";
fi;
# set GITEA_* if not set
if [ -z "${GITEA_TMP}" ]; then
if [ -z "${GITEA_WORKING_DIR}" ]; then
# run gitea backup (user mktemp?)
GITEA_TMP="/tmp/gitea/";
GITEA_WORKING_DIR="/var/tmp/gitea/";
fi;
# general temp folder for temporary data storage, this is not working output folder
if [ -z "${GITEA_TEMP_DIR}" ]; then
GITEA_TEMP_DIR="/var/tmp";
fi;
if [ -z "${GITEA_BIN}" ]; then
GITEA_BIN="/usr/local/bin/gitea";
@@ -39,6 +46,10 @@ fi;
if [ -z "${GITEA_CONFIG}" ]; then
GITEA_CONFIG="/etc/gitea/app.ini"
fi;
# This one is not advertised in the config file as it is not recommended to change
if [ -z "${GITEA_EXPORT_TYPE}" ]; then
GITEA_EXPORT_TYPE="zip";
fi;
if [ ! -f "${GITEA_BIN}" ]; then
echo "[! $(date +'%F %T')] Cannot find gitea binary";
. "${DIR}/borg.backup.functions.close.sh" 1;
@@ -49,6 +60,16 @@ if [ ! -f "${GITEA_CONFIG}" ]; then
. "${DIR}/borg.backup.functions.close.sh" 1;
exit 1;
fi;
# some basic checks with abort
if [ ! -d "${GITEA_TEMP_DIR}" ]; then
echo "Temp directory does not exist: ${GITEA_TEMP_DIR}";
exit;
fi;
# we should check GITEA_EXPORT_TYPE too at some point for an allow list
# At the moment warn if not zip
if [ "${GITEA_EXPORT_TYPE}" != "zip" ]; then
echo "[!!!!] The gitea export type has been changed from 'zip' to '${GITEA_EXPORT_TYPE}'. This can either break or make exports take very ling";
fi;
# Filename
FILENAME="gitea.backup.zip";
# backup set and prefix
@@ -60,7 +81,7 @@ BORG_CALL=$(echo "${_BORG_CALL}" | sed -e "s/##FILENAME##/${FILENAME}/" | sed -e
BORG_PRUNE=$(echo "${_BORG_PRUNE}" | sed -e "s/##BACKUP_SET_PREFIX##/${BACKUP_SET_PREFIX}/");
printf "${PRINTF_SUB_BLOCK}" "BACKUP: git data and database" "$(date +'%F %T')" "${MODULE}";
if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
echo "sudo -u ${GIT_USER} ${GITEA_BIN} dump -c ${GITEA_CONFIG} -w ${GITEA_TMP} -L -f - | ${BORG_CALL}";
echo "sudo -u ${GIT_USER} ${GITEA_BIN} dump -c ${GITEA_CONFIG} -w ${GITEA_WORKING_DIR} -t ${GITEA_TEMP_DIR} --type ${GITEA_EXPORT_TYPE} -L -f - | ${BORG_CALL}";
if [ -z "${ONE_TIME_TAG}" ]; then
echo "${BORG_PRUNE}";
fi;
@@ -70,13 +91,13 @@ if [ ${DRYRUN} -eq 0 ]; then
# below was an old workaround
#export USER="${LOGNAME}" # workaround for broken gitea EUID check
# make sure temp folder is there and is set as git. user
if [ ! -d "${GITEA_TMP}" ]; then
mkdir -p "${GITEA_TMP}";
if [ ! -d "${GITEA_WORKING_DIR}" ]; then
mkdir -p "${GITEA_WORKING_DIR}";
fi;
chown -R ${GIT_USER}. "${GITEA_TMP}";
chown -R ${GIT_USER}: "${GITEA_WORKING_DIR}";
# this needs to be run in a folder that can be stat by git user
cd "${GITEA_TMP}";
sudo -u ${GIT_USER} ${GITEA_BIN} dump -c ${GITEA_CONFIG} -w ${GITEA_TMP} -L -f - | ${BORG_CALL};
cd "${GITEA_WORKING_DIR}" || exit 1;
sudo -u ${GIT_USER} ${GITEA_BIN} dump -c ${GITEA_CONFIG} -w ${GITEA_WORKING_DIR} -t ${GITEA_TEMP_DIR} --type ${GITEA_EXPORT_TYPE} -L -f - | ${BORG_CALL};
) 2>&1 | sed 's/\x1B\[[0-9;]\{1,\}[A-Za-z]//g' # remove all ESC strings
fi;
if [ -z "${ONE_TIME_TAG}" ]; then

View File

@@ -1,5 +1,8 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
# Backup MySQL/MariaDB
# default is per table dump, can be set to one full dump
# config override set in borg.backup.mysql.settings
@@ -7,7 +10,7 @@
# set last edit date + time
MODULE="mysql"
MODULE_VERSION="1.1.2";
MODULE_VERSION="1.1.5";
DIR="${BASH_SOURCE%/*}"
if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
@@ -62,10 +65,11 @@ if [ -z "${_MYSQL_OK}" ]; then
fi;
# below is for file name only
# set DB_VERSION (Distrib n.n.n-type)
# NEW: mysql Ver 15.1 Distrib 10.5.12-MariaDB, for debian-linux-gnu (x86_64) using EditLine wrapper
# OLD: mysql Ver 14.14 Distrib 5.7.35, for Linux (x86_64) using EditLine wrapper
# OLD: mysql Ver 14.14 Distrib 5.7.35, for Linux (x86_64) using EditLine wrapper
# NEW: mysql Ver 15.1 Distrib 10.5.12-MariaDB, for debian-linux-gnu (x86_64) using EditLine wrapper
# VARIANT: mysql from 11.8.3-MariaDB, client 15.2 for debian-linux-gnu (x86_64) using EditLine wrapper
_DB_VERSION_TYPE=$("${MYSQL_CMD}" --version);
_DB_VERSION=$(echo "${_DB_VERSION_TYPE}" | sed 's/.*Distrib \([0-9]\{1,\}\.[0-9]\{1,\}\)\.[0-9]\{1,\}.*/\1/');
_DB_VERSION=$(echo "${_DB_VERSION_TYPE}" | sed 's/.*\(Distrib\|from\) \([0-9]\{1,\}\.[0-9]\{1,\}\)\.[0-9]\{1,\}.*/\2/');
DB_VERSION=$(echo "${_DB_VERSION}" | cut -d " " -f 1);
# temporary until correct type detection is set
DB_TYPE="mysql";
@@ -86,13 +90,14 @@ EVENTDB="mysql"
EVENTS="--events"
# ALL IN ONE FILE or PER DATABASE FLAG
if [ ! -z "${DATABASE_FULL_DUMP}" ]; then
if [ -n "${DATABASE_FULL_DUMP}" ]; then
SCHEMA_ONLY='';
schema_flag='data';
if [ "${DATABASE_FULL_DUMP}" = "schema" ]; then
SCHEMA_ONLY='--no-data';
schema_flag='schema';
fi;
LOCAL_START=$(date +'%s');
printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "all databases" "$(date +'%F %T')" "${MODULE}";
# We only do a full backup and not per table backup here
# Filename
@@ -124,9 +129,12 @@ if [ ! -z "${DATABASE_FULL_DUMP}" ]; then
echo "Prune repository with keep${KEEP_INFO:1}";
${BORG_PRUNE};
fi;
DURATION=$(( $(date +'%s') - LOCAL_START ));
printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "all databases" "${MODULE}" "$(convert_time ${DURATION})";
else
${MYSQL_CMD} ${MYSQL_DB_CONFIG_PARAM} -B -N -e "show databases" |
while read db; do
while read -r db; do
LOCAL_START=$(date +'%s');
printf "${PRINTF_DB_SUB_BLOCK}" "DB" "${db}" "${MODULE}";
printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "${db}" "$(date +'%F %T')" "${MODULE}";
# exclude checks
@@ -143,7 +151,7 @@ else
fi;
exclude=0;
if [ -f "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
while read excl_db; do
while read -r excl_db; do
if [ "${db}" = "${excl_db}" ]; then
exclude=1;
break;
@@ -173,7 +181,7 @@ else
SCHEMA_ONLY=''; # empty for all
schema_flag='data'; # or data
if [ -s "${BASE_FOLDER}${SCHEMA_ONLY_FILE}" ]; then
while read schema_db; do
while read -r schema_db; do
if [ "${db}" = "${schema_db}" ]; then
SCHEMA_ONLY='--no-data';
schema_flag='schema';
@@ -213,6 +221,8 @@ else
else
echo "- [E] ${db}";
fi;
DURATION=$(( $(date +'%s') - LOCAL_START ));
printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "${db}" "${MODULE}" "$(convert_time ${DURATION})";
done;
fi;
# run compact at the end if not a dry run

View File

@@ -1,5 +1,8 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
# Backup PostgreSQL
# default is per table dump, can be set to one full dump
# config override set in borg.backup.pgsql.settings
@@ -7,7 +10,7 @@
# set last edit date + time
MODULE="pgsql"
MODULE_VERSION="1.2.2";
MODULE_VERSION="1.2.8";
DIR="${BASH_SOURCE%/*}"
@@ -32,7 +35,7 @@ BACKUP_LOCK_FILE="borg.backup.${MODULE}.lock";
# if info print info and then abort run
. "${DIR}/borg.backup.functions.info.sh";
if [ ! -z "${DATABASE_USER}" ]; then
if [ -n "${DATABASE_USER}" ]; then
DB_USER=${DATABASE_USER};
else
DB_USER='postgres';
@@ -95,16 +98,17 @@ CONN_DB_HOST=''; # -h <host>
CONN_DB_PORT=''; # -p <port>
# ALL IN ONE FILE or PER DATABASE FLAG
if [ ! -z "${DATABASE_FULL_DUMP}" ]; then
if [ -n "${DATABASE_FULL_DUMP}" ]; then
SCHEMA_ONLY='';
schema_flag='data';
if [ "${DATABASE_FULL_DUMP}" = "schema" ]; then
SCHEMA_ONLY='-s';
schema_flag='schema';
fi;
LOCAL_START=$(date +'%s');
printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "all databases" "$(date +'%F %T')" "${MODULE}";
# Filename
FILENAME-"all.${DB_USER}.NONE.${schema_flag}-${DB_VERSION}_${DB_HOST}_${DB_PORT}.c.sql"
FILENAME="all.${DB_USER}.NONE.${schema_flag}-${DB_VERSION}_${DB_HOST}_${DB_PORT}.c.sql"
# backup set:
BACKUP_SET_PREFIX="${MODULE},all-";
BACKUP_SET_NAME="${ONE_TIME_TAG}${BACKUP_SET_PREFIX}${schema_flag}-${BACKUP_SET}";
@@ -119,7 +123,7 @@ if [ ! -z "${DATABASE_FULL_DUMP}" ]; then
fi;
fi;
if [ ${DRYRUN} -eq 0 ]; then
$(${PG_DUMPALL} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} ${SCHEMA_ONLY} -c | ${BORG_CALL});
${PG_DUMPALL} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} ${SCHEMA_ONLY} -c | ${BORG_CALL};
_backup_error=$?;
if [ $_backup_error -ne 0 ]; then
echo "[! $(date +'%F %T')] Backup creation failed for full dump with error code: ${_backup_error}";
@@ -132,10 +136,13 @@ if [ ! -z "${DATABASE_FULL_DUMP}" ]; then
echo "Prune repository with keep${KEEP_INFO:1}";
${BORG_PRUNE};
fi;
DURATION=$(( $(date +'%s') - LOCAL_START ));
printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "all databases" "${MODULE}" "$(convert_time ${DURATION})";
else
# dump globals first
db="pg_globals";
schema_flag="data";
LOCAL_START=$(date +'%s');
printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "${db}" "$(date +'%F %T')" "${MODULE}";
# Filename
FILENAME="${db}.${DB_USER}.NONE.${schema_flag}-${DB_VERSION}_${DB_HOST}_${DB_PORT}.c.sql"
@@ -166,18 +173,21 @@ else
echo "Prune repository with keep${KEEP_INFO:1}";
${BORG_PRUNE};
fi;
DURATION=$(( $(date +'%s') - LOCAL_START ));
printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "${db}" "${MODULE}" "$(convert_time ${DURATION})";
# get list of tables
for owner_db in $(${PG_PSQL} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} -d template1 -t -A -F "," -X -q -c "SELECT pg_catalog.pg_get_userbyid(datdba) AS owner, datname, pg_catalog.pg_encoding_to_char(encoding) AS encoding FROM pg_catalog.pg_database WHERE datname "\!"~ 'template(0|1)' ORDER BY datname;"); do
for owner_db in $(${PG_PSQL} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} -d template1 -t -A -F "," -X -q -c "SELECT pg_catalog.pg_get_userbyid(datdba) AS owner, datname, pg_catalog.pg_encoding_to_char(encoding) AS encoding FROM pg_catalog.pg_database WHERE datname !~ 'template(0|1)' ORDER BY datname;"); do
LOCAL_START=$(date +'%s');
# get the user who owns the DB too
owner=$(echo ${owner_db} | cut -d "," -f 1);
db=$(echo ${owner_db} | cut -d "," -f 2);
encoding=$(echo ${owner_db} | cut -d "," -f 3);
owner=$(echo "${owner_db}" | cut -d "," -f 1);
db=$(echo "${owner_db}" | cut -d "," -f 2);
encoding=$(echo "${owner_db}" | cut -d "," -f 3);
printf "${PRINTF_DB_SUB_BLOCK}" "DB" "${db}" "${MODULE}";
printf "${PRINTF_SUBEXT_BLOCK}" "BACKUP" "${db}" "$(date +'%F %T')" "${MODULE}";
include=0;
if [ -s "${BASE_FOLDER}${INCLUDE_FILE}" ]; then
while read incl_db; do
while read -r incl_db; do
if [ "${db}" = "${incl_db}" ]; then
include=1;
break;
@@ -188,7 +198,7 @@ else
fi;
exclude=0;
if [ -f "${BASE_FOLDER}${EXCLUDE_FILE}" ]; then
while read excl_db; do
while read -r excl_db; do
if [ "${db}" = "${excl_db}" ]; then
exclude=1;
break;
@@ -204,7 +214,7 @@ else
# default is data dump
SCHEMA_ONLY='';
schema_flag='data';
while read schema_db; do
while read -r schema_db; do
if [ "${db}" = "${schema_db}" ]; then
SCHEMA_ONLY='-s';
schema_flag='schema';
@@ -216,7 +226,7 @@ else
# default to schema, unless in data list
SCHEMA_ONLY='-s';
schema_flag='schema';
while read data_db; do
while read -r data_db; do
if [ "${db}" = "${data_db}" ]; then
SCHEMA_ONLY='';
schema_flag='data';
@@ -249,7 +259,7 @@ else
fi;
fi;
if [ ${DRYRUN} -eq 0 ]; then
${PG_DUMP} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} -c ${SCHEMA_ONLY} --format=c ${db} | ${BORG_CALL};
${PG_DUMP} -U ${DB_USER} ${CONN_DB_HOST} ${CONN_DB_PORT} -c ${SCHEMA_ONLY} --format=c "${db}" | ${BORG_CALL};
_backup_error=$?;
if [ $_backup_error -ne 0 ]; then
echo "[! $(date +'%F %T')] Backup creation failed for ${db} dump with error code: ${_backup_error}";
@@ -265,6 +275,8 @@ else
else
echo "- [E] ${db}";
fi;
DURATION=$(( $(date +'%s') - LOCAL_START ));
printf "${PRINTF_DB_RUN_TIME_SUB_BLOCK}" "DONE" "${db}" "${MODULE}" "$(convert_time ${DURATION})";
done;
fi;
# run compact at the end if not a dry run

View File

@@ -13,20 +13,21 @@ TARGET_PORT="";
TARGET_BORG_PATH="";
# folder where the backup folder will be created
TARGET_FOLDER="";
# the backup file (folder) for this host $(hostname), must end with .borg
# the backup file (folder) for this backup, avoid dynamic variables in here, must end with .borg
BACKUP_FILE="";
# compression settings (empty for none, lz4, zstd, zlib, lzma)
# level, if empty then default, else number between 0 and 9, or 1 to 22 for zstd
# default is zstd, 3
# compression settings (none, lz4, zstd, zlib, lzma)
# empty is default zstd
COMPRESSION="";
# compression level, if empty then default 3, else number between 0 and 9, or 1 to 22 for zstd
COMPRESSION_LEVEL="";
# encryption settings:
# SHA-256: 'none', 'authenticated', 'repokey', 'keyfile'
# BLAKE2b: 'authenticated-blake2', 'repokey-blake2', 'keyfile-blake2'
# Note: none or empty does not encrypt
# Blank passwords allowed for only key (if used, use keyfile)
# Note: none does not encrypt and is not recommended
# Default is keyfile
# Blank passwords allowed for only keyfile, else passwords have to be set via BORG_PASSPHRASE or BORG_PASSCOMMAND
# See: http://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically
ENCRYPTION="";
ENCRYPTION=""
# force repository verify, default is off, set to true for verify on every run
FORCE_VERIFY="";
# compact interval, only if using borg 1.2 or higher
@@ -40,12 +41,15 @@ COMPACT_INTERVAL="";
# if set to 1 then every time the script runs
# any other value it means ever n days, eg 90 would be every 90 days
CHECK_INTERVAL="";
# default is %Y-%m-%d
# default is {now:%Y-%m-%d}
# todays date, if more than one per day add -%H%M for hour/minute
# it can also be "{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S.%f}"
BACKUP_SET="";
# prune times, how many are kept in each time frame
KEEP_LAST="";
KEEP_HOURS="";
KEEP_DAYS="";
KEEP_WEEKS="";
KEEP_MONTHS="";
KEEP_YEARS="";
KEEP_WITHIN="";

View File

@@ -15,3 +15,5 @@ ZABBIX_DATABASE="";
ZABBIX_CONFIG="";
# unknown tables, default ignore, is set to true, will force backup them
ZABBIX_UNKNOWN_TABLES="";
# overide zabbix db port read from zabbix config, if not set read from zabbix config
ZABBIX_DB_PORT="";

View File

@@ -1,9 +1,12 @@
#!/usr/bin/env bash
# allow variables in printf format string
# shellcheck disable=SC2059
# Backup zabbix config and settings only
MODULE="zabbix"
MODULE_VERSION="1.1.3";
MODULE_VERSION="1.1.4";
DIR="${BASH_SOURCE%/*}"
if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
@@ -26,7 +29,7 @@ BACKUP_LOCK_FILE="borg.backup.${MODULE}.lock";
if [ -z "${ZABBIX_DUMP_BIN}" ]; then
ZABBIX_DUMP_BIN="/usr/local/bin/zabbix-dump";
fi;
if [ ! -z "${ZABBIX_CONFIG}" ] && [ ! -f "${ZABBIX_CONFIG}" ]; then
if [ -n "${ZABBIX_CONFIG}" ] && [ ! -f "${ZABBIX_CONFIG}" ]; then
echo "[! $(date +'%F %T')] Cannot find zabbix config: ${ZABBIX_CONFIG}";
. "${DIR}/borg.backup.functions.close.sh" 1;
exit 1;
@@ -37,6 +40,10 @@ fi;
if [ "${ZABBIX_DATABASE}" = "psql" ]; then
OPT_ZABBIX_DUMP="-C";
fi;
OPT_ZABBIX_DB_PORT="";
if [ -n "${ZABBIX_DB_PORT}" ]; then
OPT_ZABBIX_DB_PORT="-P ${ZABBIX_DB_PORT}";
fi;
if [ "${ZABBIX_DATABASE}" != "psql" ] && [ "${ZABBIX_DATABASE}" != "mysql" ]; then
echo "[! $(date +'%F %T')] Zabbix dump must have database set to either psql or mysql";
. "${DIR}/borg.backup.functions.close.sh" 1;
@@ -70,13 +77,13 @@ fi;
printf "${PRINTF_SUB_BLOCK}" "BACKUP: zabbix settings" "$(date +'%F %T')" "${MODULE}";
if [ ${DEBUG} -eq 1 ] || [ ${DRYRUN} -eq 1 ]; then
echo "${ZABBIX_DUMP_BIN} -t ${ZABBIX_DATABASE} ${OPT_ZABBIX_UNKNOWN_TABLES} ${OPT_ZABBIX_DUMP} ${OPT_ZABBIX_CONFIG} -o - | ${BORG_CALL}"
echo "${ZABBIX_DUMP_BIN} ${OPT_ZABBIX_DB_PORT} -t ${ZABBIX_DATABASE} ${OPT_ZABBIX_UNKNOWN_TABLES} ${OPT_ZABBIX_DUMP} ${OPT_ZABBIX_CONFIG} -o - | ${BORG_CALL}"
if [ -z "${ONE_TIME_TAG}" ]; then
echo "${BORG_PRUNE}";
fi;
fi;
if [ ${DRYRUN} -eq 0 ]; then
${ZABBIX_DUMP_BIN} -t ${ZABBIX_DATABASE} ${OPT_ZABBIX_UNKNOWN_TABLES} ${OPT_ZABBIX_DUMP} ${OPT_ZABBIX_CONFIG} -o - | ${BORG_CALL};
${ZABBIX_DUMP_BIN} ${OPT_ZABBIX_DB_PORT} -t ${ZABBIX_DATABASE} ${OPT_ZABBIX_UNKNOWN_TABLES} ${OPT_ZABBIX_DUMP} ${OPT_ZABBIX_CONFIG} -o - | ${BORG_CALL};
fi;
if [ -z "${ONE_TIME_TAG}" ]; then
printf "${PRINTF_SUB_BLOCK}" "PRUNE" "$(date +'%F %T')" "${MODULE}";

View File

@@ -33,7 +33,7 @@ function usage ()
}
# set options
while getopts ":c:m:uf:h" opt do
while getopts ":c:m:uf:h" opt; do
case "${opt}" in
c|config)
BASE_FOLDER=${OPTARG};