QNAPでvirsh仮想マシンとPlex、Subversion、Mailbox用NFS共有ドライブをrcloneでOneDriveにバックアップ

pjbondi · 2025 年 9 月 8 日午前 11:13

Hello to all:

On QNAP, I run Plex and a Debian 12 virtual machine. On the Debian 12 VM, I
mount NFS shared drives on the QNAP. All data accessed by the VM is resident
on the QNAP. In this manner, the VM has stayed small (40 Gb) and is easy to
backup to OneDrive where family members have 1 Tb storage. On the VM, I want
to backup sensitive data. Right now, that is Subversion and email mailboxes.

I would like to thank on-line communities who have helped me. I would like
to share my script for your benefit, which includes these features:

online backup of virsh virtual machines
backup of Plex metadata
backup of all Subversion files in the Subversion repo
backup of all mbox folders/files

Install software required on QNAP:
https://www.myqnap.org/install-the-repo/
https://www.myqnap.org/product/mutt-cli/
https://www.myqnap.org/product/rclone/

Setup rclone. To verify, observe configurations:
rclone config --config=/share/CACHEDEV1_DATA/homes/admin/.config/rclone/rclone.conf

Setup shared directory to install script.
Modify script as required for your environment.
Install script into cron.

#!/bin/bash -xv
# ****************************************************************************
#
#          $Id: qnap-backup.sh 167775 2025-09-07 21:30:25Z svn_beechwood $
#        $Date: 2025-09-07 17:30:25 -0400 (Sun, 07 Sep 2025) $
#
#  Description: On QNAP Linux, the root crontab must be edited directly:
#               /etc/config/crontab
#
#               Then re-launch:
#               crontab /etc/config/crontab && /etc/init.d/crond.sh restart; date
#
#               https://www.myqnap.org/product/mutt-cli/
#               https://www.myqnap.org/product/rclone/
#
#               To verify, observe configurations:
#               rclone config --config=/share/CACHEDEV1_DATA/homes/admin/.config/rclone/rclone.conf
#
#
# ****************************************************************************
# http://www.davidpashley.com/articles/writing-robust-shell-scripts.html
set -o nounset
shopt -s extglob
# 
# 2025-08-03: https://forum.qnap.com/viewtopic.php?t=156082
export LD_LIBRARY_PATH=/QVS/usr/lib:/QVS/usr/lib64/; export PATH=$PATH:/QVS/usr/bin/:/QVS/usr/sbin
#
# private
SCRIPT_DIR=/share/CACHEDEV3_DATA/hostname-scripts
if [ ! -d ${SCRIPT_DIR}/.run ]; then
    mkdir ${SCRIPT_DIR}/.run
fi
if [ ! -d ${SCRIPT_DIR}/log ]; then
    mkdir ${SCRIPT_DIR}/log
fi
LOGDIR=${SCRIPT_DIR}/log
LOGDATE=$(date +%Y-%m-%d_%H-%M-%S)
LOGFILE_REDIRECT=$LOGDIR/$(basename $0 .sh).${LOGDATE}.html
echo '<BODY><PRE>' > ${LOGFILE_REDIRECT}
exec 3>&1 4>&2 >> ${LOGFILE_REDIRECT} 2>&1
# ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 
# | | | | | | | | | | | | | | | | 
# 2024-04-11: Philip Bondi
# https://stackoverflow.com/questions/25474854/after-using-exec-1file-how-can-i-stop-this-redirection-of-the-stdout-to-file
#
#
PLEX_BACKUP=${SCRIPT_DIR}/backup
SVN_DIR=/share/CACHEDEV2_DATA/svn
MAIL_DIR=/share/CACHEDEV3_DATA/Debian-12-dovecot-Maildir
VM_DIR=/share/CACHEDEV2_DATA/Virtual-Machines
PLEXDATA_DIR=/share/CACHEDEV3_DATA/PlexData/
VM_LOCAL_BACKUP_DIR=/share/CACHEDEV2_DATA/Virtual-Machines-local-backup
V_DOY=$(( $(date +%j) % 4 ))
V_7Z_FILE=${PLEX_BACKUP}/metadata-backup-${V_DOY:-}.7z
# private
VM_DOMAIN_GUID=snipped
VM_DOMAIN_HOSTNAME=vm_hostname
V_ONE_DRIVE_REMOTE="family_member_onedrive"
V_ONE_DRIVE_VM_PATH="${V_ONE_DRIVE_REMOTE:-}:FORTYORK-Virtual-Machines-img-(DO-NOT-ERASE)/"
V_ONE_DRIVE_MAIL_PATH="${V_ONE_DRIVE_REMOTE:-}:FORTYORK-Debian-12-dovecot-Maildir-(DO-NOT-ERASE)/"
V_ONE_DRIVE_SVN_PATH="${V_ONE_DRIVE_REMOTE:-}:FORTYORK-backups-(DO-NOT-ERASE)/"
V_ONE_DRIVE_PLEX_PATH="${V_ONE_DRIVE_REMOTE:-}:FORTYORK-Plex-backups-(DO-NOT-ERASE)/"
V_EMAIL_TO=pjbondi@systemdatabase.com
V_STRONG_PASSWORD=somestrongpassword
V_SVN_REPO_LIST="repo1 repo2 repo3"
#
declare -i V_RETURN_CODE
declare -i V_GLOBAL_RET_CODE
declare -i QTY_NON_ZERO_RET_CODE
QTY_NON_ZERO_RET_CODE=0
if [ ! -d ${PLEX_BACKUP} ]; then
    mkdir ${PLEX_BACKUP}
fi
DAYS_TO_KEEP=90
#
#
#
function svn_backup () {
    for repo in ${V_SVN_REPO_LIST:-}; do
        LOGDATE=$(date +%Y-%m-%d_%H-%M-%S)
        LOGFILE=$LOGDIR/$(basename $0 .sh).${LOGDATE}.${repo}.log
        find ${SVN_DIR}/${repo} -type f ! -perm -o=r | xargs -rt chmod o+rx
        repo_target=$repo
        # Workaround one svn repo that exceeds rclone limit of 150k files as of date hard-coded
        if [ $repo == "svn-sdi" ]; then
            #
            # 2025-08-01: We do not need to re-copy and verify every iteration
            # P_MIN_AGE='--min-age 2020-11-10T09:07:00'
            # rclone -v --log-file ${LOGFILE} ${P_MIN_AGE} copy ${SVN_DIR}/${repo}    "${V_ONE_DRIVE_SVN_PATH:-}${repo_target:-}" 
            # V_RETURN_CODE=$?
            # printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
            # if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
            # cat ${LOGFILE}
            #
            repo_target=${repo}"-GTE150k"
            LOGFILE=$LOGDIR/$(basename $0 .sh).${LOGDATE}.${repo_target}.log
            P_MAX_AGE='--max-age 2020-11-10T09:07:00'
        fi
        rclone -v --log-file ${LOGFILE} ${P_MAX_AGE:-} copy ${SVN_DIR}/${repo}      "${V_ONE_DRIVE_SVN_PATH:-}${repo_target:-}"
        V_RETURN_CODE=$?
        printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
        if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
        cat ${LOGFILE}
        #
        if [[ ${QTY_NON_ZERO_RET_CODE:-} -ne 0 ]]; then
            break
        fi
    done
}
#
#
#
function Plex_backup () {
    $(getcfg -f /etc/config/qpkg.conf  PlexMediaServer Install_path)/plex.sh stop
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
    #
    LOGDATE=$(date +%Y-%m-%d_%H-%M-%S)
    7z a -r -x\!Cache -p${V_STRONG_PASSWORD:-} ${V_7Z_FILE:-} ${PLEXDATA_DIR:-}
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
    #
    LOGDATE=$(date +%Y-%m-%d_%H-%M-%S)
    repo=PlexBackup
    LOGFILE=$LOGDIR/$(basename $0 .sh).${LOGDATE}.${repo}.log
    rclone -v --log-file ${LOGFILE} copy ${V_7Z_FILE:-}    ${V_ONE_DRIVE_PLEX_PATH:-}
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
    #
    $(getcfg -f /etc/config/qpkg.conf  PlexMediaServer Install_path)/plex.sh start
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
    #
    rm ${V_7Z_FILE:-}
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
    #
    #
}
#
#
#
function mbox_backup () {
    repo=Debian-12-dovecot-Maildir
    LOGDATE=$(date +%Y-%m-%d_%H-%M-%S)
    LOGFILE=$LOGDIR/$(basename $0 .sh).${LOGDATE}.${repo}.log
    rclone -v --log-file ${LOGFILE} sync ${MAIL_DIR}                                ${V_ONE_DRIVE_MAIL_PATH:-}  --exclude=.svn/** --exclude=.imap/** --exclude="\@Recently-Snapshot/**" --exclude="\@Recycle/**"
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
    #
    #
}
#
#
#
function online_backup () {
    repo=${VM_DOMAIN_HOSTNAME}

    virsh -v
    #
    LOGDATE=$(date +%Y-%m-%d_%H-%M-%S)
    V_XML=$LOGDIR/$(basename $0 .sh).${LOGDATE}.xml
    virsh dumpxml ${VM_DOMAIN_GUID:-} > ${V_XML}
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); return; fi 
    #
    #
    # https://libvirt.org/kbase/live_full_disk_backup.html
    # 2. Enumerate the disk(s) in use:
    virsh domblklist ${VM_DOMAIN_GUID:-}
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); return; fi 
    #
    V_VDA_IMG=$(virsh domblklist ${VM_DOMAIN_GUID:-} | awk '/vda/{print $2}')
    #
    V_BACKUP_IMG=${VM_LOCAL_BACKUP_DIR:-}/${repo}/$( basename ${V_VDA_IMG/.[0-9]*/} )-rotation-copy-${V_DOY:-}
    #
    if [[ -f ${V_BACKUP_IMG:-} ]]; then
        rm ${V_BACKUP_IMG:-}
        V_RETURN_CODE=$?
        printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
        if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); return; fi 
    fi
    sed -e "s,__FILE__,${V_BACKUP_IMG:-}," ${SCRIPT_DIR}/qnap-backup-1.xml > ${SCRIPT_DIR}/qnap-backup.xml
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); return; fi 
    #
    cat ${SCRIPT_DIR}/qnap-backup.xml
    #
    #
    # https://libvirt.org/kbase/live_full_disk_backup.html
    # 3. Begin the backup:
    virsh backup-begin ${VM_DOMAIN_GUID:-} ${SCRIPT_DIR}/qnap-backup.xml
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); return; fi 
    #
    # 2025-08-03: https://libvirt.org/kbase/live_full_disk_backup.html
    while true; do
        sleep 30
        # https://libvirt.org/kbase/live_full_disk_backup.html
        # 4. Check the job status ("None" means the job has likely completed):
        virsh domjobinfo ${VM_DOMAIN_GUID:-}
        virsh domjobinfo ${VM_DOMAIN_GUID:-} --completed --keep-completed
        V_VIRSH_STATUS=$(virsh domjobinfo ${VM_DOMAIN_GUID:-} --completed|awk '$1~/Job/{print $3}')
        if [[ ${V_VIRSH_STATUS:-} == "Completed" ]]; then
            break
        fi
    done
    #
    # https://libvirt.org/kbase/live_full_disk_backup.html
    # 5. Check the completed job status:
    virsh domjobinfo ${VM_DOMAIN_GUID:-} --completed
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
    #
    #
    LOGDATE=$(date +%Y-%m-%d_%H-%M-%S)
    LOGFILE=$LOGDIR/$(basename $0 .sh).${LOGDATE}.${repo}.log
    rclone -v --log-file ${LOGFILE} copy ${V_BACKUP_IMG:-}    "${V_ONE_DRIVE_VM_PATH:-}${repo:-}"
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
    #
    virsh domblklist ${VM_DOMAIN_GUID:-}
    V_RETURN_CODE=$?
    printf "V_RETURN_CODE: %d\n" ${V_RETURN_CODE:-}
    if [[ ${V_RETURN_CODE:-} -ne 0 ]]; then QTY_NON_ZERO_RET_CODE=$((QTY_NON_ZERO_RET_CODE+1)); fi 
    #
    # https://libvirt.org/kbase/live_full_disk_backup.html
    # 6. Now we see the copy of the backup:
    ls -alF ${VM_LOCAL_BACKUP_DIR:-}/${repo}
    #
}
#
#
#
LOCKFILE=${SCRIPT_DIR}/.run/$(basename $0).lock
if ( set -o noclobber; echo "$$" > "$LOCKFILE" ) 2> /dev/null; then
    trap 'rm -f "$LOCKFILE"; echo "trap completed"' INT TERM EXIT
    #
    if [[ ${QTY_NON_ZERO_RET_CODE:-} -eq 0 ]]; then
        online_backup
    fi
    if [[ ${QTY_NON_ZERO_RET_CODE:-} -eq 0 ]]; then
        Plex_backup
    fi
    if [[ ${QTY_NON_ZERO_RET_CODE:-} -eq 0 ]]; then
        mbox_backup
    fi
    if [[ ${QTY_NON_ZERO_RET_CODE:-} -eq 0 ]]; then
        svn_backup
    fi
    #
    find $LOGDIR -type f -mtime +${DAYS_TO_KEEP} -print0 | xargs -0 -t rm -f 
else
    echo "Failed to acquire lockfile: $LOCKFILE." 
    echo "Held by PID $(cat $LOCKFILE)"
    #
    echo "CHILD_RET_CODE:1"
    #
    exec 1>&3 3>&-
    # ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 
    # | | | | | | | | | | | | | | | | 
    # 2024-04-11: Philip Bondi
    # https://stackoverflow.com/questions/25474854/after-using-exec-1file-how-can-i-stop-this-redirection-of-the-stdout-to-file
    exit 1
fi
#
echo "\$QTY_NON_ZERO_RET_CODE: " ${QTY_NON_ZERO_RET_CODE:-}
if [[ ${QTY_NON_ZERO_RET_CODE:-} -ne 0 ]]; then
    echo '00000000000000000000000000000000000000000000000000000000000000000000000000000000'
    echo '00000000000000000000000    ERROR. Execution is halted.   00000000000000000000000'
    echo '00000000000000000000000000000000000000000000000000000000000000000000000000000000'
fi
#
exec 1>&3 3>&- 2>&4 4>&-
# ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 
# | | | | | | | | | | | | | | | | 
# 2024-04-11: Philip Bondi
# https://stackoverflow.com/questions/25474854/after-using-exec-1file-how-can-i-stop-this-redirection-of-the-stdout-to-file
#
#
#
echo '</PRE></BODY>' >> ${LOGFILE_REDIRECT}
V_SUBJECT="$(basename $0) \$QTY_NON_ZERO_RET_CODE: ${QTY_NON_ZERO_RET_CODE:-} on $(hostname)"
mutt -s "$V_SUBJECT" $V_EMAIL_TO -a ${LOGFILE_REDIRECT} -- << EOF
$V_SUBJECT
#          \$Id: qnap-backup.sh 167775 2025-09-07 21:30:25Z svn_beechwood $
EOF

SteveKo · 2025 年 9 月 10 日午前 9:54

共有していただきありがとうございます。コミュニティ内でこれほど多くのさまざまなアプリケーションが共有されているのを見て、とても嬉しく思います。

BuddyCat · 2025 年 9 月 10 日午前 10:17

情報ありがとうございます！ちょっと気になったのですが、Plex（プレックス）のバックアップで何か問題は発生しましたか？すべて順調に動作していますか？

pjbondi · 2025 年 9 月 10 日午後 12:25

はい。このスクリプトは数週間にわたって自宅サーバーで稼働しており、私はほぼ毎日Plexサーバーを利用しています。Plexのメタデータバックアップはオフラインバックアップです。幸いなことに、バックアップのためにオフラインになる午前2時30分に映画を観たくなったことはありません。

このスクリプトは継続的にリファクタリングと改良を重ねてきました。また、RedHatのスタッフがvirshフォーラムで私の質問に答えてくれました。コードを見れば分かる通り、非常に詳細なスプールファイルを提供しています。そのため、実行前後のbashデバッグを調査し、（うまくいけば）素早く問題を特定し解決することができます。私は10代になる前からプログラミングをしており、もう45年以上になります。コミュニティへの貢献ができて嬉しいです。