diff --git a/IMAP-Sync.md b/IMAP-Sync.md index f78b64c..d825ce1 100644 --- a/IMAP-Sync.md +++ b/IMAP-Sync.md @@ -1,9 +1,22 @@ -Sync email via IMAP from host1/domain1 to a subfolder on host2/domain2 via a cron/timer. Can be reversed as well, just update `Patterns` to exclude the subfolders from being cross-replicated (looped). +# IMAP Sync + +Sync email via IMAP from `host1/domain1` to a subfolder on `host2/domain2` via a cron/timer. Can be reversed as well, just update `Patterns` to exclude the subfolders from being cross-replicated (looped). + +Sync email via IMAP from `host1/domain1` to a local `Maildir` folder via cront/timer. This can be used to restore a backup to either `host1/domain1` or `host2/domain2` as well as by a local MUA (mutt, e.g.). + +Note: an argument can be made it's more effecient to sync `host1/domain1` to a local `Maildir` first, then sync the local `Maildir` to `host2/domain2`. This however comes at a cost of risk - what if the local drive is full, fails or has an error? One runs the risk of using a possible corrupt local datastore to sync to `host2/domain2` when the intent is to have multiple verbatim backups of `host1/domain1`. Choose wisely. + +## Preparation + +Documentation: - https://isync.sourceforge.io/mbsync.html + +Debian package: + - Install the `isync` package: `apt-get update && apt-get install isync` -Passwords for IMAP must be left on disk in plain text +Passwords for IMAP must be left on disk in plain text: - Generate "app passwords" at the email providers, host1 can be READ only - Keep `${HOME}/.secure` contents on encrypted volume unlocked manually @@ -12,12 +25,17 @@ The `mbsync` program keeps it's transient index files in `${HOME}/.mbsync/` with By design, `mbsync` will not delete a destination folder if it's not empty first; this means if you delete a folder and all emails on the source in one step, a sync will break with an error/warning. Instead, delete all emails in the folder first, sync those deletions, then delete the empty folder on the source and sync again. See: https://sourceforge.net/p/isync/mailman/isync-devel/thread/f278216b-f1db-32be-fef2-ccaeea912524%40ojkastl.de/#msg37237271 -Simple crontab to run the script: +## Crontabs + +Simple crontabs to run the scripts, 30 minutes apart: ``` 0 */6 * * * /home/USER/bin/hasync.sh +30 */6 * * * /home/USER/bin/halocal.sh ``` +## Config + Main config for the `mbsync` program: **${HOME}/.mbsyncrc** @@ -44,6 +62,12 @@ SystemCertificates yes PipeLineDepth 1 #CertificateFile /etc/ssl/certs/ca-certificates.crt +# Local +MaildirStore local-dest +Path /home/USER/Maildir/ +AltMap no +Subfolders Verbatim + # Source map IMAPStore imap-src Account imap-src-account @@ -62,9 +86,22 @@ Remove Near Expunge Near Patterns * CopyArrivalDate yes + +# Local options +Channel dmlocal +Far :imap-src: +Near :local-dest: +Sync Pull +Create Near +Remove Near +Expunge Near +Patterns * +CopyArrivalDate yes ``` -This script leverages healthchecks.io to alert on failure; replace XXXXX with the UUID of your monitor URL. +## Scripts + +These scripts leverage https://healthchecks.io to alert on failure; replace XXXXX with the UUID of your monitor URL. **${HOME}/bin/hasync.sh** ```bash @@ -73,7 +110,7 @@ This script leverages healthchecks.io to alert on failure; replace XXXXX with th # vars LOGDIR="${HOME}/log" TIMESTAMP=$(date +%Y-%m-%d_%H%M) -LOGFILE="${LOGDIR}/mbsync_${TIMESTAMP}.log" +LOGFILE="${LOGDIR}/hasync_${TIMESTAMP}.log" HCPING="https://hc-ping.com/XXXXXXXXXXXXXXXXXXXXXXXXX" # preflight @@ -91,7 +128,38 @@ echo -e "\nEND $(date +%Y-%m-%d_%H%M)\n" >> "${LOGFILE}" # report if [[ $EC -eq 0 ]]; then curl -fsS -m 10 --retry 5 -o /dev/null "${HCPING}" - find "${LOGDIR}" -type f -mtime +30 -delete + find "${LOGDIR}" -type f -name hasync\* -mtime +30 -delete +fi + +exit $EC +``` + +**${HOME}/bin/halocal.sh** +```bash +#!/bin/bash + +# vars +LOGDIR="${HOME}/log" +TIMESTAMP=$(date +%Y-%m-%d_%H%M) +LOGFILE="${LOGDIR}/halocal_${TIMESTAMP}.log" +HCPING="https://hc-ping.com/XXXXXXXXXXXXXXXXXXXXXXXXX" + +# preflight +if [[ ! -d "${LOGDIR}" ]]; then + mkdir -p "${LOGDIR}" +fi + +# sync +echo -e "\nBEGIN $(date +%Y-%m-%d_%H%M)\n" >> "${LOGFILE}" +/usr/bin/mbsync -c ${HOME}/.mbsyncrc -V halocal 1>>"${LOGFILE}" 2>&1 +EC=$? +echo -e "\nEC: ${EC}" >> "${LOGFILE}" +echo -e "\nEND $(date +%Y-%m-%d_%H%M)\n" >> "${LOGFILE}" + +# report +if [[ $EC -eq 0 ]]; then + curl -fsS -m 10 --retry 5 -o /dev/null "${HCPING}" + find "${LOGDIR}" -type f -name halocal\* -mtime +30 -delete fi exit $EC