Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .coderabbit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ reviews:
- Escape anything echoed into HTML or inline script. Config is saved to `/boot`, so saved values from older versions must keep working.
- File names, ZFS user properties and anything from another server may not be valid UTF-8. htmlspecialchars() needs ENT_SUBSTITUTE, or it returns '' and the text disappears. json_encode() needs JSON_INVALID_UTF8_SUBSTITUTE, or its false result must be handled; never echo, publish or hash a json_encode() that can fail.
- Every `zfs receive` takes `-s -u -o readonly=on` and never `-F`. A copy of a volume must not appear as a block device (volmode=none).
- Replicate Every Dataset (include/ZFSMScheduleCopies.php) makes an empty holder (`zfs create -o canmount=off -o readonly=on -o zfs.master:holder=on`) only where a copy's parent is missing. The property inherits, so a dataset is a holder only where its source is `local` (zfsmReplicaHolder).
- path: "zfs.master/{include/ZFSMDiag*.php,scripts/diagnostics.php}"
instructions: |
Diagnostics: a zip the owner may post in a public issue. Its privacy scope is fixed; report gaps inside it and nothing past it.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ See and manage your ZFS pools, datasets and snapshots from Unraid's **Main** tab
- **Snapshots** — take, roll back, rename, clone, hold, release and delete; an icon warns when the newest gets old
- **Snapshots Admin** — filter, sort, browse a snapshot's files, or see what changed since it
- **Snapshot schedules** — snapshot every dataset hourly or on chosen days and keep the newest, plus the newest of each of the last days, weeks, months or years you choose; a dataset can have its own (not a read-only, legacy-mounted or Sanoid/Syncoid one), or none
- **Replication** — copy datasets to another pool or ZFS Master NG server, once or on a schedule
- **Replication** — copy datasets to another pool or ZFS Master NG server, once or on a schedule, or every dataset into one dataset here on a schedule (Replicate Every Dataset)
- **Reviews** — destroy, rollback and bulk delete show what they'll remove first
- **Encryption** — create encrypted datasets, lock and unlock them
- **Directories** — list, move or delete folders, or convert one to a dataset (cloned instead of copied where it can)
Expand Down
10 changes: 8 additions & 2 deletions tests/admin_route_test.php
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,12 @@
'schedulesnapshotstiers' => array('cmd' => 'schedulesnapshots', 'every' => 'hourly', 'time' => '', 'days' => '', 'keep' => ' 24 7D, 4w '),
'schedulesnapshotswrong' => array('cmd' => 'schedulesnapshots', 'every' => 'days', 'time' => '', 'days' => '9', 'keep' => 'x5'),
'schedulesnapshotsown' => array('cmd' => 'schedulesnapshots', 'zdataset' => 'tank/data', 'every' => 'days', 'time' => '04:00', 'days' => '1', 'keep' => ''),
'schedulecopies' => array('cmd' => 'schedulecopies', 'under' => 'backup', 'every' => 'hourly', 'time' => '', 'days' => '', 'keep' => '3'),
'schedulecopiesnowhere' => array('cmd' => 'schedulecopies', 'under' => 'nowhere', 'every' => 'hourly', 'time' => '', 'days' => '', 'keep' => ''),
'previewschedules' => array('cmd' => 'previewschedules', 'zdataset' => 'tank/data'),
'previewschedulesbad' => array('cmd' => 'previewschedules', 'zdataset' => 'tank/bad name!'),
);
$replication = array('previewreplicate', 'replicatedataset', 'schedulepicked', 'schedulenowhere', 'schedulekeep', 'schedulesnapshots', 'schedulesnapshotstiers', 'schedulesnapshotswrong', 'schedulesnapshotsown', 'previewschedules', 'previewschedulesbad');
$replication = array('previewreplicate', 'replicatedataset', 'schedulepicked', 'schedulenowhere', 'schedulekeep', 'schedulesnapshots', 'schedulesnapshotstiers', 'schedulesnapshotswrong', 'schedulesnapshotsown', 'schedulecopies', 'schedulecopiesnowhere', 'previewschedules', 'previewschedulesbad');

$route = $argv[1] ?? '';
if (!isset($routes[$route])) exit(2);
Expand All @@ -59,7 +61,7 @@

// A schedule to remove, or a snapshot schedule saved (the default or a dataset's own), on the tmpfs the runner mounts on /boot/config. No update_cron here, so
// Unraid's crontab is not rebuilt.
if ($route === 'scheduleremovesaved' || $route === 'schedulesnapshots' || $route === 'schedulesnapshotstiers' || $route === 'schedulesnapshotsown') {
if ($route === 'scheduleremovesaved' || $route === 'schedulesnapshots' || $route === 'schedulesnapshotstiers' || $route === 'schedulesnapshotsown' || $route === 'schedulecopies') {
zfsmTestScratch('/boot/config');
putenv('ZFSM_TEST_JOB_DIR='.sys_get_temp_dir().'/zfsm-route-jobs-'.getmypid());
@mkdir('/boot/config/plugins/zfs.master', 0700, true);
Expand Down Expand Up @@ -114,6 +116,10 @@
'schedulesnapshotsown' => ($answer['said'] ?? '') === 'Saved: a snapshot of each of 1 dataset(s) (tank/data and below) every Monday at 04:00; it keeps all it takes'
&& strpos($answer['schedules_html'] ?? '', '>Snapshots of tank/data and below, every Monday at 04:00, keeps all it takes <input type="button" value="Remove"') !== false
&& array_column(json_decode(file_get_contents('/boot/config/plugins/zfs.master/schedules.json'), true), 'dataset') === array('tank/data'),
'schedulecopies' => preg_match('/^Saved: a copy of each of 1 dataset\(s\) under backup every hour at :[0-5][0-9]; it keeps the newest 3 of the snapshots of each dataset it copies$/D', $answer['said'] ?? '') === 1
&& strpos($answer['schedules_html'] ?? '', '>Replicate Every Dataset under backup, every hour at :') !== false
&& array_column(json_decode(file_get_contents('/boot/config/plugins/zfs.master/schedules.json'), true), 'dest') === array('backup/'),
'schedulecopiesnowhere' => $answer === array('error' => 'Nothing was saved: nowhere does not exist on this server') && !is_file('/boot/config/plugins/zfs.master/schedules.json'),
'previewschedules' => $answer === array('every' => 'default', 'time' => '00:00', 'days' => range(0, 6), 'keep' => '', 'inherited' => 'Snapshot Every Dataset, which is off: no scheduled snapshots', 'skipped' => ''),
'previewschedulesbad' => ($answer['failed'] ?? null) === array('request' => 'Invalid dataset name'),
)[$route];
Expand Down
13 changes: 12 additions & 1 deletion tests/diagnostics_bundle_test.php
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,10 @@ function has($needle, $text, $message) {
'Schedule 0a1b2c3e: Scheduled snapshots of tank/CanaryOwnSan not taken: tank/CanaryOwnSan has Sanoid or Syncoid snapshots (autosnap_, syncoid_), so it is left to that tool; remove its schedule in Settings',
'Schedule 0a1b2c3e: Scheduled snapshots of 1 of 1 dataset(s) (tank/CanaryOwnSelf and below) taken. Not taken: tank/CanaryOwnSelf has Sanoid or Syncoid snapshots (autosnap_, syncoid_), so it is left to that tool',
'Schedule 0a1b2c3e: Scheduled snapshots of tank/CanaryOwnOff not taken: its pool is not imported',
// Replicate Every Dataset: how many it copied, those waiting for the copy above them, a holder it could not make, one in a copy's place.
'Schedule 0a1b2c40: Scheduled copies under tank/CanaryUnder: 1 of 8 dataset(s) copied. Not copied, as the copy of a dataset above it is missing (tank/CanaryWaitA, tank/CanaryWait B, tank/CanaryWaitC, and 2 more). '
."The dataset to hold the copies below tank/CanaryUnder/tank/CanaryHold could not be made: cannot create 'tank/CanaryUnder/tank/CanaryHold': out of space",
'Replication of tank/CanaryCpSrc to tank/CanaryUnder/tank/CanaryCpSrc not started: tank/CanaryUnder/tank/CanaryCpSrc only holds the copies of datasets below this one, made while it was not copied; its own copy cannot go there without destroying them',
'ZFS Dataset Rename: Snapshot schedules of tank/CanaryMvFrom failed: not moved to tank/CanaryMvTo (Another change to the schedules is being saved; try again): Remove them in Settings and set them again under Actions, Schedules',
// A schedule's Keep: what it removed, what stayed and why, and holds a run could not put on or take off.
'Removed tank/CanaryDs One@CanaryKeepOld (not among the newest 3, 7 daily and 4 weekly its schedule keeps)',
Expand Down Expand Up @@ -306,6 +310,13 @@ function has($needle, $text, $message) {
has($own.'of 1 of 1 dataset(s) (tank/'.$token('CanaryOwnSelf').' and below) taken. Not taken: tank/'.$token('CanaryOwnSelf')." has Sanoid or Syncoid snapshots (autosnap_, syncoid_), so it is left to that tool\n",
$log, 'and one that took those below but not its own');
has($own.'of tank/'.$token('CanaryOwnOff')." not taken: its pool is not imported\n", $log, 'of one whose pool is not imported');
$under = 'tank/'.$token('CanaryUnder');
// Below Copy Under a copy's path names its pool again, as a dataset there: a token too.
$cp = $under.'/'.zfsmDiagToken('tank').'/'.$token('CanaryCpSrc');
$hold = $under.'/'.zfsmDiagToken('tank').'/'.$token('CanaryHold');
has('Schedule '.zfsmDiagToken('0a1b2c40').": Scheduled copies under $under: 1 of 8 dataset(s) copied. Not copied, as the copy of a dataset above it is missing (tank/".$token('CanaryWaitA').', tank/'.$token('CanaryWait B').', tank/'.$token('CanaryWaitC').', and 2 more). '
."The dataset to hold the copies below $hold could not be made: cannot create ?$hold?: out of space\n", $log, 'a run of copies under a dataset: each name a token, the plugin\'s words in clear');
has('Replication of tank/'.$token('CanaryCpSrc')." to $cp not started: $cp only holds the copies of datasets below this one, made while it was not copied; its own copy cannot go there without destroying them\n", $log, 'and one in a copy\'s place');
has('ZFS Dataset Rename: Snapshot schedules of tank/'.$token('CanaryMvFrom').' failed: not moved to tank/'.$token('CanaryMvTo').' (Another change to the schedules is being saved; try again): Remove them in Settings and set them again under Actions, Schedules'."\n",
$log, 'a rename whose schedules were not moved');
// Keep Replicas on a copy received here, which the next receive tries again.
Expand Down Expand Up @@ -438,7 +449,7 @@ function has($needle, $text, $message) {
$plugins_own = '/boot/config/plugins/ /boot/config/plugins/zfs.master/zfs.master-2026.11.11.3-x86_64-1.txz /var/log/syslog.2 /var/run/zfs.master.rewrite/123.lock /tmp/zfs.master-diag.0123456789abcdef/plugin/log.txt /boot/logs'
.' /var/log/packages/zfs.master-2026.11.11.3-x86_64-1 /var/run/zfs.master.alerts.json.lock /var/run/zfs.master.diagnostics/pack.lock'
.' /boot/config/plugins/zfs.master/ssh/remotes.json /boot/config/plugins/zfs.master/ssh/new-0123abcd.pub /var/run/zfs.master.replica-send/settings.lock /var/run/zfs.master.replica-send/ssh-0123456789ab/known_hosts'
.' /var/run/zfs.master.replica-send/ssh-0123456789ab/cm /boot/config/plugins/zfs.master/schedules.json /boot/config/plugins/zfs.master/schedules.cron /var/run/zfs.master.schedule/0a1b2c3d.json /var/run/zfs.master.schedule/settings.lock';
.' /var/run/zfs.master.replica-send/ssh-0123456789ab/cm /boot/config/plugins/zfs.master/schedules.json /boot/config/plugins/zfs.master/schedules.cron /var/run/zfs.master.schedule/0a1b2c3d.json /var/run/zfs.master.schedule/0a1b2c3d.lock /var/run/zfs.master.schedule/settings.lock';
same(explode(' ', $plugins_own), array_map('zfsmDiagAnonText', explode(' ', $plugins_own)), 'the plugin\'s own paths stay, each a field of its own');
// In a row a field is the whole path: one of the plugin's names with a space and more after it is a folder's, not the plugin's.
same('/var/'.zfsmDiagToken('run').'/'.zfsmDiagToken('zfs.master.rewrite ClientA')."\tlocal", zfsmDiagAnonText("/var/run/zfs.master.rewrite ClientA\tlocal"), 'a field that only starts with one of the plugin\'s names is a folder\'s');
Expand Down
25 changes: 18 additions & 7 deletions tests/diagnostics_test.php
Original file line number Diff line number Diff line change
Expand Up @@ -427,19 +427,19 @@ function has($needle, $text, $message) {
putenv('ZFSM_TEST_REPL_DIR='.$work.'/pool');
list($findings, $text) = $found('zfsmDiagReplicaSchedules', $snapped);
same(array(array(), false), array($findings, $GLOBALS['zfsm_diag']['unread']), 'Snapshot Every Dataset with its lines in both has no findings');
has("records in schedules.json the plugin does not use: 0\ndatasets each snapshot schedule takes now: 0b0b0b0b 2\n\n", $text, 'how many datasets it takes, read-only ones left out');
has("records in schedules.json the plugin does not use: 0\ndatasets each schedule of snapshots or of copies under a dataset covers now: 0b0b0b0b 2\n\n", $text, 'how many datasets it takes, read-only ones left out');
has("\n0b0b0b0b\t(every dataset)\t(snapshots only)\tevery hour at :30\t2\t0\n", $text, 'its row');
putenv('ZFSM_TEST_COVER_FAIL=1');
$text = $found('zfsmDiagReplicaSchedules', $snapped)[1];
putenv('ZFSM_TEST_COVER_FAIL');
same(true, $GLOBALS['zfsm_diag']['unread'], 'a count that can\'t be read marks the file (zfsmDiagRead)');
has("datasets each snapshot schedule takes now: (could not be read: cannot iterate filesystems: Input/output error)\n", $text, 'and says why');
has("datasets each schedule of snapshots or of copies under a dataset covers now: (could not be read: cannot iterate filesystems: Input/output error)\n", $text, 'and says why');
putenv('ZFSM_TEST_COVER_FAIL=tank/data');
$text = $found('zfsmDiagReplicaSchedules', $snapped)[1];
putenv('ZFSM_TEST_COVER_FAIL');
has("datasets each snapshot schedule takes now: 0b0b0b0b 1 (not every dataset could be read: cannot open 'tank/data': Input/output error)\n", $text, 'one it can\'t read: the rest counted, and it named');
has("datasets each schedule of snapshots or of copies under a dataset covers now: 0b0b0b0b 1 (not every dataset could be read: cannot open 'tank/data': Input/output error)\n", $text, 'one it can\'t read: the rest counted, and it named');
file_put_contents($work.'/pool/marks', "tank/data@autosnap_2026-11-17_00:00:01_hourly\t1\t1\n");
has("datasets each snapshot schedule takes now: 0b0b0b0b 1\n", $found('zfsmDiagReplicaSchedules', $snapped)[1], 'one with Sanoid\'s snapshots left out');
has("datasets each schedule of snapshots or of copies under a dataset covers now: 0b0b0b0b 1\n", $found('zfsmDiagReplicaSchedules', $snapped)[1], 'one with Sanoid\'s snapshots left out');
file_put_contents($work.'/pool/marks', '');
// A dataset's own (Main, Schedules) and a None: each row named, each count apart; None has no cron line to miss, and is not counted.
$own = array('id' => '0c0c0c0c', 'dataset' => 'tank/data', 'minute' => 45) + $every;
Expand All @@ -450,8 +450,19 @@ function has($needle, $text, $message) {
file_put_contents($root.'/etc/cron.d/root', zfsmScheduleCronText(array($every, $own, $none)), FILE_APPEND);
});
same(array(), $findings, 'own schedules and a None with their lines have no findings');
has("datasets each snapshot schedule takes now: 0b0b0b0b 0, 0c0c0c0c 1\n", $text, 'each counts only the datasets that follow it');
has("datasets each schedule of snapshots or of copies under a dataset covers now: 0b0b0b0b 0, 0c0c0c0c 1\n", $text, 'each counts only the datasets that follow it');
same(1, preg_match('~^0c0c0c0c\t(name-[0-9a-f]{8})/name-[0-9a-f]{8}\t\(snapshots only\)\tevery hour at :45\t2\t0\n0d0d0d0d\t\1\t\(snapshots only\)\tnever\t0\t0$~m', $text), 'their rows, the datasets as tokens');
// Replicate Every Dataset and a dataset's None of copies: rows named, where the copies go; counted apart from the snapshot schedule's.
$copies = array('id' => '0e0e0e0e', 'dest' => 'backup/', 'minute' => 50, 'keep' => 0) + $every;
$nocopy = array('id' => '0f0f0f0f', 'dataset' => 'tank/data', 'dest' => '/', 'every' => 'none', 'minute' => 0, 'keep' => 0) + $every;
list($findings, $text) = $found('zfsmDiagReplicaSchedules', function($root) use ($every, $copies, $nocopy) {
file_put_contents($root.'/boot/config/plugins/zfs.master/schedules.json', json_encode(array($every, $copies, $nocopy)));
file_put_contents($root.'/boot/config/plugins/zfs.master/schedules.cron', zfsmScheduleCronText(array($every, $copies, $nocopy)));
file_put_contents($root.'/etc/cron.d/root', zfsmScheduleCronText(array($every, $copies, $nocopy)), FILE_APPEND);
});
same(array(), $findings, 'Replicate Every Dataset and a None with their lines have no findings');
has("datasets each schedule of snapshots or of copies under a dataset covers now: 0b0b0b0b 2, 0e0e0e0e 1\n", $text, 'each counts its own: the copies leave out the dataset with None');
same(1, preg_match('~^0e0e0e0e\t\(every dataset\)\tname-[0-9a-f]{8}/ \(each dataset under it\)\tevery hour at :50\t0\t0\n0f0f0f0f\tname-[0-9a-f]{8}/name-[0-9a-f]{8}\t\(no copies\)\tnever\t0\t0$~m', $text), 'their rows, where the copies go a token');
// A dataset only the default takes can't be read: a dataset's own after it, which never takes it, doesn't hide that.
putenv('ZFSM_TEST_COVER_FAIL=backup/copy');
$text = $found('zfsmDiagReplicaSchedules', function($root) use ($every, $own) {
Expand All @@ -460,7 +471,7 @@ function has($needle, $text, $message) {
file_put_contents($root.'/etc/cron.d/root', zfsmScheduleCronText(array($every, $own)), FILE_APPEND);
})[1];
putenv('ZFSM_TEST_COVER_FAIL');
has("datasets each snapshot schedule takes now: 0b0b0b0b 1, 0c0c0c0c 1 (not every dataset could be read: cannot open 'backup/copy': Input/output error)\n", $text, 'the default\'s unread dataset, with an own one after it');
has("datasets each schedule of snapshots or of copies under a dataset covers now: 0b0b0b0b 1, 0c0c0c0c 1 (not every dataset could be read: cannot open 'backup/copy': Input/output error)\n", $text, 'the default\'s unread dataset, with an own one after it');
// None never runs to clear a count a run left as it was saved: not found, its row 0; the own one's count still shows.
list($findings, $text) = $found('zfsmDiagReplicaSchedules', function($root) use ($every, $own, $none, $work) {
file_put_contents($root.'/boot/config/plugins/zfs.master/schedules.json', json_encode(array($every, $own, $none)));
Expand Down Expand Up @@ -578,7 +589,7 @@ public function url_stat($path, $flags) { return self::real(function() use ($pat
putenv('ZFSM_TEST_LIST_HANG');
same(array('plugin/replication-schedules.txt', 'timed out'), array_slice(explode("\t", file($work.'/hung-schedules/collectors.txt', FILE_IGNORE_NEW_LINES)[1]), 0, 2), 'the count\'s read times out');
$text = file_get_contents($work.'/hung-schedules/plugin/replication-schedules.txt');
has("datasets each snapshot schedule takes now: (could not be read: zfs timed out)\n", $text, 'and says so');
has("datasets each schedule of snapshots or of copies under a dataset covers now: (could not be read: zfs timed out)\n", $text, 'and says so');
has("\n0b0b0b0b\t(every dataset)\t(snapshots only)\tevery hour at :30\t2\t0\n", $text, 'with the rest of the file');

// A read its collector's limit cut short says nothing about the pool; one that ran its whole limit does.
Expand Down
11 changes: 11 additions & 0 deletions tests/fixtures/bin/zfs
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,15 @@ if [ -n "${ZFSM_TEST_REPL_DIR:-}" ]; then
exit 0 ;;
esac
exists "$last_arg" || { printf "cannot open '%s': dataset does not exist\n" "$last_arg" >&2; exit 1; }
# -o source: local where it was set, else inherited from the nearest dataset above that set it, as a user property is.
if [ "$5" = source ]; then
d=$last_arg
while :; do
awk -F'\t' -v n="$d" -v p="$6" '$1 == n && $2 == p { f = 1 } END { exit !f }' "$R/props" && { if [ "$d" = "$last_arg" ]; then echo local; else echo "inherited from $d"; fi; exit 0; }
[ "${d%/*}" = "$d" ] && { echo -; exit 0; }
d=${d%/*}
done
fi
if [ "$5" = property,value ]; then for p in $(printf '%s' "$6" | tr , ' '); do printf '%s\t%s\n' "$p" "$(prop "$last_arg" "$p" -)"; done; exit 0; fi
case "$6" in
type) awk -F'\t' -v n="$last_arg" '$1 == n { print $2 }' "$R/datasets" ;;
Expand Down Expand Up @@ -148,6 +157,8 @@ if [ -n "${ZFSM_TEST_REPL_DIR:-}" ]; then
snap=$(printf '%s' "$header" | cut -f2); base=$(printf '%s' "$header" | cut -f3); guid=$(printf '%s' "$header" | cut -f4)
[ "${ZFSM_TEST_RECV_FAIL:-0}" = changes ] && [ "$base" != - ] && [ "$base" != RESUME ] && { printf 'cannot receive incremental stream: out of space\n' >&2; exit 1; }
if [ "$base" = - ] && exists "$dest" && [ "$(prop "$dest" receive_resume_token -)" = - ]; then printf "cannot receive new filesystem stream: destination '%s' exists\n" "$dest" >&2; exit 1; fi
# zfs makes a new copy only inside a dataset that exists.
exists "$dest" || exists "${dest%/*}" || { printf "cannot receive new filesystem stream: parent of '%s' does not exist\n" "$dest" >&2; exit 1; }
if [ "$base" != - ] && [ "$base" != RESUME ] && [ "$(newest "$dest")" != "$(mark "$base" 2)" ]; then
printf 'cannot receive incremental stream: most recent snapshot of %s does not match incremental source\n' "$dest" >&2; exit 1
fi
Expand Down
Loading
Loading