Tiered media storage with rclone and mergerfs

A Pulsed Media storage box turns your seedbox into something much larger than its own disk. Keep the fast seedbox as a hot tier for active downloads and streaming, keep the cheap storage box as a cold archive, and mount the two together so Jellyfin, Sonarr, Radarr and your torrent client see one library and never care which tier a file sits on. Traffic between a Pulsed Media seedbox and a Pulsed Media storage box in the same datacenter is unmetered at up to 10Gbps, so pulling a file back from the cold tier costs you nothing.

Everything on this page runs as a normal seedbox user. No root. The setup was built and tested by Pulsed Media community member StupidGenius on a live seedbox and storage box pair; the pitfalls below are the things that actually went wrong along the way.

Two setups, in order of effort:

Just streaming? Cache the storage box for streaming — one rclone mount, five minutes. Point Jellyfin at it and watch.
Want a real hot/cold library? Build a tiered library — a union filesystem plus a mover script, so new downloads land on the fast disk and age out to the storage box automatically.

How it works

apps see ONE folder ─────────────►  ~/media/{Movies,TV,Music}
                                          ▲
                     union  =  ~/media-local   (hot tier: seedbox disk)
                              + ~/mnt/storage   (cold tier: rclone SFTP mount)
                                          ▲
                                    storagebox:media   (your storage box)
                                          ▲
                     mover ships aged files hot ──► cold on a schedule

New writes land on the hot tier and seed at full local speed. A scheduled mover ships older files to the storage box. Reads fall through to whichever tier holds the file. The apps only ever use the one merged path, so nothing they store ever has to change.

Before you start

You need an active seedbox and a storage box, and SSH access to the seedbox. Any seedbox tier works; the SSD and NVMe tiers gain the most, because the whole point is keeping hot data on fast disk.

Know your quota, not df. On a shared seedbox df -h shows the whole underlying filesystem, not your slice. Your real limit comes from:

quota -s

Always pass -s. Without it, quota can report raw kilobyte block counts that look like gigabyte numbers and quietly break any threshold script that reads them.

Replace these placeholders throughout: USERNAME (your seedbox username), STORAGE.pulsedmedia.com (your storage box hostname), PORT (its SSH port).

Cache a storage box for streaming

The simplest use: mount the storage box over SFTP with an rclone VFS cache on the seedbox's fast disk, and stream straight from it. The bare mount shown on the Jellyfin page works, but it reads directly from the remote with no local cache, so seeks stutter and a dropped read restarts the file. For a media server, use --vfs-cache-mode full plus the read-ahead and permission flags the short command leaves out. In full mode rclone buffers reads and writes to the cache directory as sparse files, keeps only the parts of each file you actually touch, retries failed reads, and lets the player seek anywhere.

# One-time: let a service user (e.g. jellyfin) see the mount
sudo sed -i 's/^#user_allow_other/user_allow_other/' /etc/fuse.conf   # or edit the file by hand

mkdir -p ~/mnt/storage ~/.cache/rclone-storage

rclone mount storagebox:/media ~/mnt/storage \
  --vfs-cache-mode full \
  --cache-dir ~/.cache/rclone-storage \
  --vfs-cache-max-size 50G \
  --vfs-cache-max-age 24h \
  --vfs-cache-min-free-space 10G \
  --vfs-read-ahead 1G \
  --buffer-size 64M \
  --dir-cache-time 1h \
  --allow-other \
  --uid $(id -u jellyfin) --gid $(id -g jellyfin) --umask 002 \
  --daemon

The flags the short command omits are the ones that matter for smooth playback:

Flag	What it does
`--vfs-read-ahead 1G`	Reads ahead onto the cache disk beyond the in-memory buffer. The single biggest smoothness win for streaming — set it large (1–2G for 4K remuxes) and keep `--buffer-size` modest so RAM does not blow out on a 4GB box.
`--vfs-cache-min-free-space 10G`	Stops the cache from filling the disk. Without it, only `--vfs-cache-max-size` applies, and the cache can crowd out everything else on a shared partition.
`--allow-other` + `user_allow_other`	Lets a different user — the Jellyfin or Plex service account — read the mount. Requires the one-time `/etc/fuse.conf` edit above, or the mount fails with `only allowed if 'user_allow_other' is set`.
`--uid` / `--gid` / `--umask`	Makes files inside the mount appear owned by the media-server account so its own permission checks pass.

Put the cache directory on an ext4, xfs or btrfs filesystem. full mode relies on sparse files; exFAT and FAT do not support them, and rclone will log an error and crawl.

Mount to watch, rclone copy to move. A mount is tuned to play one file smoothly, not to move a library fast. Copying a whole folder through the mountpoint with cp or rsync serialises everything. To move data in bulk, run rclone against the remote directly so it parallelises across files:

rclone copy storagebox:/media/Movies ~/local/Movies -P --transfers 8

Over a single stream, an SFTP transfer settles around 33–40 MB/s no matter how fast the link is — that is the nature of one stream, not a server fault. --transfers 8 runs eight files at once and fills the pipe; 4 to 8 is the sweet spot, higher rarely helps. If a cold file buffers while streaming, raise the SFTP packet size: rclone defaults to 32 KiB, and --sftp-chunk-size 255k (the safe maximum) lifts throughput noticeably on a same-datacenter mount.

A tiered library

Caching is bounded and self-managing, but it does not decide what lives where. A tiered library goes further: the seedbox holds new and recently-watched content on fast disk, the storage box holds the deep archive, and a union filesystem presents both as one folder so the apps never see a file move between tiers.

Three rules make the design work:

The apps only ever use the union path (~/media/…). Set root folders and library paths once; the union reshuffles what is underneath.
New writes always land on the hot tier. Both union tools below can guarantee this.
The torrent directory lives inside the union too, so Sonarr and Radarr imports are hardlinks (no extra disk), and it is excluded from the mover so active seeds never migrate.

First mount the storage box (the rclone mount from the section above works as the cold-tier branch). Then pick one union tool.

Option A: rclone union (simplest, no extra software)

rclone is already installed on your seedbox, and its built-in union backend does exactly this job — no second program to install. Add a union remote whose upstreams are your local hot directory and the storage box, with the local branch marked writable so new files land there:

# ~/.config/rclone/rclone.conf
[storagebox]
type = sftp
host = STORAGE.pulsedmedia.com
port = PORT
user = USERNAME
key_file = ~/.ssh/id_ed25519_storagebox

[media]
type = union
upstreams = /home/USERNAME/media-local storagebox:media
action_policy = epall
create_policy = ff

rclone mount media: ~/media --vfs-cache-mode full --dir-cache-time 1h --allow-other --daemon

The create_policy = ff ("first found") sends every new file to the first upstream to respond — your local disk, which always answers before the storage box over SFTP — so new downloads land on the fast hot tier, not the cold one. (Do not use the epmfs default here: it picks the branch with the most free space, which is your big storage box, and would send new writes straight to the slow tier.) Reads fall through to the storage box. This is the lowest-dependency path and the one to start with.

Option B: mergerfs (the Sonarr/Radarr power-user setup)

mergerfs is the union filesystem most of the self-hosted media community uses, because its hardlink behaviour and per-branch policies are battle-tested with the *arr apps. It is not part of the seedbox image — download the static linux-amd64 binary from the mergerfs releases page into ~/bin, then union your local directory with the rclone mount:

~/bin/mergerfs \
  -o category.create=ff,cache.files=partial,dropcacheonclose=true,minfreespace=50G \
  /home/USERNAME/media-local:/home/USERNAME/mnt/storage /home/USERNAME/media

category.create=ff ("first found", local branch listed first) sends every new file to the hot tier; the storage box only ever receives files from the mover. Verify both mounts came up:

mountpoint ~/mnt/storage && mountpoint ~/media && ls ~/media

To unmount either FUSE mount, always use fusermount -u <path>. Never Ctrl+C or kill a FUSE process — that leaves a kernel mount registration behind that ps cannot see but still occupies the mountpoint, producing confusing "it says mounted but shows nothing" states.

Route your torrent client through the union

Hardlinks cannot cross a mount boundary. If your torrent client saves to a plain directory while the library lives on the union, Sonarr and Radarr silently copy every import instead of hardlinking — doubling disk use per item. Put both ends inside the union and the hardlink passes straight through to the local branch. In qBittorrent (Options → Downloads):

Default Save Path: ~/media/torrents
Keep incomplete torrents in: ~/media/torrents/incomplete

Verify after your first import that the seed file and the library file share an inode (link count 2):

stat -c '%h %n' ~/media/torrents/SomeFile.mkv ~/media/Movies/SomeFile.mkv

If the library copy shows link count 1, imports are copying — recheck the paths. Set seed goals in the *arr apps, set qBittorrent's seeding-limit action to pause (never "remove and delete"), and let the *arr apps be the single authority for removing finished torrents.

The quota-aware mover

A scheduled script ships aged files from the hot tier to the storage box when your disk fills. Two safety habits are baked in: it defaults to a dry run unless you explicitly arm it, and it moves only files that are not still hardlinked to an active seed (moving a still-seeding file frees no space and wastes upload bandwidth).

#!/bin/bash
# ~/scripts/media-mover.sh
THRESHOLD_G=4500          # ~85-90% of YOUR quota — check with: quota -s

# Fail-safe: real transfers require an explicit arming file, else dry-run.
EXTRA=(--dry-run)
[ -f ~/scripts/media-mover.ARMED ] && EXTRA=()

# Parse quota in GiB; bail out safely if the number is not clean.
USED_G=$(quota -s 2>/dev/null | awk '/\/dev\// {u=$2; sub(/G$/,"",u); if (u ~ /[MT]$/) exit 1; print u; exit}')
[[ "$USED_G" =~ ^[0-9]+$ ]] || { echo "$(date): quota unparseable, skipping" >> ~/logs/mover.log; exit 1; }
[ "$USED_G" -ge "$THRESHOLD_G" ] || { echo "$(date): ${USED_G}G below threshold, skipping" >> ~/logs/mover.log; exit 0; }

# Move only single-link files (not still hardlinked to a seed), oldest first, torrents excluded.
cd ~/media-local || exit 1
find . -type f -links 1 -mtime +14 -not -path './torrents/*' -printf '%P\n' > /tmp/movable.txt
rclone move ~/media-local storagebox:media --files-from /tmp/movable.txt \
  "${EXTRA[@]}" --delete-empty-src-dirs --transfers 4 \
  --sftp-chunk-size 255k --log-file ~/logs/mover.log --log-level INFO

Test it before trusting it: plant a throwaway file with a backdated timestamp, lower the threshold, run once (dry run), read the log, then touch ~/scripts/media-mover.ARMED, run once for real, and confirm the file is gone from the local branch, present on the storage box (rclone lsf storagebox:media), still visible at the same union path, and reads back with a matching checksum. rclone move is interruption-safe — it deletes each source file only after that file's transfer completes, so an interrupted run never leaves a half-file.

Schedule it from your own crontab (crontab -e):

30 4 * * * /home/USERNAME/scripts/media-mover.sh

Keeping the mounts alive

Mounts must survive a crash. A small watchdog on your crontab remounts anything that has dropped:

# ~/scripts/mount-watchdog.sh — re-run the rclone mount / mergerfs commands if their mountpoints are gone
*/5 * * * * /home/USERNAME/scripts/mount-watchdog.sh

Start the union only after the rclone mount is confirmed live (check with mountpoint -q) — a union built against a not-yet-ready mount comes up half-empty. A cron watchdog is the reliable choice here: it runs in the global mount namespace, so the mounts it creates are visible to your shell and your apps. Avoid mounting from systemd --user units without testing first — on a multi-tenant host the user manager can run in a private mount namespace, where the units report active and the mounts genuinely exist but nothing else can see them. Test with readlink /proc/self/ns/mnt versus systemd-run --user --pipe --wait readlink /proc/self/ns/mnt; if the two differ, do not mount from systemd user units.

Gotchas

fusermount3 missing. Pulsed Media seedboxes already ship and hold fuse3, so rclone mount and mergerfs work out of the box. If a mount fails with a missing-fusermount3 error, that is a host-side package problem — open a support ticket rather than working around it, and support can restore it.
New files do not appear. Plain SFTP has no change notification, so a file added on the storage box shows up in the mount only after --dir-cache-time expires. Raise it for a mostly-static library; send the mount process SIGHUP to refresh immediately.
Cache filling the disk. Almost always a missing --vfs-cache-min-free-space, or a --vfs-cache-max-size smaller than the largest single file you stream — an open file cannot be evicted mid-playback, so size the cache to at least twice your biggest title.
Login loops on the rclone web GUI. It uses the same password as your seedbox web panel — there is no separate rclone login. A repeating prompt is usually a browser autofilling your billing email as the username.
Do not rm -rf the cache to "fix" a stuck mount. In full mode, writes land on the cache first and flush to the storage box a few seconds later. Files that have not flushed live only in the cache directory and are recovered on the next mount with the same --cache-dir. Delete the cache and you delete unflushed writes.
A mv through the union is not always a local operation. It renames the file on whichever branch it currently lives on. Reorganising a cold file can quietly create directories on the storage box. To pull something hot on purpose, use rclone move storagebox:path ~/media-local/path explicitly.