I ended up just building a box after looking for the perfect NAS and finding it didn’t exist. The software is usually just crap or the hardware is underwhelming. Got a Node 804 case to slap in plenty of HDD space. Running NixOS so I’m in control of the software. In retrospect I wish I had gotten a rackmount type case. Tossed in an Arc GPU for better transcoding shortly after the initial setup.
I haven’t tested in Windows, but this is my setup Linux to Linux using rclone which the docs say works with Windows.
Server
Client
I use this setup for my local files and a similar setup to my Backblaze B2 off site backups.
The VFS implementation has been pretty good. You can also manually sync. Their bisync I don’t fully trust though.
I can access everything through android using https://github.com/newhinton/Round-Sync. Not great for photos though as thumbnails weren’t loading without pulling the whole file last I tested a year ago.
One method depends on your storage provider. Rsync may have incremental snapshots, but I haven’t looked because my storage provider has it.
Sometimes a separate tool like rsnapshot (but probably not rsnapshot itself as I dont think its hard links interact well with rsync) might be used to manage snapshots locally that are then rsynced.
On to storage providers or back ends. I use B2 Backblaze configured to never delete. When a file changes it uploads the new version and renames the old version with a timestamp and hides it. Rsync has tools to recover the old file versions or delete any history. Again, it only uploads the changed files so its not full snapshots.
Yes. You compose a crypted vault over your storage vault. I pay about $1/mo for B2 Backblaze. Around 150G last I checked.
Important stuff (about 150G) is synced to all my machines and a b2 Backblaze bucket.
I have a rented seed box for those low seeder torrents.
The stuff I can download again is only on a mirrored lvm pool with an lvmcache. I don’t have any redundancy for my monerod data which is on an nvme.
I’m moving towards an immutable OS with 30 days of snapshots. While not the main reason, it does push one to practicing better sync habits.
I believe you will see OOM errors in journalctl. Also considered a systemd service or something to restart the process?
Glad you figured it out though!
I use EteSync to sync my contacts. It can do calendars also and has a self-hosted option. Personally I just use Proton for my calendar at the moment.
I haven’t tested any desktop syncing with EteSync, but maybe it will work for you.
I use DavMail to proxy an Exchange account so I don’t have to install Outlook on my phone.
Can try installing Avahi on the RPi (may come on the default image). It will advertise .local over mDNS / DNS-SD. I believe Avahi will advertise on link local if there is no default route to the internet.
Your system may automatically resolve the domain if its able to pickup the mDNS records to SSH in. Been a couple years since I’ve done it, so I could be forgetting a nuanced detail, but I vaguely remember just ‘plug and play’ if internet for the RPi wasn’t required.
My NAS is an mATX mobo with an i5, 64G RAM, 8 disk drives, 3 nvme drives, and an ARC GPU for video transcoding.
Disk drives are all mirrored. One nvme runs NixOS which is easy enough to redeploy if the drive dies. One nvme is cache on top of the disk drives. Last nvme I use for temp fast storage like Jellyfin transcoding.
Its more of a combo NAS/server as I run most self hosted apps on it (tor node, monero node, jellyfin, *arr stack, etc).
Not sure what your environment is. I can tell you what I do in linux/android.
I use backblaze b2 for my cloud storage.
I use rclone to create two encrypted “remotes”: one on my local file system and one for b2. Rclone supports a bunch of cloud providers, so you don’t have to use b2.
I mount the encrypted local file system and use whatever app (e.g., paperless) to access the files like it was any other directory.
When I’m done I unmount it and sync it with the b2 encrypted remote.
I use Round Sync on android which is rclone with a mobile GUI to access the same files. Also works great for backing up my phone.
For docker access to the mount point, either run the docker daemon as your current user, enable root access to rclone’s fuse mounts, or my preferred is to remount (with root access) a scoped directory for that docker container using something like bindfs.
Just be aware if using the vfs-cache (needed for seek or append), that cache is stored decrypted in your home folder. I’ve been meaning to look into locking it down with apparmor or something.
Round Sync with whatever remote (backend provider) you want supported by the underlying library rclone. They have self hosted remote options like FTP or something. If you want off site and privacy, create a crypted remote pointed to your off site backend remote. It acts as a wrapper to do end-to-end encryption.
Round Sync isn’t in any app store, so I’d set a watcher for releases on Github. You can setup scheduled backups or restores in Round Sync to keep things moving between your phone and backend remote as a relatively set it and forget it for one way syncing.
It has some nuances with bidirectional syncing that can result in data loss if not careful. With your workflow, I recommend something like a daily copy job from your phone to the server. This never deletes files. When you curate your photos via a tool on the backend remote (only do this when all photos on your phone are on the backup server), then you can do a manual sync from the backend remote to your phone to match the two exactly.
Copy just copies or updates to newer versions from target 1 to target 2.
Sync does the same but also deletes any files on target 2 that are not on target 1. Very easy to delete files if not careful in planning out your workflow. Test first.
I use Backblaze B2 for my off site remote which only hides and doesn’t delete files with the default settings. You can manage those with the rclone CLI application on a desktop to later cleanup hidden files or set them to delete after X days or something in the b2 life cycle settings on the Backblaze website.
Its no Google Photos or Dropbox, but it works well enough for me without giving up privacy. It also decouples the syncing from curating of photos giving some additional freedom for a custom workflow.
I personally just have a daily copy job on my phone from my phone to a crypted b2 remote and a cron on my self hosted server to copy from b2 to my self hosted server. Once a year I might clean things up from my server, do a manual sync to b2, then another to my phone. Sometime later I’ll go clean up the hidden (deleted) files in b2.
That said, I care more about backups than bidirectional syncing, so your milage may vary with this solution for your use case.
Depending on the country, those laws may be different. Here is a story of a guy who ran a TOR exit node in Australia who would have been protected as a company (law was later changed).
Set myservice.example.com to an internal address like 10.10.100.x. Those addresses are not routable on the public internet. They can only be reached from a local network which the VPN tunnels you into.
If your VPN also supplies DNS to clients, can also add a private zone if its supported so nobody can resolve that internal address except those clients.
Tried Agent DVR. It works, and I’ll probably borrow the libraries they suggest for alert monitoring, but the scrubbing performance wasn’t that great.
In my research modern fast scrubbing uses spritesheets. AgentDVR just felt like overkill for capturing raw video and rotating video files if I’m going yo craft my own viewer. I can do that with gstreamer easy enough.
Don’t have to use their proxy. My gateway router uses cloudflare to set the IP via the API and I just use self-signed certificates. A record resolves to my gateway, not some cloudflare server.
They also do a lot of work in the privacy space. Encrypted Client Hello to protect SNI came from them.
If you use any company for TLS termination they can MITM (e.g. AWS certificate manager).
NVR (network video recorder). Never found anything off the shelf in the same class as Nest which I really want to move away from (alerts, face recognition, fast scrubbing, etc).
Slowly building my own solution, but its low priority with some other projects. Mostly playing with gstreamer to do the capture over RTSP which is working. Working on file rotation next and then fast scrubbing (encoder + browser widget). After that either Home Assistant integration or the alerts/detection)
Something like https://graphite.com/ to create stacked PRs that are reviewable probably would have helped. Can be replicated with local LLMs or remote AI providers with locally configured agentic workflows. Never used graphite personally, but I’ve seen some open source maintainers use it to split up large PRs.