I personally would go the other way, use your nvme for storage and have a second small drive for proxmox since that part doesn’t really need speed. That said, if you go with zfs replication it doesn’t matter; just have the one nvme drive that holds the proxmox install and the storage pool. Separate drive only matters for Ceph.

Agreed, it helps that with proxmox the cluster is a first-class feature and all installs are a cluster even if only a single node. Really removes a lot of potential pain points from operations like this.

That depends on what level of HA you want to end up with.

If you want proper HA, you’ll want to plan on adding a (small, like a Raspberry Pi) third node for quorum. If you are already taking backups and you just want “I can restore on the second system” then it’s slightly simpler, but mostly the same process:

• Setup new node, add to cluster
• Migrate all VMs and LXCs to new node
• Remove and upgrade other node
• Add rebuilt node to cluster

If you’re planning on proper HA, I’d strongly advise having the proxmox installation on a second small drive on each node and leaving your 1tb drives as data only.

This article half-explains one option for a two node setup (zfs replication), which is functional but not ideal. If you want to get your feet wet with Ceph then I can give you some pointers.

deleted by creator

My ISP eventually started supporting IPV6, but only assigned /128

This is hilarious to me.

“We’ve got 7.9 septillion addresses to play with in each of our v6/32 LIR allocations… if we follow the standard and give each customer a whole network prefix, that caps us at 4 billion customers per LIR! Nonsense, let’s just give every household a single v6 address.”

It’s like these people don’t understand what IPv6 is for.

There are a few ISPs in North America that support ipv6, but many many don’t. As much as I detest the recent push toward “5G Internet to the Home”, it at least does increase adoption of IPv6 since (from what I understand) basically all mobile carriers are v6-only and do NAT64 for v4 support.

I don’t know if that translates to the 5G-at-home offering but it wouldn’t surprise me since most customers don’t care what address scheme is being used as long as Netflix works.

yeah but then you’re interacting with oracle

hard pass

April fools joke RFCs are getting out of hand.

Why use netcat when you can just tap the bits into an ethernet cable with a bench power supply like a telegram operator?

under a pile of pillows

maybe not literally though, hard drives do need some cooling…

Thanks for the write-up! I’ve been trying and failing to do DOOD and POOP runners via forgejo, but I haven’t had the time or energy to really dig in and figure out the issue. At this point I just want something to work so I’ll give your setup a try 😎

please share, I’m interested in doing the same

Yes, if you’re going to run TrueNAS (or another solution based on ZFS) you should really get rid of the PERC and get an LSI SAS card in IT mode so that the system can see the raw disks.

When you start your SATA swap, either use the onboard SATA ports (if there are enough) or get a SATA card (more ports, probably slightly better performance than sharing the onboard controller) and start the process I described before.

Yes, you’re going to want to get SATA drives that are the same size or bigger than your SAS drives. The mini-sas will break out into 4x sas connectors but you don’t have to swap 4 at a time; disconnect one SAS drive from the SAS breakout cable and then connect one replacement SATA drive to the SATA backplane (either the one on your motherboard or to a SATA card if you don’t have enough mobo ports). Do a zfs scrub. Once it’s finished with no errors, repeat all three steps. Once all drives are off the SAS card and your final scrub is done you can remove the SAS card entirely.

For this use case there’s not really an advantage using SAS over SATA. I’d suggest buying SATA drives in the future just because you don’t need a SAS card for them, and SATA drives are usually cheaper.

If you use the H700 for hardware RAID and switch to SATA later, your best bet is probably to copy the data over (or better, use the opportunity to test your backup/restore process).

If you could run the SAS disks in JBOD mode (which is possible if you sell the H700 and use another SAS card), you could set up your drives in a RAIDZ1 mode and later switch to SATA drives by replacing one drive at a time and doing a scrub between each swap.

This is a PERC H700 which does not support IT mode (even if you cross-flash to an LSI firmware).

You could use that card as-is but for truenas I’d suggest grabbing a proper SAS card. I got one off ebay (LSI 9207) for about USD$35 already flashed and ready to go.

Can confirm, this is my setup and it works great.

I am a sysadmin with over 30 years of experience managing servers and networks for businesses of all sizes as well as for myself, friends, and family.

The FUTO guide is extremely detailed, accurate, and accessible. It does not always follow best practices, and it’s not a comprehensive guide to all of the possibilities for self-hosting. It’s not trying to be. It is a guide for someone with no technical expertise (but with basic technical ability) to degoogle/deapple themselves at a reasonable level of cost and effort.

You do not have to do everything in the list, you can pick and choose the parts you’re interested in. That said, I would recommend reading through the whole article as you have time, because it does a very good job of explaining the concepts involved in building a self-hosted setup, and understanding how everything works is the biggest step toward being able to effectively troubleshoot problems when they inevitably crop up.

If you have specific questions about things that aren’t answered in the guide or via a quick web search, post them here.

deleted by creator

Thank you for the detailed write up. I’m going to give this a shot and see if I can save myself some space.