But why doesn’t it ever empty the swap space? I’ve been using vm.swappiness=10 and I’ve tried vm.vfs_cache_pressure at 100 and 50. Checking ps I’m not seeing any services that would be idling in the background, so I’m not sure why the system thought it needed to put anything in swap. (And FWIW, I run two servers with identical services that I load balance to, but the other machine has barely used any swap space – which adds to my confusion about the differences).

Why would I want to reduce the amount of memory in the server? Isn’t all that cache memory being used to help things run smoother and reduce drive I/O?

And how does cache space figure in to this? I have a server with 64GB of RAM, of which 46GB is being used by system cache, but I only have 450MB of free memory and 140MB of free swap. The only ‘volatile’ service I have running is slapd which can run in bursts of activity, otherwise the only thing of consequence running is webmin and some VMs which collectively can use up to 24GB (though they actually use about half that) but there’s no reason those should hit swap space. I just don’t get why the swap space is being run dry here.

If your card has an x4 pinout, then it probably needs the additional bandwidth. Plugging it into an x1 slot (if it was possible) would slow down the network traffic. Get a better motherboard with an x4 slot on it so you can use the hardware you want. or find something else that will fit your computer.

Honestly even the 1Gb quad port card I have requires an x4 slot, although I saw some dual-port 2.5Gb x1 cards on ebay. Maybe you could just use two of those?

Yeah? OK well it’s certainly worth taking a closer look at, and I was also doing some reading on Yacy. I’ve run one in the past called mnogosearch, with a lot of customization, but it would be nice to get into a community project like this.

But is it decentralized? Do the results from multiple spiders get added to give everyone the same quality searches or do I need to scan the whole internet myself?

[edit] I was looking at this earlier and couldn’t find the info. Started searching again just now and found it immediately… of course… (The answer is YES)

Yep, that’s exactly what I was looking at (https://github.com/searx/searx). As I said, it was a QUICK dive but the wording was enough to make me shy away from it. For all the years I’ve been running servers, I won’t put up anything that requires the latest/greatest of any code because that’s where about 90% of the zero-days seem to come from. Almost all the big ones I’ve seen in the last few years where things that made me panic until I realized that oh, if your updates are more than a year old then none of this affects you. And the one that DID affect me had already been updated through a security release.

I just did a quick dive into this and have some concerns. SearX appears to no longer be maintained and was last updated three years ago. SearXNG was forked to use more recent libraries but there were concerns that those are not always stable or fully vetted. There were also concerns that SearXNG did not follow the same concerns for user privacy. It’s a shame that SearX shut down, that one actually sounds like a project I would have jumped on.

More drives also equals larger power consumption so you would need a larger battery backup.

It also means more components prone to failure which increases your chance of losing data. More drives means more moving parts and electrical connections including data and power cables, backplanes, and generated heat that you need to cool down.

I’d be more concerned over how many failures you’re seeing that makes you think smaller drives would be the better option? I have historically used old drives from ebay or manufacturer refurbs, and even the worst of those have been reliable enough to only have to replace drives once every year or two. With RAID6 or raidz2 you should be plenty secure during a rebuild to prevent data loss. I wouldn’t consider using a lot of little drives unless it’s the only option I had or if someone gave them away for free.

Yeah I figured there would be multiple answers for you. Just keep in mind that you DO want to get it fixed at some point to use the disk id instead of the local device name. That will allow you to change hardware or move the whole array to another computer.

Are you sure about that? Ever hear about this supposed predictable network names in recent linux versions? Yeah those can change too. I was trying to set up a new firewall with two internal NICs plus a 4-port card, and they kept moving around. I finally figured out that if I cold-booted the NICs would come up in one order, and if I warm-booted they would come up in a completely different order (like the ports on the card would reverse which order they were detected). This was completely the fault of systemd because when I installed an older linux and used udev to map the ports, it worked exactly as predicted. These days I trust nothing.

OP – if your array is in good condition (and it looks like it is) you have an option to replace drives one by one, but this will take some time (probably over a period of days). The idea is to remove a disk from the pool by its old name, then re-add the disk under the corrected name, wait for the pool to rebuild, then do the process again with the next drive. Double-check, but I think this is the proper procedure…

zpool offline poolname /dev/nvme1n1p1

zpool replace poolname /dev/nvme1n1p1 /dev/disk/by-id/drivename

Check zpool status to confirm when the drive is done rebuilding under the new name, then move on to the next drive. This is the process I use when replacing a failed drive in a pool, and since that one drive is technically in a failed state right now, this same process should work for you to transfer over to the safe names. Keep in mind that this will probably put a lot of strain on your drives since the contents have to be rebuilt (although there is a small possibility zfs may recognize the drive contents and just start working immediately?), so be prepared in case a drive does actually fail during the process.

That is definitely true of zfs as well. In fact I have never seen a guide which suggests anything other than using the names found under /dev/disk/by-id/ or /dev/disk/by-id/uuid and that is to prevent this very problem. If the proper convention is used then you can plug the drives in through any available interface, in any order, and zfs will easily re-assemble the pool at boot.

So now this begs the question… is proxmox using some insane configuration to create drive clusters using the name they happen to boot up with???

One promising item I found are some json files from Reuters…

This one provides info on the candidates and the key for state ID’s: https://graphics.thomsonreuters.com/data/2024/us-elections/production/events/20241105/metadata.json

This one seems like it will provide the ballot counts(0) and possibly any declared winners(1): https://graphics.thomsonreuters.com/data/2024/us-elections/production/events/20241105/summary-votes/president.json

Of course I won’t know anything for sure until tomorrow evening when states start releasing their counts, but I went ahead and wrote up some code to use the files. It’s something at least, and the Reuters data should be fairly timely. I hope to play around with the collected info in real time, then maybe next election I can re-use the same code.

Yeah I found the same on the AP data. I also found that reddit thread, but haven’t been able to find a valid URL for this year’s election. Maybe they don’t make it available until voting starts?

Hmm good idea, I’ll take a look into that!

Good point, POE might be the only choice then. Although these cameras do provide x264 and x265 compression so that would save quite a lot.

You got me thinking, so I did a search and ran across this page: https://www.hanssonit.se/nextcloud-vm/ I’m not sure how old these releases are, but at the very least it might provide some hints for building your own? I’m going to keep looking to see if I can find an image built on Debian, but at least now I know some options are out there.

[Edit] I also ran across across this page which builds a VM for you using an Ubuntu machine, so I’m guessing I could probably adjust it to a Debian setup fairly easily. https://github.com/nextcloud/vm

I was considering POE as an option, and this camera does have an ethernet port (although I can’t tell yet if that’s only for configuration or if the video will also stream over it directly). I don’t really need a constant stream and this camera also provides motion options so maybe it would only send video as needed (although during a heavy storm all of the cameras would probably fire at once).

I played with Zoneminder years ago but would like to get something set up for home security. I have a full internal network plus servers and about 60TB of free storage space so there’s really no limitations to what I could set up. Ideally I’d like to just hit a local IP from a cell phone to check the cameras (and remote access isn’t really needed), so that’s where I was trying to go with my previous questions.

The software side seems easy enough, but finding compatible IP cameras has been stumping me. I see the Reolink 4K TrackMix wifi cameras on Amazon for $130, and other than a few hiccups it looks likely that this piece of hardware would work, unless anyone knows of any “gotchas” that I’ve missed? Otherwise I’ll do a bit more research and then order one of the cameras to see how far I can get with it.

Ah gotcha, so you’re not trying to to live-view the feed on them. Thanks for the camera suggestion anyway, the brand still seems to be decent enough even if they don’t support Linux.