Before you even start, consider adopting an ‘infrastructure as code’ approach. It will make your life a lot easier in the future.

Start with any actual code: If you have any existing source code, get it under git version control immediately, then prioritize getting it into a git hub like forgejo to make your life easier in the future. Make a git repository for your infrastructure documentation, and record (and comment/document too if you’re feeling ambitious) every command you run in a txt file or an md file or a script, and do that as religiously as you can while you’re setting up all this self-hosted stuff. You may want to dig it up later to try and remember exactly what you did or in case stuff goes wrong and you need to back off and try again. It might seem pointless now, but a year from now, you’ll thank me.

Especially prioritize getting your git stuff moved into a self-hosted forgejo if any of your stuff is hosted on the microsoft technoplague called github.

I ran Matrix for like a year, and pretty much hated every minute. It was fragile, complicated, and incredibly, bafflingly resource intensive. Matrix is an overengineered nightmare in my opinion, and it seems to be quickly distancing itself from self-hosters while pursuing enterprise usage. Neat technology, horrible implementation, misguided company.

XMPP is a breath of fresh air in comparison. Just like we still use email everywhere (even for authentication nowadays, fun!), XMPP is not obsolete simply because it’s older. It’s a solid foundation, plenty extensible, and does almost everything I can imagine needing to do without unnecessary complexity.

Matrix’s bridges are its killer feature, and it’s nice… when it works. But it’s simply not worth the headache of dealing with Matrix, in my opinion.

I don’t want the free petition websites online getting my personal network’s info and sharing or selling it, hence the interest in self hosting.

So either you’re creating a petition with a size of exactly “1” or you’re asking other people to trust YOU with their personal info instead, or you’re asking for a federated solution (extremely difficult to establish a verifiable web of trust framework, and STILL shares your “personal network’s info” whenever it federates or validates its data to dozens of other servers).

None of these scenarios are viable for creating a petition that anyone is going to take seriously (to the extent that anyone takes petitions seriously at all)

fail2ban mainly, but also things like scaling login delays (some sort of option often built into the software you’re running, but just as often not configured by default), or if you’re feeling particularly paranoid account locking after too many failures, and in general just not using default, predictable, common usernames or weak passwords, and honestly it’s even helped a bit by having slow hardware and throttled network bandwidth.

The goal is to make it so that someone can’t run a script that sends 100 million login attempts per second for common or stolen usernames and passwords and your server just helpfully tries them all and obediently tells them none of those worked… until one of them does.

Not only does this encourage them to TRY sending 100 million login attempts per second because your server isn’t refusing it, which is a huge waste of bandwidth and resources, it also makes it really likely that they’re eventually going to guess one right.

That’s the problem, When you’re running too many services as it is, you will be staring at a terminal at home sooner or later. Maybe you’ve gotten lucky and haven’t been ravaged by the cruel gods of fate yet, but it absolutely happens, and eventually it will happen to you. When you’re relying on family notifications and disaster response, you don’t get to choose when that happens, and sometimes you’ll have to spend a LONG time staring at a terminal at home. And when it happens often enough, or badly enough, you end up not just staring at the terminal at home, but also thinking about the terminal at home, and losing sleep over it, and that’s just not a great way to live your self-hosting life. I’ve been there.

Making the investment in repeatable, reproducible, maintainable infrastructure now means you get to decide WHEN you’re staring at a terminal, and for exactly how long. Even when you don’t make it through as much progress as you wanted to, you can just close it down without any stress, get back to your life and continue from where you left off next time. You can’t do that, at least not without some significant consequences when your server got hacked and is sending spam or your entire server is refusing to boot and you need the files on it.

You may still have to hit the terminal sometimes when you don’t choose to, but it’s going to be less often, and less complex when you do. That’s when the investment pays off, and your return on investment is the goal of having ultimately less time spent at the terminal at home, and that payoff is especially rewarding if you’re good at prioritizing the time you do choose to spend on the terminal at home, to find low-value moments to effectively repurpose for this hobby, and save the actually valuable times of your life from ever having to be used for emergency maintenance.

FWIW I don’t find Apache dated at all. It’s mature software, yes, but it’s also incredibly powerful and flexible, and regularly updated and improved. It’s probably not the fastest by any benchmark, but it was never intended to be (and for self-hosting, it doesn’t need to be). It’s an “everything and the kitchen sink” web server, and I don’t think that’s always the wrong choice. Personally, I find Apache’s litlte-known and perhaps misleadingly named Managed Domains (mod_md/MDomain) by far the easiest and clearest way to automatically manage and maintain SSL certificates, it’s really nice and worth looking into if you use Apache and are using any other solution for certificate renewal.

If it helps motivate you to give it a shot, I found gitea’s runner very confusing to set up, but I felt like forgejo was better designed, pretty easy and well documented.

100% completion is not required, but you’ll share whatever progress you have made.

Granted, that still underestimates my ability to make actual progress. “Hooray, it’s now twice as complicated and actually worse!”

This looks less intimidating than Authentik. Any guides on getting it set up with any common self-hosted stuff?

Looks really nice and seems like it should be a great foundation for future development. Personally I can’t lose Nextcloud until there are sufficiently featureful and reliable clients for Linux, Windows, Android that synchronize a local copy and help manage the inevitable file deconfliction (Nextcloud Desktop only barely qualifies at this, but it does technically qualify and that represents the minimum viable product for me). I’m not sure a WebDav client alone is enough to satisfy this criteria, but I am not going to pretend I am actually familiar with any WebDav clients so maybe they already exist.

You’re on the right track. Like everything else in self-hosting you will learn and develop new strategies and scale things up to an appropriate level as you go and as your homelab grows. I think the key is to start with something immediately achievable, and iterate fast, aiming for continuous improvement.

My first idea was much like yours, very traditional documentation, with words, in a document. I quickly found the same thing you did, it’s half-baked and insufficient. There’s simply no way to make make it match the actual state of the system perfectly and it is simply inadequate to use English alone to explain what I did because that ends up being too vague to be useful in a technical sense.

My next realization was that in most cases what I really wanted was to be able to know every single command I had ever run, basically without exception. So I started documenting that instead of focusing on the wording and the explanations. Then I started to feel like I wasn’t capturing every command reliably because I would get distracted trying to figure out a problem and forget to, and it was duplication of effort to copy and paste commands from the console to the document or vice versa. That turned into the idea of collecting bunches of commands together into a script, that I could potentially just run, which would at least reduce the risk of gaps and missing steps. Then I could put the commands I wanted to run right into the script, run the script, and then save it for posterity, knowing I’d accurately captured both the commands I ran and the changes I made to get it working by keeping it in version control.

But upon attempting to do so, I found that just a bunch of long lists of commands on their own isn’t terribly useful so I started to group all the lists up, attempting to find commonalities by things like server or service, and then starting organize them better into scripts for different roles and intents that I could apply to any server or service, and over time this started to develop into quite a library of scripts. As I was doing this organizing I realized that as long as I made sure the script was functionally idempotent (doesn’t change behaviors or duplicate work when run repeatedly, it’s an important concept) I can guarantee that all my commands are properly documented and also that they have all been run – and if they haven’t, or I’m not sure, I can just run the script again as it’s supposed to always be safe to re-run no matter what state the system is in. So I started moving more and more to this strategy, until I realized that if I just organized this well enough, and made the scripts run automatically when they are changed or updated, I could not only improve my guarantees of having all these commands reliably run, but also quickly run them on many different servers and services all at once without even having to think about it.

There are some downsides of course, this leaves the potential of bugs in the scripts that make it not idempotent or not safe to re-run, and the only thing I can do is try to make sure they don’t happen, and if they do, identify and fix these bugs when they happen. The next step is probably to have some kind of testing process and environment (preferably automated) but now I’m really getting into the weeds. But at least I don’t really have any concerns that my system is undocumented anymore. I can quickly reference almost anything it’s doing or how it’s set up. That said, one other risk is that the system of scripts and automation becomes so complex that they start being too complex to quickly untangle, and at that point I’ll need better documentation for them. And ultimately you get into a circle of how do you validate the things your scripts are doing are actually working and doing what you expect them to do and that nothing is being missed, and usually you run back into the same ideas that doomed your documentation from the start, consistency and accuracy.

It also opens an attack vector, where somebody gaining access to these scripts not only gains all the most detailed knowledge of how your system is configured but also the potential to inject commands into those scripts and run them anywhere, so you have to make sure to treat these scripts and systems like the crown jewels they are. If they are compromised, you are in serious trouble.

By now I have of course realized (and you all probably have too) that I have independently re-invented infrastructure-as-code. There are tools and systems (ansible and terraform come to mind) to help you do this, and at some point I may decide to take advantage of them but personally I’m not there yet. Maybe soon. If you want to skip the intermediate steps I did, you might even be able to skip directly to that approach. But personally I think there is value in the process, it helps defining your needs and building your understanding that there really isn’t anything magical going on behind the scenes and that may help prevent these tools from turning into a black box which isn’t actually going to help you understand your system.

Do I have a perfect system? Of course not. In a lot of ways it’s probably horrific and I’m sure there are more experienced professionals out there cringing or perhaps already furiously warming up their keyboards. But I learned a lot, understand a lot more than I did when I started, and you can too. Maybe you’ll follow the same path I did, maybe you won’t. But you’ll get there.

Nextcloud is just really slow. It is what it is, I don’t use it for any things that are huge, numerous, or need speed. For that I use SyncThing or something even more specialized depending on what exactly I’m trying to do.

Nextcloud is just my easy and convenient little dropbox, and I treat it like it’s an oldschool free dropbox with limited space that’s going to nag me to upgrade if I put too much stuff in it. It won’t nag me to upgrade, but it will get slow. So I just don’t stress it out. So I only use it to store little convenience things that I want easy access to on all my machines without any fuss. For documents and “home directory” and syncing my calendars and stuff like that it’s great and serves the purpose.

I haven’t used Seafile. Features sound good, minus the AI buzzword soup, but it looks a little too corporate-enterprisey for me, with minimal commitment to open source and no actual link to anything open source on their website, I don’t doubt that it exists, somewhere, but that raises red flags for potential future (if not in-progress) enshittification to me. After eventually finding their github repo (with no help from them) I finally found a link to build instructions and… it’s a broken link. They don’t seem to actually be looking for contributions or they’re just going through the motions. Open source “community” is clearly not the target audience for their “community edition”, not really.

I’ll stick to SyncThing.

Sounds like you’re doing fine to me. The stakes are indeed higher, but that is because what you’re doing is important.

As the Bene Gesserit teaches: I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear.

Make your best effort at security and backups, use your fears to inform a sober assessment of the risks and pitfalls, and ask for help when you need to, but don’t let it stop you from accomplishing what you want to. The self-hosting must flow.

It is a perfectly valid approach, and there are also many other perfectly valid approaches. “Better” requires a definition of what you want to be better. If there’s something that’s making you uncomfortable about the process, let us know what concern or issue you’re seeing with it and maybe we can guide you to a better way for you. But there’s nothing wrong with the way they’re doing it. Others may have different preferences (including you, YOU might have different preferences!) but they’re just preferences. It’s not right or wrong, even if some people argue that it is, they’re always going to have some preferences embedded in that judgement. There’s always more than one way to do it. That’s the joy of it, really, and sometimes you’ll have to experiment yourself to find out what ways YOU like the best, that make sense to you, that are comfortable for you, or that do things the way you want to do them.

It’s your own self-hosting setup, you get to make the choices. Sometimes the number of choices can be intimidating and lead to analysis paralysis but the only way out of that is to realize that there really is no way of finding the “best” until you’ve tried many different ways and figured out the “best” yourself. That’s why the only real advice I can give you is to just go through the tutorial you’ve found and do it the way they do it for now. You can change later, as you learn more, when not if you decide you want to do something differently. Because you will. We all do. It’s part of the process.

Forgejo also has significantly more active development. Feature-wise, I believe they’re currently working on adding federation via ActivityPub as one of the main goals. I also think they have a much better designed and documented system for workers and actions at least at the time I went to spin one up. I switched to Forgejo and haven’t really looked back.

Yes, the famous communist fascist. His name is Stalin. You’ve probably heard of him.

In a world gone mad, you have to be insane to stay sane.

Horrible idea. You’ll likely end up syncing a mess of unnecessary, incompatible and conflicting binary build files onto different platforms, you’ll end up with internal file conflicts that are impossible to properly resolve and will destroy your repo, especially if you’re still using git on top of it. Don’t do this. Git has its own synchronization mechanisms for a reason, they are extremely mature and specifically designed for maximum efficiency, safety and correctness for the task at hand, which is managing source code. Millions of people use git for source code every day. It is a solved problem.

Syncthing is literally the WRONG tool for this job. It is a great tool for many situations, but you are using it as a hammer when what you need is a saw.

You are correct, and I am still a bit sad about it, because gitea was a cuter name and logo TBH. But Forgejo is pursuing a technically superior design and socially better path at this point.

Redundancy. I have two independent firewalls, each separately routing traffic out through two totally independent multi-homed network connections (one cable, one DSL, please god somebody give me fiber someday) that both firewalls have access to. For awhile was thinking of replacing the DSL with starlink until Elon turned out to be such a pile of nazi garbage, so for now DSL remains the backup link.

To make things as transparent as possible, the firewalls manage their IPs with CARP. Obviously there’s no way to have a single public IP that ports itself magically from one ISP to another, but on the LAN side it works great and on the WAN side it at least smooths out a lot of possible failure scenarios. Some useful discussions of this setup are here.

You’re absolutely incorrect about IRC. Would you like to learn? Open IRC federation is basically never used anymore and the few networks that exist are very stable (if not completely calcified), but it is a core feature of the design, and in the old days, massive interconnected networks of IRC servers like EFnet and Undernet spanned the globe, there were even some servers that allowed open federation (EFnet is actually named for it – eris-free-net referring to the last server “eris” that supported free federation), and at some points Netsplits were a frustratingly daily occurrence. Like with any federation, abuse is the reason we can’t really have nice things anymore, but IRC absolutely supports federation. Not very well from a modern standpoint since it didn’t really keep up with the abuse arms race, but when it was first conceived it was way ahead of its time.