I like to code, garden and tinker
When I say residential IP addresses, I mostly mean proxies using residential IPs, which allow scrappers to mask themselves as organic traffic.
Edit: Your point stands on there are a lot of services without these protections in place, but a lot of services are protective against scrapping.
The only way I can think of is require users to authenticate themselves, but this isn’t much of a hurdle.
To get into the details of it, what do you define as an AI bot? Are you worried about scrappers grabbing the contents of you website? What is the activities of an “AI Bot”. Are you worried about AI bots registering and using your platform?
The real answer is not even cloudflare will fully defend you from this. If anything cloudflare is just making sure they get paid for access to your website by AI scappers. As someone who has worked around bot protections (albeit in a different context than web scrapping), it’s a game of cat and mouse. If you or some company you hire are not actively working against automated access, you lose as the other side is active.
Just think of your point that they are using residential IP addresses. How do they get these addresses? They provide addons/extensions for browsers that offer some service (generally free VPNs) in exchange for access to your PC and therefore your internet in the contract you agree to. The same can be used by any addon, and if the addon has permissions to read any website they can scrape those websites using legit users for whatever purposes they want. The recent exposure of the Honey scam highlights this, as it’s very easy to get users to install addons by selling users they might save a small amount of money (or make money for other programs). There will be users who are compromised by addons/extensions or even just viruses that will be able to extract the data you are trying to protect.
I’d just skip OpenVPN altogether and get started with Wireguard or Headscale/Tailscale.
This one was huge for me. OpenVPN is pretty heavy with CPU overhead, where as wireguard is almost free. I was getting throttled due to the overhead of OpenVPN and roasting the CPU on my Netgear R6350 (it’s what I had lying around). With wireguard I get nearly the same speeds as without a VPN and my loads are very reasonable.
Also with weaker routers like mine, be wary of trying to use QoS, this will probably not help network congestion and instead become a bottleneck (like it did for me). This is where a beefy dedicated router really shines.
My question would be, why do you need a more powerful server? Are you monitoring your load and seeing it’s overloaded often? Are you just looking to be able to hook more drives to it? Do you need to re-encode video on the fly for other devices? Giving some more details would help someone to give a more insightful answer. I personally am using a Raspberry Pi 4, Chromebox w/ an i7, an old HP rack server, and an old desktop PC for my self hosting needs, as this is cheaper than buying all new hardware (though the electricity bill isn’t the greatest haha, but oh well). If you are just looking for more storage, using the USB 3.0 slots on the Raspberry Pi 4b you can add a couple extra SSDs using a NVMe to USB 3.0 enclosure. For most purposes the speeds will be fine for most applications.
As for SSD vs HDD, SSD hands down. The only reason you’d pick an HDD is if your trying to get more storage cheaper and don’t mind a higher rate of failure. If your data is at all valuable, and it almost always is, redundancy should be added as well.
And as for running Linux, if it can’t run Linux I wouldn’t want to own it.
Edit: Fixed typo
This might help, sorry if it doesn’t, but here is a link to CloudFlares 5xx error code page on error 521. If you’ve done everything in the resolution list your ISP might be actively blocking you from hosting websites, as it is generally against the ISPs ToS to do such on residential service lines. This is why I personally rent a VPS and have a wireguard VPN setup to host from the VPN, which is basically just a roll your own version of Tailscale using any VPS provider. This way you don’t need to expose anything via your ISPs router/WAN and they can’t see what you are sending or which ports you are sending on (other than the encrypted VPN traffic to your VPS of course).
I use my own router with DD-WRT in-between the ISPs router/modem and my LAN, and use a different subnet. I haven’t had any issues with this myself, and my router just sees the ISP router/modem as the WAN.
Sounds like some QoS software is also limiting LAN traffic, seeing as it still works if the internet is disconnected. I would look if your router has “Adaptive QoS” or something similar enabled.
In most setups I have seen, the nginx instance provided by Lemmy is used due to the routing needed between lemmy/lemmy-ui being handled in nginx. Your reverse proxy can then point to the nginx instance to expose lemmy.
Why not a Raspberry Pi? The supply chain issues are clearing up.
In the US it seems the supply chain issues are alive as ever. Most of the official resellers are sold out on anything but the Pico and Zero boards. Some do have 4B boards for sale if you buy their starter kit with them, increasing the price by $65 on canakit. The supply issues are definitely not resolved for home users no matter what the CEO wants to say.
Hard to believe you used to have to pay for a TLS certificate. I use Let’s Encrypt with cert-manager on my kubernetes cluster and it still amazes me how SSL just happens. Even just using certbot makes the job extremely simple.
I chose Vultr because a friend recommended it, and the generous egress bandwidth provided. With a single $5 VPS you get 2TB free egress (this is for the whole account) + 1TB earned over the month (it gives the 1TB evenly out as the month progresses). This is more bandwidth than I’ll probably ever use for my services so I don’t have to be too concerned about bandwidth usage. So far I’ve hosted a few game servers and some HTTP web services (including my lemmy instance) with no issue.
Is there a DHCP server at play? Is the static IP outside of the DHCP range? This does sound like a typical IP collision.
If you are seeing your routers config page, and you are sure you are connecting from outside your network, it sounds like the router’s 443 page is overriding the port forwarding. Otherwise, it’s like @fixmycode@feddit.cl said and you just need a local DNS that points to the right spot locally, and let your public DNS point for external connections.
As for the hosts file, you can see a guide here for windows/linux/mac. Basically this is a override of any DNS entries. Here you can point jellyfin.domain.com to your jellyfin servers LAN IP and test the connection works.
To me this seems like a routing issue. Some things to check:
jellyfin.mydomain.com to your jellyfin server’s LAN IP?jellyfin.mydomain.com on the LAN?My guess is the router is routing traffic to it’s external IP from the LAN back to itself, without following port forwarding rules. Good luck figuring it out though!
At the moment it’s a bit convoluted, but you’ll need to go to the search page (https://tortoisewrath.com/search) and then search for the name of the community, in this case !selfhosted@lemmy.world. This will at first say no results but should eventually show the community (5-10 seconds in my case).
I’m not sure if there really is issues, I think it’s just new ground since most lemmy instances have been able to run on a single node due to the low populations. It seems most large public instances are just adding bigger servers to deal with the problem short term.
From what I can tell (I am not an expert in this field), it seems most of the architecture would spread horizontally without much issue. I haven’t seen anywhere this is done yet, but I could be missing the obvious.
The lemmy backend api just takes HTTP requests (and at the present websockets, but this is changing in 0.18 to only HTTP requests), and it uses postgres as the backend storage. Using a kubernetes postgres operator to scale the database and then running multiple lemmy backend api instances (and frontend as needed) seems like it would work, or would require minimal work to get running.
I currently am running the instance I am responding from on kubernetes. I published a helm chart, and others are working on them too. I feel being able to quickly deploy a kubernetes instance will help a lot of smaller instances pop up, and eventually be a good method of handling larger instances once horizontal scaling is figured out.
I currently host a few services (including the lemmy instance I am replying from) behind a commodity $5 VPS, while the services are actually hosted locally. I setup WireGuard to have a simple hard-coded peer-to-peer VPN connection from my local client to the remote VPS, and then setup some iptable rules on the VPS to redirect traffic to the VPN network. This allows me to host behind a NAT (my biggest issue), but also handles IP changes and hides your home’s public IP. I am no networking engineer, so I am not sure how safe this is, manually routing packets can be tricky.
There are also a few services this does not work for. So far I’ve found CS:GO dedicated servers (if the public IP of the local machine differs from the VPS) and email servers cannot be behind a NAT to function properly. Other services likely exist, but you’ll be able to run most services. You do lose the originating IP addresses in this case, which can complicate things (the case for email servers).
This process is explained in detail on wickedyoda.com and with a video tutorial.
Centralization is a weakness. These services can be targeted by governments that want to limit communication. Free speech is a commodity, and servers host this free speech. If a hostile organization, such as a government, targets a channel of free speech such as those hosted on a platform that makes it easy to setup a mastodon instance, this become an easy target that will affect a large portion of users. If you are serious about freedom, you have the freedom to self-host your own platforms.
Edit: I realize my post doesn’t answer the question proposed, but it’s more of an argument against such services. I would argue self-hosting doesn’t rely on paying third-parties to host your software, but I guess this is in the eye of the hoster.