MudMan

A quick look at US Amazon spits out that the only 24Gb card in stock is a 3090 for 1500 USD. A look at the European storefront shows 2400EUR for a 4090. Looking at other assorted stores shows a bunch of out of stock notices.

It’s quite competitive, I’m afraid. Things are very stupid at this point and for obvious reasons seem poised to get even dumber.

Yeah, for sure. That I was aware of.

We were focusing on the Mini instead because… well, if the OP is fretting about going for a big GPU I’m assuming we’re talking user-level costs here. The Mini’s reputation comes from starting at 600 bucks for 16 gigs of fast shared RAM, which is competitive with consumer GPUs as a standalone system. I wanted to correct the record about the 24Gig starter speccing up to 64 because the 64 gig one is still in the 2K range, which is lower than the realistic market prices of 4090s and 5090s, so if my priority was running LLMs there would be some thinking to do about which option makes most sense in the 500-2K price range.

I am much less aware of larger options and their relative cost to performance because… well, I may not hate LLMs as much as is popular around the Internet, but I’m no roaming cryptobro, either, and I assume neither is anybody else in this conversation.

You didn’t, I did. The starting models cap at 24, but you can spec up the biggest one up to 64GB. I should have clicked through to the customization page before reporting what was available.

That is still cheaper than a 5090, so it’s not that clear cut. I think it depends on what you’re trying to set up and how much money you’re willing to burn. Sometimes literally, the Mac will also be more power efficient than a honker of an Nvidia 90 class card.

Honestly, all I have for recommendations is that I’d rather scale up than down. I mean, unless you also want to play kickass games at insane framerates with path tracing or something. Then go nuts with your big boy GPUs, who cares.

But for LLM stuff strictly I’d start by repurposing what I have around, hitting a speed limit and then scaling up to maybe something with a lot of shared RAM (including a Mac Mini if you’re into those) and keep rinsing and repeating. I don’t know that I personally am in the market for AI-specific muti-thousand APUs with a hundred plus gigs of RAM yet.

Thing is, you can trade off speed for quality. For coding support you can settle for Llama 3.2 or a smaller deepseek-r1 and still get most of what you need on a smaller GPU, then scale up to a bigger model that will run slower if you need something cleaner. I’ve had a small laptop with 16 GB of total memory and a 4060 mobile serving as a makeshift home server with a LLM and a few other things and… well, it’s not instant, but I can get the sort of thing you need out of it.

Sure, if I’m digging in and want something faster I can run something else in my bigger PC GPU, but a lot of the time I don’t have to.

Like I said below, though, I’m in the process of trying to move that to an Arc A770 with 16 GB of VRAM that I had just lying around because I saw it on sale for a couple hundred bucks and I needed a temporary GPU replacement for a smaller PC. I’ve tried running LLMs on it before and it’s not… super fast, but it’ll do what you want for 14B models just fine. That’s going to be your sweet spot on home GPUs anyway, anything larger than 16GB and you’re talking 3090, 4090 or 5090, pretty much exclusively.

This is… mostly right, but I have to say, macs with 16 gigs of shared memory aren’t all that, you can get many other alternatives with similar memory distributions, although not as fast.

A bunch of vendors are starting to lean on this by providing small, weaker PCs with a BIG cache of shared RAM. That new Framework desktop with an AMD APU specs up to 128 GB of shared memory, while the mac minis everybody is hyping up for this cap at 24 GB instead.

I’d strongly recommend starting with a mid-sized GPU on a desktop PC. Intel ships the A770 with 16GB of RAM and the B580 with 12 and they’re both dirt cheap. You can still get a 3060 with 12 GB for similar prices, too. I’m not sure how they benchmark relative to each other on LLM tasks, but I’m sure one can look it up. Cheap as the entry level mac mini is, all of those are cheaper if you already have a PC up and running, and the total amount of dedicated RAM you get is very comparable.

Man, that was a time, with all the weird pop up cameras. They all mostly sucked and kept phones from actually useful features like water resistance, but it sure was fun.

I’ll even take an under-display camera, although you can almost always see it and it’s still unnecessary, when the actual camera lens can be crammed into a couple of mms of space at the top and that also gets you better speakers and room for additional sensors. The insanity is that now all that nonsensical innovation is in foldables instead, where you have a second full screen you can use to take much better selfies with the main camera package… and they’re still putting punch holes in the large screens for some reason. Samsung went to all the trouble of going under-screen in one of those. It’s insane.

I’m holding on to my older Xperia 1 and will consider upgrading if they do release the 1 VII this year, if only because I’ll be shocked if it isn’t the very last one. I have major gripes with some parts of their software, but at least the phone itself makes sense and, as you say, maybe if I sit on it long enough someone will make a compatible replacement ROM after I’ve migrated all my Google hostage apps and see what happens.

You know, for all the complaints about phones all being the same, I don’t see anybody trying to get rid of the stupid punch hole anymore. I haven’t taken a selfie since 2014, Sony is certainly looking like it doesn’t have many more Xperias in its back pocket and I really, really would like a replacement that isn’t afraid of having a thin forehead where you can put sensors without defacing the display. I would take something with expandable storage and a headphone jack for the complete package, but let’s start with a usable screen without holes in it. It’s gotten to the point where I haven’t seen a single phone in years I didn’t look at and immediately go “nope, not for me”.

Goes to show the different standards for different device types. MS is getting a beating out of stopping Windows 10 updates for some users after a decade.

I want to say I get why and that my phones cycle faster than my PCs, but these days I’m not sure that’s the case.

I feel like this conversation does a very good job of explaining why FOSS alternatives so often have terrible usability. “Not how most people would do it in a selfhost environment” is effectively “not how a tiny, teensy, borderline irrelevant proportion of users would do it”.

Selfhosting is moving towards being accessible to the average user in some areas. Not coincidentally, I suspect, mostly in areas where someone is trying to make money on the side (see Home Assistant increasingly trying to upsell you into their cloud subscription and branded hardware, for instance). This idea that structuring the software for the average phone user as opposed to the average home server admin is “bad” or “complicated” is baffling to me.

Oh, and for the record, no, that’s not the line for legality when it comes to watching the media I own. I am perfectly within my rights to access the files in my hard drive in any way I want. At least where I live. I make no promises for whatever dystopian crap is legal in the US. If anything there is a gray area on my using a specific type of drive to be able to rip commercial optical media that is theoretically DRMd in ways that my drive just happens to ignore. But remotely accessing my legal backups in my local storage? Nah, even if I was more worried about piracy than I am I’d feel fine on those grounds.

But also, copyright as currently designed is broken and not fit for purpose, and I suspect you don’t disagree and your pearl clutching here may have more to do with disliking Plex and not wanting to acknowledge an actually useful feature they provide than anything else. Maybe I’m reading too much into that.

I am very confused here. You seem to have slipped from arguing that it was difficult and complicated to arguing that it’s bad to be able to share content remotely because it’s a felony, which seems like a pretty big leap.

For one thing, it’s not illegal and I do rip my own media. I will access it from my phone or my laptop remotely whenever I want, thank you very much.

For another, and this has been my question all along, how is it possibly more difficult and complicated to have remote access ready to go than being “a DNS record away”? Most end users don’t even know what a DNS is.

And yes, not having (obvious) server configurations up front is transparent. That’s what I’m saying. It does mix at least two sources (their unavoidable, rather intrusive free streaming TV stuff and your library), but it doesn’t demand that you set it up. The entire idea is to not have to worry about whether it’s local content. Like I said, there are edge cases where that can lead to a subpar experience (mainly when it’s downsampling your stuff to route it the long way around without telling you), but from a UX perspective I do get prioritizing serving you the content over warning you of networking issues.

I don’t know, man, I’m not saying you shouldn’t prefer Jellyfin. I wouldn’t know, I never used it long enough to have a particularly strong opinion. I just don’t get this approach where having the thing NOT surface a bunch of technical stuff up front reads as “complicated and difficult”. I just get hung up on that.

Okay, but… how is it confusing from the front end if what you’re doing is going through the same steps of creating an account? You punch in a login and password in both.

Sure, Plex is doing this extra thing where it’s also bringing in centralized content along with your library and it will default to its remote access system if you log in from outside your network. But again, from the front-end that is transparent. You log in and you have your library. If anything they’re being a bit too transparent, I’ve had times where networking stuff got in the way and it took me a minute to notice that Plex was routing my library through their remote access system instead.

I can see objections to it working that way, you trade a (frankly super convenient) way to share content remotely and access content from outside your network without too much hassle for… well, going through someone else’s server and having their content sitting alongside yours. But “confusing and difficult” isn’t how I’d describe it. It seems to work like any other service, self-hosted or not, as far as the user-facing portions are concerned. I guess I just don’t see the confusing part there.

Wait, isn’t Jellyfin the same way? Pretty much every self-hosted app I run uses some web interface you log into so you can use it anywhere on the network. Sure, Plex also has some pre-set remote connection thing, but from the end user perspective it’s the same set of steps. I also had to make a login for all the stuff I fully self-host.

Is there no account management on Jellyfin? I would probably want that as a feature.

I barely even remember what the specific dealbreaker was, honestly. I was just dabbling, considering expanding my NAS and maybe getting the gear to dump my 4K BluRays. I gave Jellyfin a try first, I went through the setup process and I remember it being a) confusing to set up directly on my NAS, and b) very ugly.

I gave Plex a try to cover my bases and that looked better and got me up and running faster, so I just stuck with it. Easier remote access was a feature for me there, too, but the choice was made purely on the onboarding process, there was nothing activist to it. It’s maybe the most user-level, unresearched decision I’ve taken on software in a while, honestly. I was already trying to figuring out the ripping and encoding at the same time, so I didn’t want to put any additional attention on library management.

If anything I gave Jellyfin a bit more of a chance than I otherwise would have because I had heard a lot of angry chatter from people about Plex. I guess I came in after they made the changes that pissed people off and didn’t mind the state of the current product without a frame of reference. I would have bailed if there was a subscription, but they do have a one-and-done purchase, so now I’m set up, it’s working and I’ve paid them as much as I’m going to, so I’m fine with it. I do appreciate a free alternative existing, though.

Not the UI, the UX. The UI may be editable, but if I have to make my own UI to be happy with what it looks like or works like, then that’s bad UX.

I get that sometimes those terms are used interchangeably, but they’re not the same.

Hm. I gave Jellyfin a try and the UX was a turnoff, so I ended up in Plex. The separate management of metadata does sound like a pain to me, too, but maybe there’s a bit of sunk cost fallacy to that.

Either way it seems people are mostly fine with their choices and there is a viable free alternative, so… all good there.

What do you mean? That thing ran for a decade and had literal thousands of games.

I mean, sure, if you just have different flavours of the same PC parts put together in slightly different configurations they are relatively redundant.

But the Vita is very much not that. It has cameras, microphones, not one but two touch surfaces, gyroscope inputs and a wildly different config of contemporaneous hardware that required adaptations in many ports.

“Identical” is a high bar. I don’t think it’s uninteresting to be able to check out what is different in, say, the Vita version of Wipeout or Metal Gear HD or LittleBigPlanet. Plus there are also many of exclusives or very different releases on Vita. Tearaway is very much not the same on PS4, Virtua Tennis 4 has unique features on Vita (and is otherwise stuck on PS4 anyway), Uncharted never even got ported up. There are unique entries of Dynasty Warriors, Killzone, Resistance and Silent Hill in there…

I get that it’s challenging hardware to emulate and a lot of people don’t give it enough credit, but it is certainly not a platform that is trivialized by identical hardware elsewhere.

I mean… all of them? I don’t recognize that as a valid question, honestly. That’s not how emulation or game preservation work.

Like I said…

Start working on Vita emulation, you cowards.