Hacker Newsnew | past | comments | ask | show | jobs | submit | dotinvictim's commentslogin

local llm don't make sense currently consumer compute is not upto mark it may take atleast 7 more years to be usable

It literally is usable now. A 5060 for $800 can run qwen3.8-27b 4bit at >40t/s, and the model beats opus 4.6 (max).

Beats Opus 4.6 at what exactly? It certainly isn't code.

I use a combination of a Claude Max subscription and local inference, including qwen3.8-27b, 4bit. I have found qwen to be absolutely useless at anything but very specific, surgical code changes. In my experience, for anything even remotely nuanced, a frontier model is required.


https://artificialanalysis.ai/?models=qwen3-8-27b%2Cclaude-o...

Index methodologies here: https://artificialanalysis.ai/evaluations/artificial-analysi...

Also see some specific benchmarks here: https://huggingface.co/Qwen/Qwen3.8-27B e.g. qwen scores 61.7 on swe bench pro, while opus 4.6 scores 53.4.

If you want to argue with the benchmarks, go for it. Fwiw i am not saying qwen3.8-27b is better or as good as the frontier. But i am saying it has crossed the threshold and is now a useful tool for coding and debugging. From my experience, Qwen3.6-35b-a3b was what you describe - it could do surgical edits only.


What configuration? What harness? These matter greatly to how a local model performs, in my experience.

> 5060

I think you're referring to a 5060Ti 16GB, yes?

32k context is easily done there. 64k can work with a more aggressive quant, but you lose a bit of speed.


Yeah 16gb. For longer context, a 3bit quant is needed. Yes it’s tight on a 16gb card - can’t wait for the bubble to pop so hardware prices fall.

But I don’t quite follow you - how does a more aggressive quant slow it down? Less bits per token means faster inference not slower.


In my own experience, qwen3.8-27b 4bit can consistently find bugs in software written by sonnet 5 and opus 5. But it does do that at maybe 1/10th the speed. Still a pretty good deal if you're coding without wanting to spend big.

qwen3.8-27b 4bit has a following specifically for being exceptionally gifted for such a small model.


It’s genuinely the first local model that has actually made me keep it around, it’s pretty good at spotting things existing tooling can’t, like comment rot/drift/summarising accurate git changes, it’s also the first local model that pushes back when it thinks it is right (I don’t dislike that even when it is wrong, it still hallucinates things that don’t actually work the way it confidently states they do).

It does clear the useful enough to be worth it bar though.

Zero interest in remote models but local ones if they offer utility, sure.

Runs pretty well on a 7900XTX as well.


unleash gemini 4 stop saving dario

I am wasting money on gemini pro sub release gemini 4 already

enough with this anti ai noise resistance is futile it's getting popular due to improved ui

You’re wrong. Despite its usefulness in our profession, AI is deeply unpopular with the general public and products that don’t include it are rapidly gaining in popularity. People are tired of AI being shoved down their throats without consent.

Are you speaking from personal experience or do you have data to back that up? I honestly would love to see some study around this. I haven't done my research but --anecdotally-- everyone around me is ok with it or even enjoying AI to some degree. No strong stances either way tbh.

Here you go: https://archive.ph/9erfx bask in that 20% support of "more good than bad when it comes to news" and the 70% thinking it's destroying jobs.

There's also this, quite related one, showing that the BigAI CEOs are very disliked by below 34's https://futurism.com/artificial-intelligence/young-people-ai...


I'm not anti-AI (I have a ChatGPT subscription), I just don't like bloat and nagging.

Nonsense. I can (and do) resist a lot of things. Society in general may well be screwed, but I'll at least hold onto a bit of self-respect.

Enough with this pro-ai noise.

I have 2 thanks for tell worst exp ever avoid dot in

What happened with you?

register two domain after a week both went serverhold then you have to email nixi with aadhar, kyc , pan then update details according to gst details in domain address satisfy their agent can't use encrypeted mail like proton after that he will call point out few things missing after fixing them they might lift server hold status took like 1 month of constant email exchange to make it work

My god

man .in is horrible exprience you have to send aadhar card pan etc after 30 days of continous effort they lift the server hold status fing rediculsas

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: