I don't know.... With all those approaches to run AI in your own premise, people need their own hardware. It makes them independent. Of course hyperscalers and AI labs don't like that, but I argue that consumers prefer independence over some costs of operations. If this would be the case, people wouldn't own cars.
I am quite familiar with Resolve..I am not a pro, and it's probably totally overload for my purpose, but I think it's easy to learn and even quite intuitive
I tried many others, but mostly all others are not stable enough or have an even worse UI/UX... so I wonder... is this different?
I’m also not pro and just need an editor for Youtube or work. I think open source editors like Openshot and Kdenlive are generally underrated and easier to use than they used to be but even for just cutting together clips, adding text, and using keyframes it’s hard to beat how nice Resolve is.
Dont want to crash the party here, but I am still sceptic about all those on-premise-llm-approaches.
I think we strongly need something like that (shameless plug, I tried to build something around bitNet for the same reason: https://github.com/nickyreinert/bitNetRTR).
But at the end, all aproaches I saw, however genius they are: the actual results are always a mess. It's a better chat buddy, nothing else. It's e.g. far away from an decent coding assistants. I fine tuned Gemma with domain specific knowledge. Running it on a 16GB VRM GForce. Even then it's okai'sh but far way from a mind blowing experience. I ran some of the promised open source model on my 36GB MBPro M3, in Pi, Hermes, Continue. Can't compare the results to what Claude or Codex are offering.
You need at least something that's far away from consumer hardware, like those 7k'ish GForce machines with 96GB VRAM to get an idea of a good competitive model.
Gemini uses MoE and context caching, which is a similar approach.
You are not really accessing the biggest frontier model every time, and you're not really doing an end-to-end LLM request on each prompt.
I would go so far to say frontier models have peaked and improvements from here come from clever (or very elaborate) harnessing. "LLLMHs" - Large Large Language Model Harnessing !
This is an odd comment: the project is right there for you to use, so just try it and see if it holds up to the claims? Then you can comment about the fact that it either doesn't hold up, with numbers to back that up, or on how awesome it is because it works =)
Having an M3 with 36 GByte I was under the assumption, that I can utilize like Qwen and similar models. It's quite easy to set up, you can use pi or hermes for CLI access, or "Continue" to use it in VS Code. You can choose between omlx, Ollama and even more to run the model itself. It's no rocket science, but the results are also not satisfying.
I use it occassionally for very easy tasks, fix typos or update meta data in blog posts. So yeah, it improves productivity. But coding-wise it's far away from Codex, Claude et al.
reply