Quote:
Originally Posted by Shub-Nigurrath
Why LMstudio when you can run Ollama and configure it in a couple of clicks to switch the engine under the Claude app with an Ollama model? Moreover LMStudio is less efficient. The last time I used it had a problem, it tries to load the model entirely in memory.
The way I proposed uses Claude Coworker for the active parts which has longer token windows as far as I remember. And, yes I used it with a 20$ subscription. If you use the real Claude for commercial targets there are several risks: being banned, blacklisted, identified etc. the only safe way is to run on a strong enough NVIDIA a local abliterated model.
By the way today OpenAI declared what’s below, about their GPT 6 Astra cybersecurity model on steroids. Its capabilities of doing automatic RCE are definitely super good.
"We also tested Astra on SRE-Bench a benchmark that measures whether models can reverse engineer software binaries to understand its core logic without access to raw source code. Astra solved 88.0% of tasks in a single attempt and 99.2% within four attempts, compared with 55.9% and 68.7% for GPT-5.6 Sol, respectively."
|
I agree that LM Studio is less efficient in terms of memory usage, but it does offer a very polished and user-friendly interface.
I was considering running llama.cpp in server mode with the models. How would that compare with Ollama?
I’m a bit confused: why wouldn’t we get banned if we used Claude Coworker? I had assumed it would still rely on a Claude subscription behind the scenes. Did you mean that we would be using it with our own models instead?
I’m not interested in Astra or other frontier models, because I expect they would blacklist or ban us as soon as they suspect we’re trying to reverse-engineer a commercial target.
What I’m trying to find is a way to run an abliterated Qwen 3.6 or 3.8 locally with this mcp rather than through a cloud-based service. Need to possibly use it on commercial targets, so I am avoiding Cloud models, at least the ones that ban or blacklist easily.
I suspect that squareD may have been asking the same thing yesterday:
Code:
https://forum.exetools.com/showpost.php?p=135961&postcount=13