So if I remember correctly, what happens with auto mode is they run a second smaller LLM called the “classifier” to evaluate the tool uses of the main one.

I’ve used auto mode at work now for many months, and it approves most things because most things Claude does are reasonable.

Once or twice I have seen it reject Claude. I can’t remember the exact scenario, but I had asked Claude to diagnose an issue but not fix it yet, and when later on it tried to make the change the classifier rejected it, giving the reason that what it was trying to do did not match my request.

In my opinion, the trick to using auto mode safely here is to:

• Commit and push your changes before handing over the reins (it rarely commits or pushes without you telling it to, especially if you have never asked in that session).
• Don’t give it access to things it shouldn’t have access to or that can do significant damage
• Don’t give ambiguous prompts.
• Never use LLMs against untrusted code or files, as it may contain prompt injections.

Having run models locally, RAM use seems to be almost directly proportional to number of parameters. 8 Billion parameters requires approx 8GB of VRAM at 1/4 precision.

Therefore, if this pattern holds you somehow need 10 Terabytes of VRAM at 4K and 40 Terabytes at full precision.

I think I saw some estimates that Claude’s Opus models may be and Opus model equivalents may be at around 100B parameters (100-400GB VRAM).

TLDR its clear why RAM is so expensive.

Thanks. That’s very helpful.

Why is it supposedly illegal to do this? Being American is not a protected characteristic, no?