You use podman-compose with the Podman backend?

And there are no volumes mounted, just to be sure?

No expert but don’t the logs look like the requests are coming through?

I guess some people consider it cool. I’m not one of them.

Yeah right? At least a layer of VPN on top.

You might want to check out heretic or similar tools. I did not try it but there are a lot of heretic finetunes available ond HF.

A bunch

Late to the party, but this was just released: LiquidAI/LFM2.5-8B-A1B-GGUF

I guess you could fully fit it at Q4 with a little context if you need all the speed you can get, or offload the experts to RAM if you prefer higher quality and/or more context.