• Womble@piefed.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      22 hours ago

      You almost certainly weren’t using a Deepseek model offline (unless you have 100s of GB of RAM/VRAM). You might have been using a small model distilled on Deepseek responses, but the smallest model made by Deepseek is about a 300B param model.

      • SteveTech@aussie.zone
        link
        fedilink
        English
        arrow-up
        11
        ·
        21 hours ago

        the smallest model made by Deepseek is about a 300B param model.

        This is not correct. DeepSeek has plenty of smaller models. I just had a scroll through their huggingface, and picked out a few: DeepSeek-V2-Lite (16B), deepseek-moe-16b-chat, deepseek-llm-7b-chat, but there’s quite a few more too.

        Though DeepSeek R1 is their flagship model, which is the one people usually think of, and that’s a 684B param model.

        • Womble@piefed.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          1 hour ago

          huh fair enough, I hadn’t seen those before, they’re from before deepseek made a name for themselves (which as you say was with R1). They are very out of date though, as is R1 (from the end of 2024 and start of 2025 respectively). the current set of models from Deepseek come in 291B and 1.6T varieties and the previous version only was released in 685B

          You’d almost certainly be better off upgrading to qwen or Gemma4 for a local model of that kind of size.