South Korea says it's uncovered evidence that DeepSeek has secretly been sharing data with ByteDance, the parent company of popular social media app TikTok.
I use 32b and the 672b side by side. The performance hit is around 20% and I keep all my data local. I am not conflating the two however self hosting works for me just fine. Your usecase is your own certainly. However I’d rather take the performance hit for the added data privacy.
Also it’s nice to he able to set my own weights and further distil R1
I have a local python expert a local golang expert and both have my local gitlab repository and I’ve tied their respective Ollama keys to my VSCode IDE.
With the distilled models I have, I’ve been able to build and troubleshoot pretty complicated apps in Golang and Python. However, these distilled models are very specialized and will not do things like write me a story about a duck made out of duct tape or properly summarize articles. There are absolutely limits to my workflow and setup. But I’m pretty happy with it.
Have you had any luck importing even a medium-sized codebase and doing reasoning on it? All of my experiments start to show subtle errors past 2k tokens, and at 5k tokens the errors become significant. Any attempt to ingest and process a decent-sized project (say 20k SLOC plus tooling/overhead/config) has been useless, even on models that “should” have a good-enough sized context window.
My codebase is almost 1.2GB of raw python and go files no images. I think it’s somewhere near 15k tokens for the python codebase and 22k for golang due to all the .mod and .io connectors to python libraries… it was a much bigger mess before if you can believe it.
What size model are you using? I’m getting pretty good results with R1 32b but these have been distilled to be experts in the languages of the codebases. I’m not using any general models for this.
Also it depends on the language you’re targeting as well. Rust or Lisp have issues due to how much less they’ve been documented. I think golf type languages like brainfuck are impossible. It really comes down to how the language has been documented. Python gave me issues in the beginning until I specified 3.11 in my weights and distillation/training, and that definitely fixed a lot of the hallucinations I was getting from the model.
I think static typing languages that have consistent documentation would be the easiest for this. Now that I think of it, maybe getting a typescript expert would be something I could tool around with.
Edited for legibility and the fact that I just went and looked at my datasets again. Much bigger than I initially thought.
I use 32b and the 672b side by side. The performance hit is around 20% and I keep all my data local. I am not conflating the two however self hosting works for me just fine. Your usecase is your own certainly. However I’d rather take the performance hit for the added data privacy.
Also it’s nice to he able to set my own weights and further distil R1
I have a local python expert a local golang expert and both have my local gitlab repository and I’ve tied their respective Ollama keys to my VSCode IDE.
Depends for sure. I usually try the 32B first, but give really “hard” queries to some API model.
With the distilled models I have, I’ve been able to build and troubleshoot pretty complicated apps in Golang and Python. However, these distilled models are very specialized and will not do things like write me a story about a duck made out of duct tape or properly summarize articles. There are absolutely limits to my workflow and setup. But I’m pretty happy with it.
Have you had any luck importing even a medium-sized codebase and doing reasoning on it? All of my experiments start to show subtle errors past 2k tokens, and at 5k tokens the errors become significant. Any attempt to ingest and process a decent-sized project (say 20k SLOC plus tooling/overhead/config) has been useless, even on models that “should” have a good-enough sized context window.
My codebase is almost 1.2GB of raw python and go files no images. I think it’s somewhere near 15k tokens for the python codebase and 22k for golang due to all the .mod and .io connectors to python libraries… it was a much bigger mess before if you can believe it.
What size model are you using? I’m getting pretty good results with R1 32b but these have been distilled to be experts in the languages of the codebases. I’m not using any general models for this.
Also it depends on the language you’re targeting as well. Rust or Lisp have issues due to how much less they’ve been documented. I think golf type languages like brainfuck are impossible. It really comes down to how the language has been documented. Python gave me issues in the beginning until I specified 3.11 in my weights and distillation/training, and that definitely fixed a lot of the hallucinations I was getting from the model.
I think static typing languages that have consistent documentation would be the easiest for this. Now that I think of it, maybe getting a typescript expert would be something I could tool around with.
Edited for legibility and the fact that I just went and looked at my datasets again. Much bigger than I initially thought.