llamacpp
1 post tagged “llamacpp”.
-
The 42 KB context audit
The reader was serving a quarter of the context the model was trained for, and the plan said there was nothing above it worth reaching for. One unit error, one constant, and 12 GB of headroom.