Text Generation
Transformers
Safetensors
English
qwen2
prisma
orm
text-to-code
code-generation
qlora
dpo
grpo
reinforcement-learning
conversational
text-generation-inference
Instructions to use sandeeprdy1729/PrismaCoder-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use sandeeprdy1729/PrismaCoder-3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="sandeeprdy1729/PrismaCoder-3B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("sandeeprdy1729/PrismaCoder-3B") model = AutoModelForCausalLM.from_pretrained("sandeeprdy1729/PrismaCoder-3B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use sandeeprdy1729/PrismaCoder-3B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "sandeeprdy1729/PrismaCoder-3B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "sandeeprdy1729/PrismaCoder-3B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/sandeeprdy1729/PrismaCoder-3B
- SGLang
How to use sandeeprdy1729/PrismaCoder-3B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "sandeeprdy1729/PrismaCoder-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "sandeeprdy1729/PrismaCoder-3B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "sandeeprdy1729/PrismaCoder-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "sandeeprdy1729/PrismaCoder-3B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use sandeeprdy1729/PrismaCoder-3B with Docker Model Runner:
docker model run hf.co/sandeeprdy1729/PrismaCoder-3B
updated readme with the benchmark comparision
Browse files
README.md
CHANGED
|
@@ -24,14 +24,23 @@ schemas and client code from natural-language instructions. It was trained
|
|
| 24 |
with a full SFT β DPO β GRPO pipeline, using a verifier (gate) as both the
|
| 25 |
data-quality filter and the reinforcement-learning reward signal.
|
| 26 |
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
## Model Details
|
| 37 |
|
|
@@ -81,6 +90,14 @@ environment. See Limitations.
|
|
| 81 |
matches a PrismaBench benchmark task are excluded from the training
|
| 82 |
splits before SFT.
|
| 83 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
## Evaluation
|
| 85 |
|
| 86 |
**Benchmark:** PrismaBench, a 1,200-task benchmark (categories T1βT12,
|
|
@@ -160,28 +177,6 @@ card would be a number this project didn't actually measure.
|
|
| 160 |
- Assistance with schema migrations, CRUD operations, and query
|
| 161 |
composition against an existing Prisma schema.
|
| 162 |
|
| 163 |
-
## Out-of-Scope / Limitations
|
| 164 |
-
|
| 165 |
-
- **Static gates only, by default.** Unless the model card above states
|
| 166 |
-
full V1βV5 gates were used, evaluation numbers reflect syntactic/grounding
|
| 167 |
-
correctness, not verified compile-and-run correctness.
|
| 168 |
-
- **RL round 2 completed but is not the published checkpoint.** In the
|
| 169 |
-
training run that produced these results, RL round 2 (self-play
|
| 170 |
-
distillation on top of GRPO) completed successfully and matched GRPO's
|
| 171 |
-
proxy score, but has not yet been scored on the full 1,200-task benchmark.
|
| 172 |
-
The checkpoint published here is **GRPO**, not RL round 2. If RL round 2
|
| 173 |
-
is later found to score higher on the full benchmark, this card and the
|
| 174 |
-
published weights will be updated to reflect that β check the repo's
|
| 175 |
-
commit history for the latest evaluated checkpoint.
|
| 176 |
-
- **DPO's curated (non-mined) pair supply was very small** in this training
|
| 177 |
-
run (2 static gate-differential pairs); most of DPO's signal came from
|
| 178 |
-
on-policy mining rather than the project's intended curated pair set.
|
| 179 |
-
- **Domain-specific.** This model is fine-tuned narrowly for Prisma ORM
|
| 180 |
-
code generation and is not evaluated on, or intended for, general-purpose
|
| 181 |
-
coding tasks outside that domain.
|
| 182 |
-
- Not evaluated for safety, security-sensitive code generation (e.g.
|
| 183 |
-
auth/permission logic beyond the benchmark's T12_security category), or
|
| 184 |
-
adversarial inputs.
|
| 185 |
|
| 186 |
## How to Use
|
| 187 |
|
|
@@ -198,10 +193,16 @@ messages = [
|
|
| 198 |
]
|
| 199 |
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 200 |
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
| 201 |
-
out = model.generate(**inputs, max_new_tokens=640)
|
| 202 |
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
|
| 203 |
```
|
| 204 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 205 |
A GGUF export (`PrismaCoder-3B-f16.gguf`) is also provided for
|
| 206 |
llama.cpp/Ollama-based inference.
|
| 207 |
|
|
@@ -224,4 +225,4 @@ If you use this model, please cite the base model and note this fine-tune:
|
|
| 224 |
}
|
| 225 |
```
|
| 226 |
|
| 227 |
-
Base model: [Qwen2.5-Coder-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct).
|
|
|
|
| 24 |
with a full SFT β DPO β GRPO pipeline, using a verifier (gate) as both the
|
| 25 |
data-quality filter and the reinforcement-learning reward signal.
|
| 26 |
|
| 27 |
+
**Baseline comparison β 200-task PrismaBench subset, static V0 gates, greedy decoding, identical prompts:**
|
| 28 |
+
|
| 29 |
+
| Model | pass@1 | Wilson 95% CI |
|
| 30 |
+
|---|---|---|
|
| 31 |
+
| **PrismaCoder-3B** | **1.0000** | [0.9812, 1.0000] |
|
| 32 |
+
| StarCoder2-3B | 0.8950 | [0.8448, 0.9303] |
|
| 33 |
+
| DeepSeek-Coder-1.3B-Instruct | 0.8550 | [0.7995, 0.8971] |
|
| 34 |
+
| CodeLlama-7B-Instruct | 0.7400 | [0.6751, 0.7959] |
|
| 35 |
+
| Qwen2.5-Coder-3B-Instruct (base model) | 0.6700 | [0.6022, 0.7314] |
|
| 36 |
+
|
| 37 |
+
Run with the same evaluator, same prompt construction, same decoding settings
|
| 38 |
+
(greedy, `max_new_tokens=768`) across every model β see Evaluation Methodology.
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
**This 200-task result is consistent with, not a replacement for, the
|
| 42 |
+
authoritative full-benchmark number: 0.9992 pass@1 (Wilson [0.9953, 0.9999])
|
| 43 |
+
on the complete 1,200-task PrismaBench**, reported below.
|
| 44 |
|
| 45 |
## Model Details
|
| 46 |
|
|
|
|
| 90 |
matches a PrismaBench benchmark task are excluded from the training
|
| 91 |
splits before SFT.
|
| 92 |
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
**Practical implication:** always run `prisma validate` (or the full
|
| 97 |
+
V1βV5 gate chain, not yet run against this model β see Limitations) on any
|
| 98 |
+
output before using it. Do not treat a high pass@1 score alone as a
|
| 99 |
+
guarantee of correct, usable Prisma code, particularly for relation fields.
|
| 100 |
+
|
| 101 |
## Evaluation
|
| 102 |
|
| 103 |
**Benchmark:** PrismaBench, a 1,200-task benchmark (categories T1βT12,
|
|
|
|
| 177 |
- Assistance with schema migrations, CRUD operations, and query
|
| 178 |
composition against an existing Prisma schema.
|
| 179 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 180 |
|
| 181 |
## How to Use
|
| 182 |
|
|
|
|
| 193 |
]
|
| 194 |
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 195 |
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
| 196 |
+
out = model.generate(**inputs, max_new_tokens=640, do_sample=False) # greedy - matches published eval
|
| 197 |
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
|
| 198 |
```
|
| 199 |
|
| 200 |
+
**Always validate generated schemas before use**, given the known
|
| 201 |
+
`@relation` issue above:
|
| 202 |
+
```bash
|
| 203 |
+
npx prisma validate --schema=./generated_schema.prisma
|
| 204 |
+
```
|
| 205 |
+
|
| 206 |
A GGUF export (`PrismaCoder-3B-f16.gguf`) is also provided for
|
| 207 |
llama.cpp/Ollama-based inference.
|
| 208 |
|
|
|
|
| 225 |
}
|
| 226 |
```
|
| 227 |
|
| 228 |
+
Base model: [Qwen2.5-Coder-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct).
|