How to use from
vLLM
Install from pip and serve model
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "analyticspro/slm-125m-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "analyticspro/slm-125m-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'
Use Docker
docker model run hf.co/analyticspro/slm-125m-base
Quick Links

slm-125m-base

A 125.8M-parameter Llama-shaped language model pretrained from scratch on a legal-first corpus (US case law + SEC filings + a slice of FineWeb-Edu). Built with a custom 16,384-token BPE tokenizer.

This is a base / foundation model -- it does next-token continuation, not instruction following or chat. Expect rough, domain-flavored completions; it was trained on a small budget (step 19,334, ~10.14B tokens seen).

Architecture

Params ~125.8M (tied embeddings)
Layers 12
Hidden size 768
Heads / KV heads 12 / 12 (MHA)
Context length 1,024
Vocab 16,384 (custom BPE)
Activation SwiGLU (silu)
Position RoPE (theta 10000)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("analyticspro/slm-125m-base")
model = AutoModelForCausalLM.from_pretrained("analyticspro/slm-125m-base")
model.eval()

ids = tok("The court held that", return_tensors="pt")
out = model.generate(**ids, max_new_tokens=120, do_sample=True,
                     temperature=0.8, top_p=0.95,
                     pad_token_id=tok.eos_token_id)
print(tok.decode(out[0], skip_special_tokens=True))

Limitations

Small model, small pretraining budget, and a legal-heavy corpus: outputs can be factually wrong, repetitive, or biased toward legal/regulatory phrasing. Not suitable for production or any high-stakes use. For research and demos only.

Downloads last month
26
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for analyticspro/slm-125m-base

Finetunes
2 models