Instructions to use Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M # Run inference directly in the terminal: llama cli -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M # Run inference directly in the terminal: llama cli -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
Use Docker
docker model run hf.co/Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
- Ollama
How to use Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1 with Ollama:
ollama run hf.co/Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
- Unsloth Desktop
- Pi
How to use Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1 with Docker Model Runner:
docker model run hf.co/Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
- Lemonade
How to use Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
Run and chat with the model
lemonade run user.qwen3loop-0.6b-tool-calling-sft-v1-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen3Loop-0.6B-Tool-Calling-SFT (E013_pareto_bundle)
Qwen3Loop-0.6B-Tool-Calling-SFT has been updated from the earlier v3 SFT weights to E013_pareto_bundle, an unrolled 56-layer standard Qwen3ForCausalLM checkpoint combining:
- SFT + L2 RL (Step 61) trained on multi-step tool calling, abstention, and mid-trajectory recovery.
- Surgical Layer-49 zero-hole attention repair (
E001), removing the late-pass collapse present in the raw RL checkpoint. - Orthogonal Pareto-KKT
lm_headweight-space bundles (E002โE013), touching6,144/1,192,098,816(0.000515%) parameters across six output rows (19,1703,12509,17453,23657,46254).
Note on
llama.cpp -hftags: All GGUF filenames follow the GGUF naming convention with a hyphen before the quant tag (e.g.,-BF16.gguf,-Q8_0.gguf,-Q4_K_M.gguf). Becausellama.cpp -hfwithout an explicit:TAGoften defaults to:Q4_K_Mโand 4-bit quantization degrades safety and reliability at this parameter countโalways pass an explicit tag such as:BF16or:F32(or:Q8_0under tighter VRAM limits).
Architecture
- Hugging Face architecture: standard unrolled
Qwen3ForCausalLM(model_type: qwen3, natively supported by stocktransformers,llama.cpp,vLLM, Ollama, and LM Studio with notrust_remote_codeor runtime patches). - Recurrent origin: 28 physical blocks unrolled across the recurrent middle stack (
7prefix +14 ร 3loop passes +7suffix =56logical layers,hidden_size: 1024,intermediate_size: 3072,16attention heads,8KV heads,max_position_embeddings: 40960). - Untied output head:
tie_word_embeddings: falsebecauseE002โE013optimize43lm_head.weightrows independently ofmodel.embed_tokens.weight. The recurrent triple-loop middle stack (layers 7..20,21..34,35..48) remains 100% weight-identical (max_diff = 0.0), whileE001repairs suffix layer49(physical layer21, outside the loop). - Earlier v3 release: The older 28-block folded SFT checkpoint (
Qwen3LoopForCausalLM,engine/patches,latent_halting_probe.pt) remains preserved in git history.
Measured benchmark (tool-eval-bench, seed 42, 69 scenarios, 16k context)
Evaluated locally on RTX 3060 via llama-server (--jinja -ngl 999 -c 16384 --flash-attn on):
| Artifact / Precision | File Size | Score | Points | Pass / Partial / Fail | Safety Warnings | Recommendation |
|---|---|---|---|---|---|---|
Step 61 baseline (E000, F32) |
โ | 24% | 33/136 | โ | 3 | Superseded baseline |
E013 F32 (-F32.gguf) |
4.77 GB | 66% | 90/136 | 35 / 20 / 14 | 0 | Canonical reference precision |
E013 BF16 (-BF16.gguf) |
2.39 GB | 65% | 88/136 | 34 / 20 / 15 | 0 | Recommended half-precision default |
E013 F16 (-F16.gguf) |
2.39 GB | 64% | 87/136 | 34 / 19 / 16 | 0 | Wide FP16 backend compatibility |
E013 Q8_0 (-Q8_0.gguf, F32 output.weight) |
1.73 GB | 59% | 80/136 | 32 / 16 / 21 | 0 | Smaller footprint (output.weight kept in F32; vanilla Q8_0 drops to 57% / 77/136) |
E013 Q4_K_M (-Q4_K_M.gguf) |
0.75 GB | 48% | 66/138 | 23 / 20 / 26 | 2 | Not recommended for agentic use (TC-48, TC-49 safety warnings); kept for :Q4_K_M tag compatibility |
Canonical E013 F32 (FF4953BBAD048859405EFE87B40D57EE60DA60080E5C91DDB18CD6F83FE79E5E) breakdown:
deployability:73,responsiveness:89,0regressions relative toE011(80/136) andE012(82/136).- Per-category points:
A=6/6,B=4/6,C=3/8,D=6/6,E=2/6,F=4/6,G=4/6,H=7/8,I=10/20,J=5/6,K=17/26,L=6/8,M=3/6,N=4/6,O=9/12.
Explicit limitations
- Quantization sensitivity: Because
E002โE013rely on small float32lm_head.weightlogit margin edits (0.0168โ0.2710L2 norm per edited row), 4-bit body quantization (:Q4_K_M) shifts hidden-state geometry enough to trigger early-send safety warnings (TC-48,TC-49). Use:F32or:BF16whenever memory allows. - Remaining 0.6B failure modes: On
seed 42F32,14of69scenarios still fail and20receive partial creditโmostly complex argument extraction, multi-turn planning depth, and unfamiliar tool schemas. - English primary: Training and tool-calling validation are overwhelmingly in English; non-English prompts degrade tool reliability.
- External gating advised for irreversible actions: Pair the model with application-level confirmation checks before destructive writes, payments, or outbound emails.
Files
GGUF (llama.cpp, Ollama, LM Studio)
qwen3loop-0.6b-tool-calling-sft-v1-BF16.gguf(2.39 GB) โ recommended half-precision (65%,0warnings)qwen3loop-0.6b-tool-calling-sft-v1-F32.gguf(4.77 GB) โ exact evaluation weights (66%,0warnings)qwen3loop-0.6b-tool-calling-sft-v1-F16.gguf(2.39 GB) โ standard FP16 (64%,0warnings)qwen3loop-0.6b-tool-calling-sft-v1-Q8_0.gguf(1.73 GB) โ Q8_0 transformer blocks + F32output.weight(59%,0warnings)qwen3loop-0.6b-tool-calling-sft-v1-Q4_K_M.gguf(0.75 GB) โ 4-bit K-quant (48%,2safety warnings; compatibility tag only)
Hugging Face Safetensors (transformers / vLLM)
model-00001-of-00002.safetensors(3.01 GB) &model-00002-of-00002.safetensors(1.76 GB)model.safetensors.index.json,config.json,generation_config.json,tokenizer.json,tokenizer_config.json,chat_template.jinja
Quickstart
llama.cpp (-hf tag matching enabled)
llama-server -hf Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1:BF16 --jinja -ngl 999 -c 16384 --flash-attn on
Other tags: :F32, :F16, :Q8_0 (and :Q4_K_M, not recommended for safety-sensitive agents).
PyTorch / transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "Lordnyx/qwen3loop-0.6b-tool-calling-sft-v1"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
repo_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
tools = [
{
"type": "function",
"function": {
"name": "convert_measurement_units",
"description": "Convert physical quantities between units.",
"parameters": {
"type": "object",
"properties": {
"value": {"type": "number"},
"from_unit": {"type": "string"},
"to_unit": {"type": "string"},
},
"required": ["value", "from_unit", "to_unit"],
},
},
}
]
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "Convert 42 km to miles."}],
tools=tools,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))
(Note: generation_config.json preserves Qwen3's default sampling parameters temperature=0.6, top_p=0.95, top_k=20, while the tool-eval-bench scores above were measured with OpenAI-compatible --jinja server tool calling.)
- Downloads last month
- 1,040