Text Generation
Transformers
Safetensors
English
qwen2
security
compliance
iso27001
gdpr
eu-ai-act
endpoint-security
lora
conversational
text-generation-inference
Instructions to use Devseis/endpoint-auditor-0.5b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Devseis/endpoint-auditor-0.5b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Devseis/endpoint-auditor-0.5b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Devseis/endpoint-auditor-0.5b") model = AutoModelForCausalLM.from_pretrained("Devseis/endpoint-auditor-0.5b", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Devseis/endpoint-auditor-0.5b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Devseis/endpoint-auditor-0.5b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Devseis/endpoint-auditor-0.5b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Devseis/endpoint-auditor-0.5b
- SGLang
How to use Devseis/endpoint-auditor-0.5b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Devseis/endpoint-auditor-0.5b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Devseis/endpoint-auditor-0.5b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Devseis/endpoint-auditor-0.5b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Devseis/endpoint-auditor-0.5b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Devseis/endpoint-auditor-0.5b with Docker Model Runner:
docker model run hf.co/Devseis/endpoint-auditor-0.5b
v0.5: trained on dataset v0.5
Browse files- README.md +21 -21
- config.json +1 -1
- eval/base-eval.json +12 -6
- eval/model-eval.json +13 -7
- lora/adapter_config.json +5 -5
- lora/adapter_model.safetensors +1 -1
- model.safetensors +1 -1
README.md
CHANGED
|
@@ -16,13 +16,13 @@ tags:
|
|
| 16 |
library_name: transformers
|
| 17 |
pipeline_tag: text-generation
|
| 18 |
---
|
| 19 |
-
# Devseis Endpoint Auditor 0.5B (v0.
|
| 20 |
|
| 21 |
A small language model fine-tuned by **Devseis** to write endpoint audit findings for **ISO 27001, GDPR and the EU AI Act**
|
| 22 |
from collected evidence. It runs on the device (WebLLM in the [Devseis Endpoint Auditor](https://huggingface.co/spaces/Devseis/Arx)
|
| 23 |
desktop app and phone check), so audit evidence never leaves the computer or phone.
|
| 24 |
|
| 25 |
-
**v0.
|
| 26 |
|
| 27 |
## What the model does — and does not do
|
| 28 |
|
|
@@ -37,14 +37,12 @@ app uses a built-in template instead. Details: [docs/MODEL_LLM.md](https://huggi
|
|
| 37 |
```python
|
| 38 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 39 |
repo = "Devseis/endpoint-auditor-0.5b"
|
| 40 |
-
tok = AutoTokenizer.from_pretrained(repo, revision="v0.
|
| 41 |
-
model = AutoModelForCausalLM.from_pretrained(repo, revision="v0.
|
| 42 |
```
|
| 43 |
|
| 44 |
The LoRA adapter alone is in `lora/` (apply it to `Qwen/Qwen2.5-0.5B-Instruct` with PEFT). Browser builds:
|
| 45 |
-
[Devseis/endpoint-auditor-0.5b-q0f16-MLC](https://huggingface.co/Devseis/endpoint-auditor-0.5b-q0f16-MLC) (
|
| 46 |
-
[Devseis/endpoint-auditor-0.5b-q4f16_1-MLC](https://huggingface.co/Devseis/endpoint-auditor-0.5b-q4f16_1-MLC) (phones) and
|
| 47 |
-
[Devseis/endpoint-auditor-0.5b-q4f32_1-MLC](https://huggingface.co/Devseis/endpoint-auditor-0.5b-q4f32_1-MLC).
|
| 48 |
|
| 49 |
## Prompt format
|
| 50 |
|
|
@@ -56,11 +54,11 @@ prompt in that file. Tasks: `finding`, `summary`, `ai_classification`; the model
|
|
| 56 |
| | |
|
| 57 |
|---|---|
|
| 58 |
| Base model | [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) (Apache-2.0) |
|
| 59 |
-
| Data | [Devseis/endpoint-auditor-synthetic](https://huggingface.co/datasets/Devseis/endpoint-auditor-synthetic) revision `v0.
|
| 60 |
-
| Examples |
|
| 61 |
-
| Method | LoRA r=16, alpha=32 (8.8 M trainable parameters), lr 2e-4, 1 epoch,
|
| 62 |
| Hardware | CPU only: a 4-core Intel MacBook Pro (PyTorch 2.2, eager attention) |
|
| 63 |
-
| Final validation loss | 0.
|
| 64 |
|
| 65 |
## Evaluation
|
| 66 |
|
|
@@ -71,19 +69,21 @@ uses; *fields match* = check id / status / risk (findings), score and top gaps (
|
|
| 71 |
|
| 72 |
| Task | n | Valid JSON | Grounded | Fields match |
|
| 73 |
|---|---|---|---|---|
|
| 74 |
-
| ai_classification |
|
| 75 |
-
|
|
| 76 |
-
|
|
| 77 |
-
|
|
|
|
|
| 78 |
|
| 79 |
-
**This model (v0.
|
| 80 |
|
| 81 |
| Task | n | Valid JSON | Grounded | Fields match |
|
| 82 |
|---|---|---|---|---|
|
| 83 |
-
| ai_classification |
|
| 84 |
-
|
|
| 85 |
-
|
|
| 86 |
-
|
|
|
|
|
| 87 |
|
| 88 |
## Limitations
|
| 89 |
|
|
@@ -101,7 +101,7 @@ Apache-2.0, by **Devseis**. Please credit "Devseis Endpoint Auditor by Devseis".
|
|
| 101 |
author = {{Devseis}},
|
| 102 |
title = {Devseis Endpoint Auditor 0.5B},
|
| 103 |
year = {2026},
|
| 104 |
-
version = {v0.
|
| 105 |
url = {https://huggingface.co/Devseis/endpoint-auditor-0.5b}
|
| 106 |
}
|
| 107 |
```
|
|
|
|
| 16 |
library_name: transformers
|
| 17 |
pipeline_tag: text-generation
|
| 18 |
---
|
| 19 |
+
# Devseis Endpoint Auditor 0.5B (v0.5)
|
| 20 |
|
| 21 |
A small language model fine-tuned by **Devseis** to write endpoint audit findings for **ISO 27001, GDPR and the EU AI Act**
|
| 22 |
from collected evidence. It runs on the device (WebLLM in the [Devseis Endpoint Auditor](https://huggingface.co/spaces/Devseis/Arx)
|
| 23 |
desktop app and phone check), so audit evidence never leaves the computer or phone.
|
| 24 |
|
| 25 |
+
**v0.5:** Qwen2.5-0.5B fine-tuned for chat and reports in one model (2,400 examples: the v0.3 report tasks plus grounded chat, a third of the chat prompts carrying MY DEVICE, what the browser detected). Small enough for phones and weak graphics chips; powers the Arx chat.
|
| 26 |
|
| 27 |
## What the model does — and does not do
|
| 28 |
|
|
|
|
| 37 |
```python
|
| 38 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 39 |
repo = "Devseis/endpoint-auditor-0.5b"
|
| 40 |
+
tok = AutoTokenizer.from_pretrained(repo, revision="v0.5")
|
| 41 |
+
model = AutoModelForCausalLM.from_pretrained(repo, revision="v0.5")
|
| 42 |
```
|
| 43 |
|
| 44 |
The LoRA adapter alone is in `lora/` (apply it to `Qwen/Qwen2.5-0.5B-Instruct` with PEFT). Browser builds:
|
| 45 |
+
[Devseis/endpoint-auditor-0.5b-q0f16-MLC](https://huggingface.co/Devseis/endpoint-auditor-0.5b-q0f16-MLC), [Devseis/endpoint-auditor-0.5b-q4f16_1-MLC](https://huggingface.co/Devseis/endpoint-auditor-0.5b-q4f16_1-MLC), [Devseis/endpoint-auditor-0.5b-q4f32_1-MLC](https://huggingface.co/Devseis/endpoint-auditor-0.5b-q4f32_1-MLC).
|
|
|
|
|
|
|
| 46 |
|
| 47 |
## Prompt format
|
| 48 |
|
|
|
|
| 54 |
| | |
|
| 55 |
|---|---|
|
| 56 |
| Base model | [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) (Apache-2.0) |
|
| 57 |
+
| Data | [Devseis/endpoint-auditor-synthetic](https://huggingface.co/datasets/Devseis/endpoint-auditor-synthetic) revision `v0.5` (synthetic, CC BY 4.0) |
|
| 58 |
+
| Examples | 2400 balanced over task, OS and status (3072 tokens) |
|
| 59 |
+
| Method | LoRA r=16, alpha=32 (8.8 M trainable parameters), lr 2e-4, 1 epoch, 300 steps of 8 examples |
|
| 60 |
| Hardware | CPU only: a 4-core Intel MacBook Pro (PyTorch 2.2, eager attention) |
|
| 61 |
+
| Final validation loss | 0.0162 |
|
| 62 |
|
| 63 |
## Evaluation
|
| 64 |
|
|
|
|
| 69 |
|
| 70 |
| Task | n | Valid JSON | Grounded | Fields match |
|
| 71 |
|---|---|---|---|---|
|
| 72 |
+
| ai_classification | 4 | 0% | 0% | 0% |
|
| 73 |
+
| chat | 38 | 100% | 95% | 0% |
|
| 74 |
+
| finding | 15 | 0% | 0% | 0% |
|
| 75 |
+
| summary | 3 | 0% | 0% | 0% |
|
| 76 |
+
| **all** | 60 | **63%** | **60%** | **0%** |
|
| 77 |
|
| 78 |
+
**This model (v0.5):**
|
| 79 |
|
| 80 |
| Task | n | Valid JSON | Grounded | Fields match |
|
| 81 |
|---|---|---|---|---|
|
| 82 |
+
| ai_classification | 4 | 100% | 100% | 100% |
|
| 83 |
+
| chat | 38 | 100% | 97% | 82% |
|
| 84 |
+
| finding | 15 | 100% | 100% | 100% |
|
| 85 |
+
| summary | 3 | 100% | 100% | 100% |
|
| 86 |
+
| **all** | 60 | **100%** | **98%** | **88%** |
|
| 87 |
|
| 88 |
## Limitations
|
| 89 |
|
|
|
|
| 101 |
author = {{Devseis}},
|
| 102 |
title = {Devseis Endpoint Auditor 0.5B},
|
| 103 |
year = {2026},
|
| 104 |
+
version = {v0.5},
|
| 105 |
url = {https://huggingface.co/Devseis/endpoint-auditor-0.5b}
|
| 106 |
}
|
| 107 |
```
|
config.json
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
{
|
| 2 |
-
"_name_or_path": "
|
| 3 |
"architectures": [
|
| 4 |
"Qwen2ForCausalLM"
|
| 5 |
],
|
|
|
|
| 1 |
{
|
| 2 |
+
"_name_or_path": "training/models/Qwen2.5-0.5B-Instruct",
|
| 3 |
"architectures": [
|
| 4 |
"Qwen2ForCausalLM"
|
| 5 |
],
|
eval/base-eval.json
CHANGED
|
@@ -1,31 +1,37 @@
|
|
| 1 |
{
|
| 2 |
"model": "base model",
|
| 3 |
-
"data": "training/out-v0.
|
| 4 |
"count": 60,
|
| 5 |
"seed": 5,
|
| 6 |
"tasks": {
|
| 7 |
"ai_classification": {
|
| 8 |
-
"n":
|
| 9 |
"valid_json": 0.0,
|
| 10 |
"grounded": 0.0,
|
| 11 |
"fields_match": 0.0
|
| 12 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
"finding": {
|
| 14 |
-
"n":
|
| 15 |
"valid_json": 0.0,
|
| 16 |
"grounded": 0.0,
|
| 17 |
"fields_match": 0.0
|
| 18 |
},
|
| 19 |
"summary": {
|
| 20 |
-
"n":
|
| 21 |
"valid_json": 0.0,
|
| 22 |
"grounded": 0.0,
|
| 23 |
"fields_match": 0.0
|
| 24 |
}
|
| 25 |
},
|
| 26 |
"overall": {
|
| 27 |
-
"valid_json": 0.
|
| 28 |
-
"grounded": 0.
|
| 29 |
"fields_match": 0.0
|
| 30 |
}
|
| 31 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"model": "base model",
|
| 3 |
+
"data": "training/out-v0.5/test.jsonl",
|
| 4 |
"count": 60,
|
| 5 |
"seed": 5,
|
| 6 |
"tasks": {
|
| 7 |
"ai_classification": {
|
| 8 |
+
"n": 4,
|
| 9 |
"valid_json": 0.0,
|
| 10 |
"grounded": 0.0,
|
| 11 |
"fields_match": 0.0
|
| 12 |
},
|
| 13 |
+
"chat": {
|
| 14 |
+
"n": 38,
|
| 15 |
+
"valid_json": 1.0,
|
| 16 |
+
"grounded": 0.9473684210526315,
|
| 17 |
+
"fields_match": 0.0
|
| 18 |
+
},
|
| 19 |
"finding": {
|
| 20 |
+
"n": 15,
|
| 21 |
"valid_json": 0.0,
|
| 22 |
"grounded": 0.0,
|
| 23 |
"fields_match": 0.0
|
| 24 |
},
|
| 25 |
"summary": {
|
| 26 |
+
"n": 3,
|
| 27 |
"valid_json": 0.0,
|
| 28 |
"grounded": 0.0,
|
| 29 |
"fields_match": 0.0
|
| 30 |
}
|
| 31 |
},
|
| 32 |
"overall": {
|
| 33 |
+
"valid_json": 0.6333333333333333,
|
| 34 |
+
"grounded": 0.6,
|
| 35 |
"fields_match": 0.0
|
| 36 |
}
|
| 37 |
}
|
eval/model-eval.json
CHANGED
|
@@ -1,23 +1,29 @@
|
|
| 1 |
{
|
| 2 |
-
"model": "adapter training/models/auditor-lora-v0.
|
| 3 |
-
"data": "training/out-v0.
|
| 4 |
"count": 60,
|
| 5 |
"seed": 5,
|
| 6 |
"tasks": {
|
| 7 |
"ai_classification": {
|
| 8 |
-
"n":
|
| 9 |
"valid_json": 1.0,
|
| 10 |
"grounded": 1.0,
|
| 11 |
"fields_match": 1.0
|
| 12 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
"finding": {
|
| 14 |
-
"n":
|
| 15 |
"valid_json": 1.0,
|
| 16 |
"grounded": 1.0,
|
| 17 |
"fields_match": 1.0
|
| 18 |
},
|
| 19 |
"summary": {
|
| 20 |
-
"n":
|
| 21 |
"valid_json": 1.0,
|
| 22 |
"grounded": 1.0,
|
| 23 |
"fields_match": 1.0
|
|
@@ -25,7 +31,7 @@
|
|
| 25 |
},
|
| 26 |
"overall": {
|
| 27 |
"valid_json": 1.0,
|
| 28 |
-
"grounded":
|
| 29 |
-
"fields_match":
|
| 30 |
}
|
| 31 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"model": "adapter training/models/auditor-lora-v0.5/final",
|
| 3 |
+
"data": "training/out-v0.5/test.jsonl",
|
| 4 |
"count": 60,
|
| 5 |
"seed": 5,
|
| 6 |
"tasks": {
|
| 7 |
"ai_classification": {
|
| 8 |
+
"n": 4,
|
| 9 |
"valid_json": 1.0,
|
| 10 |
"grounded": 1.0,
|
| 11 |
"fields_match": 1.0
|
| 12 |
},
|
| 13 |
+
"chat": {
|
| 14 |
+
"n": 38,
|
| 15 |
+
"valid_json": 1.0,
|
| 16 |
+
"grounded": 0.9736842105263158,
|
| 17 |
+
"fields_match": 0.8157894736842105
|
| 18 |
+
},
|
| 19 |
"finding": {
|
| 20 |
+
"n": 15,
|
| 21 |
"valid_json": 1.0,
|
| 22 |
"grounded": 1.0,
|
| 23 |
"fields_match": 1.0
|
| 24 |
},
|
| 25 |
"summary": {
|
| 26 |
+
"n": 3,
|
| 27 |
"valid_json": 1.0,
|
| 28 |
"grounded": 1.0,
|
| 29 |
"fields_match": 1.0
|
|
|
|
| 31 |
},
|
| 32 |
"overall": {
|
| 33 |
"valid_json": 1.0,
|
| 34 |
+
"grounded": 0.9833333333333333,
|
| 35 |
+
"fields_match": 0.8833333333333333
|
| 36 |
}
|
| 37 |
}
|
lora/adapter_config.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
{
|
| 2 |
"alpha_pattern": {},
|
| 3 |
"auto_mapping": null,
|
| 4 |
-
"base_model_name_or_path": "
|
| 5 |
"bias": "none",
|
| 6 |
"fan_in_fan_out": false,
|
| 7 |
"inference_mode": true,
|
|
@@ -21,12 +21,12 @@
|
|
| 21 |
"revision": null,
|
| 22 |
"target_modules": [
|
| 23 |
"up_proj",
|
| 24 |
-
"
|
| 25 |
"down_proj",
|
|
|
|
|
|
|
| 26 |
"v_proj",
|
| 27 |
-
"o_proj"
|
| 28 |
-
"q_proj",
|
| 29 |
-
"k_proj"
|
| 30 |
],
|
| 31 |
"task_type": "CAUSAL_LM",
|
| 32 |
"use_dora": false,
|
|
|
|
| 1 |
{
|
| 2 |
"alpha_pattern": {},
|
| 3 |
"auto_mapping": null,
|
| 4 |
+
"base_model_name_or_path": "training/models/Qwen2.5-0.5B-Instruct",
|
| 5 |
"bias": "none",
|
| 6 |
"fan_in_fan_out": false,
|
| 7 |
"inference_mode": true,
|
|
|
|
| 21 |
"revision": null,
|
| 22 |
"target_modules": [
|
| 23 |
"up_proj",
|
| 24 |
+
"q_proj",
|
| 25 |
"down_proj",
|
| 26 |
+
"gate_proj",
|
| 27 |
+
"k_proj",
|
| 28 |
"v_proj",
|
| 29 |
+
"o_proj"
|
|
|
|
|
|
|
| 30 |
],
|
| 31 |
"task_type": "CAUSAL_LM",
|
| 32 |
"use_dora": false,
|
lora/adapter_model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 35237104
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:861d4711656a9ac65192f69dd705577dc84985fd0da17d5e38c5c88cdcc3e3da
|
| 3 |
size 35237104
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 988097536
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ef55a602e0e439f543e8102603fecdff3a405075265f024f7e99e4034c257200
|
| 3 |
size 988097536
|