Reducing Privacy Risks in Online Self-Disclosures with Language Models
Paper • 2311.09538 • Published
You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
By requesting access to this model, you agree to follow these usage guidelines:
- Use the model only for research purposes.
- Do not redistribute the model or its associated resources without prior permission from the authors.
- Clearly acknowledge the original authors in any publications, derivative works, or other outputs created using this model.
Log in or Sign Up to review the conditions and access this model content.
Model Card for Llama-2-7B-lora-instruction-ft-abstraction-three-span
The model is used to abstract a given self-disclosure (personal information) in a sentence, which is rephrasing disclosures with less specific details while preserving the content utility. For example, "22 year old" -> "in early 20s".
For more details, read the paper: Reducing Privacy Risks in Online Self-Disclosures with Language Models .
Accessing this model implies automatic agreement to the following guidelines:
- Only use the model for research purposes.
- No redistribution without the author's agreement.
- Any derivative works created using this model must acknowledge the original author.
Model Description
- Model type: An instruction-finetuned model that can generate three diverse abstractions that moderately reduce privacy risks while maintaining high utility of the given self-disclosure span.
- Language(s) (NLP): English
- License: Creative Commons Attribution-NonCommercial
- Finetuned from model: meta-llama/Llama-2-7b-hf, with LoRA
Input Format
Your task is to generalize the given 'disclosure span' in the sentence. Provide three diverse generalized spans that convey similar meaning but remove any overly specific or sensitive information.
Remember the following criteria:
* Only the disclosure span should be generalized; the rest of the sentence should remain intact.
* Generalized spans should be diverse but should all retain the essence of the original span.
* Make sure the generalized span fits seamlessly into the original sentence, maintaining proper syntax and grammar.
* Provide three diverse generalized alternatives in a JSON format like this: {{"span 1": "xxx", "span 2": "xxx", "span 3": "xxx"}}.
Sentence: "{sentence}"
Disclosure Span to Revise: "{span}"
Generalized Spans:
input_format.format(sentence=, span=)
Example Code
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel, PeftConfig
peft_model_id = "douy/Llama-2-7B-lora-instruction-ft-abstraction-three-span"
config = PeftConfig.from_pretrained(peft_model_id)
tokenizer = AutoTokenizer.from_pretrained(peft_model_id, padding_side="left")
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, device_map="cuda:0")
model.resize_token_embeddings(32008)
model = PeftModel.from_pretrained(model, peft_model_id)
model.generation_config.top_p = 1
model.generation_config.temperature = 0.7
model.eval()
# put into your data
batch_text = [input_format.format(sentence=sentence1, span=span1), ...]
inputs = tokenizer(batch_text,return_tensors="pt", padding=True).to(0)
outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True, top_p=1, temperature=0.7, num_return_sequences=1)
input_length = inputs["input_ids"].shape[-1]
generated_tokens = outputs[:, input_length:]
generated_text = tokenizer.batch_decode(generated_tokens.detach().cpu().numpy(), skip_special_tokens=True)
Human Evaluation
We conduct a human eval in four aspects: privacy increase, utility preservation, and diversity, all rated on a 1-5 Likert scale, along with a binary assessment of coherence, evaluating whether each abstraction integrates seamlessly into the sentence.
| Privacy Increase | Utility Preservation | Diversity | Coherence |
|---|---|---|---|
| 3.2 | 4.0 | 4.6 | 94% |
Citation
@article{dou2023reducing,
title={Reducing Privacy Risks in Online Self-Disclosures with Language Models},
author={Dou, Yao and Krsek, Isadora and Naous, Tarek and Kabra, Anubha and Das, Sauvik and Ritter, Alan and Xu, Wei},
journal={arXiv preprint arXiv:2311.09538},
year={2023}
}
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for douy/Llama-2-7B-lora-instruction-ft-abstraction-three-span
Base model
meta-llama/Llama-2-7b