pujpuj/roberta-lora-token-classification

Model Card for RoBERTa LoRA Fine-Tuned for Insurance Review Rating

This model is a fine-tuned version of RoBERTa (roberta-large) using LoRA adapters. It is specifically designed to classify English insurance reviews and assign a rating (on a scale of 1 to 5).

Model Details

Model Description

This model uses RoBERTa (roberta-large) as its base architecture and was fine-tuned using Low-Rank Adaptation (LoRA) to adapt efficiently to the task of insurance review classification. The model predicts a rating from 1 to 5 based on the sentiment and context of a given review. LoRA fine-tuning reduces memory overhead and enables faster training compared to full fine-tuning.

• Developed by: Lapujpuj
• Finetuned from model: RoBERTa (roberta-large)
• Language(s) (NLP): English
• License: Apache-2.0
• LoRA Configuration:
  • Rank (r): 2
  • LoRA Alpha: 16
  • LoRA Dropout: 0.1
• Task: Sentiment-based rating prediction for insurance reviews

Model Sources

• Repository: pujpuj/roberta-lora-token-classification
• Demo: N/A

---

Uses

Direct Use

This model can be directly used to assign a sentiment-based rating to insurance reviews. Input text is expected to be a sentence or paragraph in English.

Downstream Use

The model can be used as a building block for larger applications, such as customer feedback analysis, satisfaction prediction, or insurance service improvement.

Out-of-Scope Use

• The model is not designed for reviews in languages other than English.
• It may not generalize well to domains outside of insurance-related reviews.
• Avoid using the model for biased or malicious predictions.

---

Bias, Risks, and Limitations

Bias

• The model is trained on a specific dataset of insurance reviews, which might include biases present in the training data (e.g., skewed ratings, linguistic or cultural biases).

Risks

• Predictions might not generalize well to other domains or review styles.
• Inconsistent predictions may occur for ambiguous or mixed reviews.

Recommendations

• Always validate model outputs before making decisions.
• Use the model in conjunction with other tools for a more comprehensive analysis.

---

How to Get Started with the Model

You can use the model with the following code snippet:

from transformers import AutoTokenizer
from peft import PeftModel

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("roberta-large")
base_model = AutoModelForSequenceClassification.from_pretrained("roberta-large", num_labels=5)
model = PeftModel.from_pretrained(base_model, "pujpuj/roberta-lora-token-classification")

# Example prediction
review = "The insurance service was quick and reliable."
inputs = tokenizer(review, return_tensors="pt", truncation=True, padding=True)
outputs = model(**inputs)
rating = torch.argmax(outputs.logits, dim=1).item() + 1
print(f"Predicted rating: {rating}")