library_name: transformers
license: apache-2.0
base_model: distilbert-base-uncased
tags:
- classification
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: imdb-distilbert
results: []
imdb-distilbert
This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.4670
- Accuracy: 0.8528
Model description
This model, named imdb-distilbert, is fine-tuned from the distilbert-base-uncased checkpoint on the IMDB movie review dataset for the sentiment classification task. It's designed to predict whether a movie review is positive or negative based on the textual content of the review. This model can be used to automatically classify new movie reviews into positive or negative categories.
Intended uses & limitations
More information needed
Training and evaluation data
The model was trained on the IMDB dataset, which contains 50,000 movie reviews split evenly into 25,000 training and 25,000 testing datasets. Each entry is labeled as either 0 (negative) or 1 (positive). It achieves the following results on the evaluation set:
- Loss: 1.4670
- Accuracy: 0.8528
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.3777 | 1.0 | 3125 | 0.4630 | 0.8263 |
| 0.2795 | 2.0 | 6250 | 0.4771 | 0.8549 |
| 0.1698 | 3.0 | 9375 | 0.5689 | 0.8526 |
| 0.1093 | 4.0 | 12500 | 0.9568 | 0.8460 |
| 0.0664 | 5.0 | 15625 | 1.0550 | 0.8470 |
| 0.0333 | 6.0 | 18750 | 1.1734 | 0.8487 |
| 0.0238 | 7.0 | 21875 | 1.1931 | 0.8482 |
| 0.0123 | 8.0 | 25000 | 1.2663 | 0.8507 |
| 0.0056 | 9.0 | 28125 | 1.3256 | 0.8549 |
| 0.0022 | 10.0 | 31250 | 1.4670 | 0.8528 |
Framework versions
- Transformers 4.47.0
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0