en-vi-machine-translation

This is a custom Transformer encoder-decoder model. Training from scratch on iwslt2015-en-vi datasets.

It achieves the following results on the evaluation set:

• Loss: 4.3761

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 350
• eval_batch_size: 350
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss
7.5582	1.0	381	6.3939
6.0664	2.0	762	5.7502
5.6536	3.0	1143	5.4572
5.3981	4.0	1524	5.2329
5.199	5.0	1905	5.0636
5.0443	6.0	2286	4.9307
4.9222	7.0	2667	4.8311
4.8242	8.0	3048	4.7455
4.7445	9.0	3429	4.6765
4.6778	10.0	3810	4.6196
4.6218	11.0	4191	4.5714
4.5751	12.0	4572	4.5287
4.5343	13.0	4953	4.4960
4.5014	14.0	5334	4.4704
4.4739	15.0	5715	4.4467
4.4506	16.0	6096	4.4270
4.4324	17.0	6477	4.4121
4.417	18.0	6858	4.3996
4.4056	19.0	7239	4.3922
4.3967	20.0	7620	4.3843
4.3908	21.0	8001	4.3807
4.3865	22.0	8382	4.3784
4.3844	23.0	8763	4.3766
4.3838	24.0	9144	4.3761
4.3829	25.0	9525	4.3761

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu118
• Datasets 3.0.1
• Tokenizers 0.19.1