--- base_model: aubmindlab/bert-base-arabertv02 tags: - generated_from_trainer model-index: - name: arabert_cross_vocabulary_task2_fold0 results: [] --- # arabert_cross_vocabulary_task2_fold0 This model is a fine-tuned version of [aubmindlab/bert-base-arabertv02](https://huggingface.co/aubmindlab/bert-base-arabertv02) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.7744 - Qwk: 0.5129 - Mse: 0.7751 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 64 - eval_batch_size: 64 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 10 ### Training results | Training Loss | Epoch | Step | Validation Loss | Qwk | Mse | |:-------------:|:------:|:----:|:---------------:|:------:|:------:| | No log | 0.1333 | 2 | 6.0135 | 0.0063 | 6.0118 | | No log | 0.2667 | 4 | 3.5790 | 0.0426 | 3.5769 | | No log | 0.4 | 6 | 1.5189 | 0.1873 | 1.5182 | | No log | 0.5333 | 8 | 1.0415 | 0.1741 | 1.0415 | | No log | 0.6667 | 10 | 1.3450 | 0.2327 | 1.3453 | | No log | 0.8 | 12 | 2.7213 | 0.1379 | 2.7215 | | No log | 0.9333 | 14 | 1.9117 | 0.2036 | 1.9120 | | No log | 1.0667 | 16 | 1.0551 | 0.2973 | 1.0556 | | No log | 1.2 | 18 | 0.8085 | 0.4178 | 0.8090 | | No log | 1.3333 | 20 | 0.8599 | 0.4235 | 0.8605 | | No log | 1.4667 | 22 | 1.2293 | 0.3473 | 1.2300 | | No log | 1.6 | 24 | 1.4481 | 0.3228 | 1.4490 | | No log | 1.7333 | 26 | 1.0749 | 0.4124 | 1.0757 | | No log | 1.8667 | 28 | 0.8047 | 0.4812 | 0.8054 | | No log | 2.0 | 30 | 0.7416 | 0.5115 | 0.7424 | | No log | 2.1333 | 32 | 0.8219 | 0.4458 | 0.8228 | | No log | 2.2667 | 34 | 1.0031 | 0.4070 | 1.0041 | | No log | 2.4 | 36 | 1.1303 | 0.4041 | 1.1312 | | No log | 2.5333 | 38 | 1.1323 | 0.4064 | 1.1331 | | No log | 2.6667 | 40 | 1.0716 | 0.4240 | 1.0724 | | No log | 2.8 | 42 | 0.8847 | 0.4894 | 0.8855 | | No log | 2.9333 | 44 | 0.8007 | 0.5087 | 0.8014 | | No log | 3.0667 | 46 | 0.7579 | 0.5083 | 0.7586 | | No log | 3.2 | 48 | 0.9311 | 0.4512 | 0.9317 | | No log | 3.3333 | 50 | 1.0232 | 0.4313 | 1.0237 | | No log | 3.4667 | 52 | 0.8885 | 0.4592 | 0.8890 | | No log | 3.6 | 54 | 0.8515 | 0.4675 | 0.8520 | | No log | 3.7333 | 56 | 0.8352 | 0.4824 | 0.8355 | | No log | 3.8667 | 58 | 0.8884 | 0.4855 | 0.8887 | | No log | 4.0 | 60 | 1.0524 | 0.4439 | 1.0527 | | No log | 4.1333 | 62 | 1.1255 | 0.4180 | 1.1260 | | No log | 4.2667 | 64 | 0.9389 | 0.4769 | 0.9395 | | No log | 4.4 | 66 | 0.8175 | 0.5051 | 0.8181 | | No log | 4.5333 | 68 | 0.8469 | 0.5060 | 0.8476 | | No log | 4.6667 | 70 | 0.9671 | 0.4843 | 0.9678 | | No log | 4.8 | 72 | 0.9258 | 0.4895 | 0.9264 | | No log | 4.9333 | 74 | 0.8740 | 0.4900 | 0.8746 | | No log | 5.0667 | 76 | 0.8529 | 0.4874 | 0.8535 | | No log | 5.2 | 78 | 0.9148 | 0.4757 | 0.9154 | | No log | 5.3333 | 80 | 0.9168 | 0.4791 | 0.9175 | | No log | 5.4667 | 82 | 0.7903 | 0.4954 | 0.7910 | | No log | 5.6 | 84 | 0.7057 | 0.5514 | 0.7065 | | No log | 5.7333 | 86 | 0.7449 | 0.5282 | 0.7458 | | No log | 5.8667 | 88 | 0.8859 | 0.4980 | 0.8867 | | No log | 6.0 | 90 | 1.0776 | 0.4447 | 1.0784 | | No log | 6.1333 | 92 | 1.0638 | 0.4471 | 1.0646 | | No log | 6.2667 | 94 | 0.8896 | 0.4741 | 0.8904 | | No log | 6.4 | 96 | 0.7857 | 0.5028 | 0.7865 | | No log | 6.5333 | 98 | 0.7723 | 0.5134 | 0.7731 | | No log | 6.6667 | 100 | 0.8110 | 0.5025 | 0.8118 | | No log | 6.8 | 102 | 0.8781 | 0.4840 | 0.8789 | | No log | 6.9333 | 104 | 0.9719 | 0.4805 | 0.9727 | | No log | 7.0667 | 106 | 1.0214 | 0.4585 | 1.0221 | | No log | 7.2 | 108 | 1.0030 | 0.4702 | 1.0037 | | No log | 7.3333 | 110 | 0.8786 | 0.5008 | 0.8793 | | No log | 7.4667 | 112 | 0.8115 | 0.5083 | 0.8122 | | No log | 7.6 | 114 | 0.8162 | 0.5036 | 0.8169 | | No log | 7.7333 | 116 | 0.8235 | 0.5011 | 0.8242 | | No log | 7.8667 | 118 | 0.8121 | 0.5015 | 0.8128 | | No log | 8.0 | 120 | 0.7687 | 0.5116 | 0.7694 | | No log | 8.1333 | 122 | 0.7624 | 0.5101 | 0.7631 | | No log | 8.2667 | 124 | 0.8069 | 0.5054 | 0.8075 | | No log | 8.4 | 126 | 0.8526 | 0.5050 | 0.8532 | | No log | 8.5333 | 128 | 0.8600 | 0.5050 | 0.8606 | | No log | 8.6667 | 130 | 0.8586 | 0.5055 | 0.8593 | | No log | 8.8 | 132 | 0.8700 | 0.5019 | 0.8707 | | No log | 8.9333 | 134 | 0.8744 | 0.5019 | 0.8750 | | No log | 9.0667 | 136 | 0.8880 | 0.4899 | 0.8886 | | No log | 9.2 | 138 | 0.8928 | 0.4840 | 0.8934 | | No log | 9.3333 | 140 | 0.8662 | 0.4960 | 0.8668 | | No log | 9.4667 | 142 | 0.8312 | 0.5059 | 0.8318 | | No log | 9.6 | 144 | 0.8041 | 0.5063 | 0.8048 | | No log | 9.7333 | 146 | 0.7847 | 0.5136 | 0.7854 | | No log | 9.8667 | 148 | 0.7768 | 0.5125 | 0.7775 | | No log | 10.0 | 150 | 0.7744 | 0.5129 | 0.7751 | ### Framework versions - Transformers 4.44.0 - Pytorch 2.4.0 - Datasets 2.21.0 - Tokenizers 0.19.1