--- library_name: transformers base_model: aubmindlab/bert-base-arabertv02 tags: - generated_from_trainer model-index: - name: Arabic_FineTuningAraBERT_AugV0_k4_task1_organization_fold1 results: [] --- # Arabic_FineTuningAraBERT_AugV0_k4_task1_organization_fold1 This model is a fine-tuned version of [aubmindlab/bert-base-arabertv02](https://huggingface.co/aubmindlab/bert-base-arabertv02) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.4553 - Qwk: 0.7059 - Mse: 0.4553 - Rmse: 0.6747 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 10 ### Training results | Training Loss | Epoch | Step | Validation Loss | Qwk | Mse | Rmse | |:-------------:|:------:|:----:|:---------------:|:-------:|:------:|:------:| | No log | 0.0645 | 2 | 3.1513 | 0.0397 | 3.1513 | 1.7752 | | No log | 0.1290 | 4 | 2.5458 | -0.0868 | 2.5458 | 1.5955 | | No log | 0.1935 | 6 | 1.9193 | 0.0 | 1.9193 | 1.3854 | | No log | 0.2581 | 8 | 1.3752 | 0.0 | 1.3752 | 1.1727 | | No log | 0.3226 | 10 | 1.0491 | -0.0201 | 1.0491 | 1.0243 | | No log | 0.3871 | 12 | 0.9697 | 0.125 | 0.9697 | 0.9847 | | No log | 0.4516 | 14 | 0.9616 | 0.125 | 0.9616 | 0.9806 | | No log | 0.5161 | 16 | 0.9347 | 0.125 | 0.9347 | 0.9668 | | No log | 0.5806 | 18 | 0.9044 | 0.125 | 0.9044 | 0.9510 | | No log | 0.6452 | 20 | 0.8474 | 0.0723 | 0.8474 | 0.9205 | | No log | 0.7097 | 22 | 0.8237 | 0.1860 | 0.8237 | 0.9076 | | No log | 0.7742 | 24 | 0.7729 | 0.0982 | 0.7729 | 0.8791 | | No log | 0.8387 | 26 | 0.9338 | 0.1091 | 0.9338 | 0.9663 | | No log | 0.9032 | 28 | 0.9713 | 0.1091 | 0.9713 | 0.9855 | | No log | 0.9677 | 30 | 0.8285 | 0.1529 | 0.8285 | 0.9102 | | No log | 1.0323 | 32 | 0.6424 | 0.2410 | 0.6424 | 0.8015 | | No log | 1.0968 | 34 | 0.7408 | 0.4667 | 0.7408 | 0.8607 | | No log | 1.1613 | 36 | 0.6525 | 0.5435 | 0.6525 | 0.8078 | | No log | 1.2258 | 38 | 0.6256 | 0.3253 | 0.6256 | 0.7909 | | No log | 1.2903 | 40 | 0.6605 | 0.1840 | 0.6605 | 0.8127 | | No log | 1.3548 | 42 | 0.6306 | 0.1840 | 0.6306 | 0.7941 | | No log | 1.4194 | 44 | 0.6558 | 0.2125 | 0.6558 | 0.8098 | | No log | 1.4839 | 46 | 0.6048 | 0.3298 | 0.6048 | 0.7777 | | No log | 1.5484 | 48 | 0.5490 | 0.51 | 0.5490 | 0.7409 | | No log | 1.6129 | 50 | 0.5520 | 0.5646 | 0.5520 | 0.7429 | | No log | 1.6774 | 52 | 0.5387 | 0.5922 | 0.5387 | 0.7340 | | No log | 1.7419 | 54 | 0.5495 | 0.2784 | 0.5495 | 0.7413 | | No log | 1.8065 | 56 | 0.5493 | 0.2784 | 0.5493 | 0.7411 | | No log | 1.8710 | 58 | 0.5913 | 0.3756 | 0.5913 | 0.7690 | | No log | 1.9355 | 60 | 0.6317 | 0.2857 | 0.6317 | 0.7948 | | No log | 2.0 | 62 | 0.5659 | 0.2054 | 0.5659 | 0.7523 | | No log | 2.0645 | 64 | 0.4859 | 0.6316 | 0.4859 | 0.6971 | | No log | 2.1290 | 66 | 0.5249 | 0.5776 | 0.5249 | 0.7245 | | No log | 2.1935 | 68 | 0.4512 | 0.6160 | 0.4512 | 0.6717 | | No log | 2.2581 | 70 | 0.5069 | 0.4936 | 0.5069 | 0.7119 | | No log | 2.3226 | 72 | 0.5434 | 0.5679 | 0.5434 | 0.7371 | | No log | 2.3871 | 74 | 0.4708 | 0.7222 | 0.4708 | 0.6861 | | No log | 2.4516 | 76 | 0.4185 | 0.7407 | 0.4185 | 0.6470 | | No log | 2.5161 | 78 | 0.4417 | 0.72 | 0.4417 | 0.6646 | | No log | 2.5806 | 80 | 0.5248 | 0.6459 | 0.5248 | 0.7245 | | No log | 2.6452 | 82 | 0.4499 | 0.6912 | 0.4499 | 0.6708 | | No log | 2.7097 | 84 | 0.4598 | 0.5062 | 0.4598 | 0.6781 | | No log | 2.7742 | 86 | 0.4707 | 0.5070 | 0.4707 | 0.6861 | | No log | 2.8387 | 88 | 0.4799 | 0.5070 | 0.4799 | 0.6927 | | No log | 2.9032 | 90 | 0.4765 | 0.6529 | 0.4765 | 0.6903 | | No log | 2.9677 | 92 | 0.4959 | 0.5 | 0.4959 | 0.7042 | | No log | 3.0323 | 94 | 0.5506 | 0.5679 | 0.5506 | 0.7420 | | No log | 3.0968 | 96 | 0.5322 | 0.6231 | 0.5322 | 0.7295 | | No log | 3.1613 | 98 | 0.4719 | 0.7260 | 0.4719 | 0.6870 | | No log | 3.2258 | 100 | 0.4583 | 0.7260 | 0.4583 | 0.6770 | | No log | 3.2903 | 102 | 0.4461 | 0.7260 | 0.4461 | 0.6679 | | No log | 3.3548 | 104 | 0.4398 | 0.7260 | 0.4398 | 0.6632 | | No log | 3.4194 | 106 | 0.4419 | 0.7260 | 0.4419 | 0.6648 | | No log | 3.4839 | 108 | 0.5156 | 0.7072 | 0.5156 | 0.7180 | | No log | 3.5484 | 110 | 0.5580 | 0.6831 | 0.5580 | 0.7470 | | No log | 3.6129 | 112 | 0.4765 | 0.6723 | 0.4765 | 0.6903 | | No log | 3.6774 | 114 | 0.4562 | 0.6667 | 0.4562 | 0.6755 | | No log | 3.7419 | 116 | 0.4956 | 0.5294 | 0.4956 | 0.7040 | | No log | 3.8065 | 118 | 0.4497 | 0.6613 | 0.4497 | 0.6706 | | No log | 3.8710 | 120 | 0.4752 | 0.6983 | 0.4752 | 0.6894 | | No log | 3.9355 | 122 | 0.6267 | 0.4979 | 0.6267 | 0.7916 | | No log | 4.0 | 124 | 0.6176 | 0.4979 | 0.6176 | 0.7859 | | No log | 4.0645 | 126 | 0.4781 | 0.6983 | 0.4781 | 0.6914 | | No log | 4.1290 | 128 | 0.4388 | 0.5463 | 0.4388 | 0.6624 | | No log | 4.1935 | 130 | 0.4497 | 0.6983 | 0.4497 | 0.6706 | | No log | 4.2581 | 132 | 0.4361 | 0.6431 | 0.4361 | 0.6604 | | No log | 4.3226 | 134 | 0.4308 | 0.6912 | 0.4308 | 0.6564 | | No log | 4.3871 | 136 | 0.4227 | 0.7482 | 0.4227 | 0.6502 | | No log | 4.4516 | 138 | 0.4763 | 0.7 | 0.4763 | 0.6902 | | No log | 4.5161 | 140 | 0.6191 | 0.5817 | 0.6191 | 0.7868 | | No log | 4.5806 | 142 | 0.7091 | 0.5405 | 0.7091 | 0.8421 | | No log | 4.6452 | 144 | 0.6241 | 0.4979 | 0.6241 | 0.7900 | | No log | 4.7097 | 146 | 0.6114 | 0.4979 | 0.6114 | 0.7819 | | No log | 4.7742 | 148 | 0.5246 | 0.6983 | 0.5246 | 0.7243 | | No log | 4.8387 | 150 | 0.4603 | 0.6471 | 0.4603 | 0.6784 | | No log | 4.9032 | 152 | 0.4531 | 0.7449 | 0.4531 | 0.6731 | | No log | 4.9677 | 154 | 0.4756 | 0.6723 | 0.4756 | 0.6897 | | No log | 5.0323 | 156 | 0.5598 | 0.5044 | 0.5598 | 0.7482 | | No log | 5.0968 | 158 | 0.6308 | 0.5484 | 0.6308 | 0.7942 | | No log | 5.1613 | 160 | 0.5514 | 0.6026 | 0.5514 | 0.7426 | | No log | 5.2258 | 162 | 0.5281 | 0.6016 | 0.5281 | 0.7267 | | No log | 5.2903 | 164 | 0.4600 | 0.7222 | 0.4600 | 0.6782 | | No log | 5.3548 | 166 | 0.4097 | 0.7758 | 0.4097 | 0.6401 | | No log | 5.4194 | 168 | 0.4141 | 0.7758 | 0.4141 | 0.6435 | | No log | 5.4839 | 170 | 0.4153 | 0.7535 | 0.4153 | 0.6445 | | No log | 5.5484 | 172 | 0.4295 | 0.7348 | 0.4295 | 0.6554 | | No log | 5.6129 | 174 | 0.4822 | 0.6097 | 0.4822 | 0.6944 | | No log | 5.6774 | 176 | 0.6248 | 0.5484 | 0.6248 | 0.7905 | | No log | 5.7419 | 178 | 0.6828 | 0.5484 | 0.6828 | 0.8263 | | No log | 5.8065 | 180 | 0.6117 | 0.5484 | 0.6117 | 0.7821 | | No log | 5.8710 | 182 | 0.4695 | 0.6111 | 0.4695 | 0.6852 | | No log | 5.9355 | 184 | 0.4259 | 0.6857 | 0.4259 | 0.6526 | | No log | 6.0 | 186 | 0.4294 | 0.6857 | 0.4294 | 0.6553 | | No log | 6.0645 | 188 | 0.4517 | 0.6202 | 0.4517 | 0.6721 | | No log | 6.1290 | 190 | 0.5042 | 0.6026 | 0.5042 | 0.7100 | | No log | 6.1935 | 192 | 0.5668 | 0.4979 | 0.5668 | 0.7529 | | No log | 6.2581 | 194 | 0.5471 | 0.5917 | 0.5471 | 0.7397 | | No log | 6.3226 | 196 | 0.5437 | 0.5917 | 0.5437 | 0.7373 | | No log | 6.3871 | 198 | 0.4935 | 0.5679 | 0.4935 | 0.7025 | | No log | 6.4516 | 200 | 0.4819 | 0.6585 | 0.4819 | 0.6942 | | No log | 6.5161 | 202 | 0.4648 | 0.6540 | 0.4648 | 0.6818 | | No log | 6.5806 | 204 | 0.4744 | 0.6540 | 0.4744 | 0.6888 | | No log | 6.6452 | 206 | 0.5413 | 0.6831 | 0.5413 | 0.7358 | | No log | 6.7097 | 208 | 0.5930 | 0.6375 | 0.5930 | 0.7701 | | No log | 6.7742 | 210 | 0.5442 | 0.7244 | 0.5442 | 0.7377 | | No log | 6.8387 | 212 | 0.4453 | 0.6540 | 0.4453 | 0.6673 | | No log | 6.9032 | 214 | 0.3990 | 0.7116 | 0.3990 | 0.6317 | | No log | 6.9677 | 216 | 0.3976 | 0.7116 | 0.3976 | 0.6305 | | No log | 7.0323 | 218 | 0.4236 | 0.6842 | 0.4236 | 0.6508 | | No log | 7.0968 | 220 | 0.5093 | 0.6831 | 0.5093 | 0.7136 | | No log | 7.1613 | 222 | 0.6180 | 0.5484 | 0.6180 | 0.7861 | | No log | 7.2258 | 224 | 0.6504 | 0.5484 | 0.6504 | 0.8065 | | No log | 7.2903 | 226 | 0.6087 | 0.5484 | 0.6087 | 0.7802 | | No log | 7.3548 | 228 | 0.5257 | 0.5917 | 0.5257 | 0.7250 | | No log | 7.4194 | 230 | 0.4673 | 0.6831 | 0.4673 | 0.6836 | | No log | 7.4839 | 232 | 0.4322 | 0.6723 | 0.4322 | 0.6574 | | No log | 7.5484 | 234 | 0.4306 | 0.7059 | 0.4306 | 0.6562 | | No log | 7.6129 | 236 | 0.4521 | 0.6983 | 0.4521 | 0.6724 | | No log | 7.6774 | 238 | 0.4868 | 0.6983 | 0.4868 | 0.6977 | | No log | 7.7419 | 240 | 0.5160 | 0.5917 | 0.5160 | 0.7183 | | No log | 7.8065 | 242 | 0.5395 | 0.5917 | 0.5395 | 0.7345 | | No log | 7.8710 | 244 | 0.5223 | 0.5917 | 0.5223 | 0.7227 | | No log | 7.9355 | 246 | 0.4758 | 0.6831 | 0.4758 | 0.6898 | | No log | 8.0 | 248 | 0.4259 | 0.6744 | 0.4259 | 0.6526 | | No log | 8.0645 | 250 | 0.4097 | 0.6818 | 0.4097 | 0.6401 | | No log | 8.1290 | 252 | 0.4141 | 0.7050 | 0.4141 | 0.6435 | | No log | 8.1935 | 254 | 0.4225 | 0.6744 | 0.4225 | 0.6500 | | No log | 8.2581 | 256 | 0.4230 | 0.6744 | 0.4230 | 0.6504 | | No log | 8.3226 | 258 | 0.4420 | 0.6744 | 0.4420 | 0.6648 | | No log | 8.3871 | 260 | 0.4831 | 0.6842 | 0.4831 | 0.6950 | | No log | 8.4516 | 262 | 0.5100 | 0.7244 | 0.5100 | 0.7142 | | No log | 8.5161 | 264 | 0.5160 | 0.7244 | 0.5160 | 0.7183 | | No log | 8.5806 | 266 | 0.4935 | 0.7154 | 0.4935 | 0.7025 | | No log | 8.6452 | 268 | 0.4860 | 0.6908 | 0.4860 | 0.6971 | | No log | 8.7097 | 270 | 0.4831 | 0.6908 | 0.4831 | 0.6951 | | No log | 8.7742 | 272 | 0.4834 | 0.6831 | 0.4834 | 0.6953 | | No log | 8.8387 | 274 | 0.4700 | 0.6908 | 0.4700 | 0.6856 | | No log | 8.9032 | 276 | 0.4587 | 0.7059 | 0.4587 | 0.6773 | | No log | 8.9677 | 278 | 0.4552 | 0.7059 | 0.4552 | 0.6747 | | No log | 9.0323 | 280 | 0.4454 | 0.7059 | 0.4454 | 0.6674 | | No log | 9.0968 | 282 | 0.4449 | 0.7059 | 0.4449 | 0.6670 | | No log | 9.1613 | 284 | 0.4452 | 0.7059 | 0.4452 | 0.6672 | | No log | 9.2258 | 286 | 0.4550 | 0.7059 | 0.4550 | 0.6746 | | No log | 9.2903 | 288 | 0.4651 | 0.7319 | 0.4651 | 0.6820 | | No log | 9.3548 | 290 | 0.4760 | 0.6983 | 0.4760 | 0.6899 | | No log | 9.4194 | 292 | 0.4772 | 0.6983 | 0.4772 | 0.6908 | | No log | 9.4839 | 294 | 0.4756 | 0.6831 | 0.4756 | 0.6897 | | No log | 9.5484 | 296 | 0.4755 | 0.6831 | 0.4755 | 0.6896 | | No log | 9.6129 | 298 | 0.4715 | 0.6831 | 0.4715 | 0.6867 | | No log | 9.6774 | 300 | 0.4637 | 0.6831 | 0.4637 | 0.6810 | | No log | 9.7419 | 302 | 0.4570 | 0.7059 | 0.4570 | 0.6760 | | No log | 9.8065 | 304 | 0.4550 | 0.7059 | 0.4550 | 0.6745 | | No log | 9.8710 | 306 | 0.4551 | 0.7059 | 0.4551 | 0.6746 | | No log | 9.9355 | 308 | 0.4554 | 0.7059 | 0.4554 | 0.6749 | | No log | 10.0 | 310 | 0.4553 | 0.7059 | 0.4553 | 0.6747 | ### Framework versions - Transformers 4.44.2 - Pytorch 2.4.0+cu118 - Datasets 2.21.0 - Tokenizers 0.19.1