README.md

metadata

library_name: transformers
base_model: aubmindlab/bert-base-arabertv02
tags:
  - generated_from_trainer
model-index:
  - name: >-
      ArabicNewSplits8_usingALLEssays_FineTuningAraBERT_run3_AugV5_k3_task2_organization
    results: []

ArabicNewSplits8_usingALLEssays_FineTuningAraBERT_run3_AugV5_k3_task2_organization

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6300
• Qwk: 0.4672
• Mse: 0.6300
• Rmse: 0.7937

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	0.125	2	4.2232	-0.0389	4.2232	2.0550
No log	0.25	4	2.3940	0.0725	2.3940	1.5473
No log	0.375	6	1.1967	0.0258	1.1967	1.0939
No log	0.5	8	0.8550	0.1725	0.8550	0.9247
No log	0.625	10	0.8075	0.1281	0.8075	0.8986
No log	0.75	12	0.8107	0.2125	0.8107	0.9004
No log	0.875	14	0.8099	0.1697	0.8099	0.8999
No log	1.0	16	0.7874	0.2416	0.7874	0.8874
No log	1.125	18	0.8037	0.2637	0.8037	0.8965
No log	1.25	20	1.0139	0.1624	1.0139	1.0069
No log	1.375	22	0.8189	0.2477	0.8189	0.9049
No log	1.5	24	0.7079	0.3242	0.7079	0.8414
No log	1.625	26	0.7761	0.1984	0.7761	0.8809
No log	1.75	28	0.8043	0.2693	0.8043	0.8969
No log	1.875	30	0.7486	0.3144	0.7486	0.8652
No log	2.0	32	0.6989	0.4324	0.6989	0.8360
No log	2.125	34	0.8100	0.3236	0.8100	0.9000
No log	2.25	36	0.9496	0.3091	0.9496	0.9745
No log	2.375	38	0.9852	0.3363	0.9852	0.9926
No log	2.5	40	0.8960	0.2818	0.8960	0.9466
No log	2.625	42	0.6725	0.3963	0.6725	0.8200
No log	2.75	44	0.6478	0.3753	0.6478	0.8049
No log	2.875	46	0.6514	0.3679	0.6514	0.8071
No log	3.0	48	0.6722	0.3728	0.6722	0.8199
No log	3.125	50	0.8170	0.3552	0.8170	0.9039
No log	3.25	52	0.7791	0.3447	0.7791	0.8827
No log	3.375	54	0.6281	0.3832	0.6281	0.7925
No log	3.5	56	0.6291	0.3855	0.6291	0.7931
No log	3.625	58	0.6261	0.4402	0.6261	0.7912
No log	3.75	60	0.9393	0.3649	0.9393	0.9692
No log	3.875	62	1.0822	0.3362	1.0822	1.0403
No log	4.0	64	0.8743	0.3614	0.8743	0.9351
No log	4.125	66	0.6161	0.4824	0.6161	0.7849
No log	4.25	68	0.6149	0.5046	0.6149	0.7841
No log	4.375	70	0.7142	0.4976	0.7142	0.8451
No log	4.5	72	0.6250	0.4877	0.6250	0.7906
No log	4.625	74	0.5578	0.4713	0.5578	0.7469
No log	4.75	76	0.5865	0.4251	0.5865	0.7658
No log	4.875	78	0.6383	0.4323	0.6383	0.7989
No log	5.0	80	0.6252	0.4255	0.6252	0.7907
No log	5.125	82	0.5551	0.4471	0.5551	0.7451
No log	5.25	84	0.5494	0.4557	0.5494	0.7412
No log	5.375	86	0.5540	0.5161	0.5540	0.7443
No log	5.5	88	0.5724	0.5616	0.5724	0.7566
No log	5.625	90	0.6108	0.5577	0.6108	0.7816
No log	5.75	92	0.6930	0.5785	0.6930	0.8324
No log	5.875	94	0.7425	0.5429	0.7425	0.8617
No log	6.0	96	0.6432	0.5537	0.6432	0.8020
No log	6.125	98	0.7911	0.5055	0.7911	0.8894
No log	6.25	100	0.8989	0.4440	0.8989	0.9481
No log	6.375	102	0.7286	0.5148	0.7286	0.8536
No log	6.5	104	0.5798	0.5121	0.5798	0.7615
No log	6.625	106	0.6838	0.4814	0.6838	0.8269
No log	6.75	108	0.6326	0.5394	0.6326	0.7954
No log	6.875	110	0.5927	0.5390	0.5927	0.7698
No log	7.0	112	0.6466	0.4709	0.6466	0.8041
No log	7.125	114	0.6195	0.5189	0.6195	0.7871
No log	7.25	116	0.5604	0.5819	0.5604	0.7486
No log	7.375	118	0.5574	0.5688	0.5574	0.7466
No log	7.5	120	0.5552	0.5288	0.5552	0.7451
No log	7.625	122	0.6560	0.4948	0.6560	0.8100
No log	7.75	124	0.6615	0.5036	0.6615	0.8133
No log	7.875	126	0.5754	0.4730	0.5754	0.7585
No log	8.0	128	0.5758	0.6107	0.5758	0.7588
No log	8.125	130	0.6034	0.5642	0.6034	0.7768
No log	8.25	132	0.5977	0.5768	0.5977	0.7731
No log	8.375	134	0.6395	0.5517	0.6395	0.7997
No log	8.5	136	0.6084	0.5807	0.6084	0.7800
No log	8.625	138	0.6661	0.4967	0.6661	0.8162
No log	8.75	140	0.6772	0.5111	0.6772	0.8229
No log	8.875	142	0.6087	0.4930	0.6087	0.7802
No log	9.0	144	0.7143	0.5405	0.7143	0.8452
No log	9.125	146	0.7175	0.4913	0.7175	0.8471
No log	9.25	148	0.6025	0.4668	0.6025	0.7762
No log	9.375	150	0.6576	0.4410	0.6576	0.8109
No log	9.5	152	0.6581	0.4379	0.6581	0.8113
No log	9.625	154	0.6462	0.4882	0.6462	0.8039
No log	9.75	156	0.6187	0.5186	0.6187	0.7866
No log	9.875	158	0.6367	0.5073	0.6367	0.7979
No log	10.0	160	0.7207	0.4832	0.7207	0.8489
No log	10.125	162	0.7290	0.4451	0.7290	0.8538
No log	10.25	164	0.6272	0.5298	0.6272	0.7920
No log	10.375	166	0.6300	0.5321	0.6300	0.7937
No log	10.5	168	0.6319	0.5132	0.6319	0.7949
No log	10.625	170	0.6350	0.5466	0.6350	0.7968
No log	10.75	172	0.7571	0.5039	0.7571	0.8701
No log	10.875	174	0.7026	0.4705	0.7026	0.8382
No log	11.0	176	0.6296	0.5044	0.6296	0.7935
No log	11.125	178	0.6683	0.4675	0.6683	0.8175
No log	11.25	180	0.6602	0.4355	0.6602	0.8125
No log	11.375	182	0.6387	0.5236	0.6387	0.7992
No log	11.5	184	0.6116	0.4686	0.6116	0.7820
No log	11.625	186	0.6271	0.4332	0.6271	0.7919
No log	11.75	188	0.7281	0.4503	0.7281	0.8533
No log	11.875	190	0.7346	0.4503	0.7346	0.8571
No log	12.0	192	0.6882	0.5122	0.6882	0.8296
No log	12.125	194	0.6778	0.5046	0.6778	0.8233
No log	12.25	196	0.7157	0.4787	0.7157	0.8460
No log	12.375	198	0.7229	0.4935	0.7229	0.8502
No log	12.5	200	0.6998	0.5117	0.6998	0.8365
No log	12.625	202	0.6777	0.5167	0.6777	0.8232
No log	12.75	204	0.6676	0.5242	0.6676	0.8171
No log	12.875	206	0.6282	0.5114	0.6282	0.7926
No log	13.0	208	0.6269	0.4575	0.6269	0.7918
No log	13.125	210	0.6110	0.4605	0.6110	0.7817
No log	13.25	212	0.6054	0.5228	0.6054	0.7780
No log	13.375	214	0.6157	0.5260	0.6157	0.7847
No log	13.5	216	0.6078	0.5232	0.6078	0.7796
No log	13.625	218	0.6684	0.4922	0.6684	0.8176
No log	13.75	220	0.7062	0.5013	0.7062	0.8404
No log	13.875	222	0.6536	0.4999	0.6536	0.8084
No log	14.0	224	0.6686	0.5287	0.6686	0.8177
No log	14.125	226	0.7243	0.5044	0.7243	0.8510
No log	14.25	228	0.6730	0.5415	0.6730	0.8204
No log	14.375	230	0.6523	0.4873	0.6523	0.8077
No log	14.5	232	0.6586	0.4837	0.6586	0.8115
No log	14.625	234	0.6628	0.4681	0.6628	0.8141
No log	14.75	236	0.6318	0.5241	0.6318	0.7948
No log	14.875	238	0.6518	0.5645	0.6518	0.8073
No log	15.0	240	0.6514	0.5667	0.6514	0.8071
No log	15.125	242	0.6290	0.5325	0.6290	0.7931
No log	15.25	244	0.6792	0.4775	0.6792	0.8241
No log	15.375	246	0.7065	0.4765	0.7065	0.8405
No log	15.5	248	0.6802	0.4664	0.6802	0.8247
No log	15.625	250	0.6812	0.4528	0.6812	0.8253
No log	15.75	252	0.7120	0.4562	0.7120	0.8438
No log	15.875	254	0.6860	0.4471	0.6860	0.8282
No log	16.0	256	0.6885	0.4716	0.6885	0.8297
No log	16.125	258	0.6929	0.5176	0.6929	0.8324
No log	16.25	260	0.6873	0.5108	0.6873	0.8291
No log	16.375	262	0.6972	0.4413	0.6972	0.8350
No log	16.5	264	0.6954	0.4644	0.6954	0.8339
No log	16.625	266	0.6484	0.4104	0.6484	0.8052
No log	16.75	268	0.6277	0.5193	0.6277	0.7923
No log	16.875	270	0.6320	0.5409	0.6320	0.7950
No log	17.0	272	0.6257	0.5367	0.6257	0.7910
No log	17.125	274	0.6353	0.5065	0.6353	0.7971
No log	17.25	276	0.6952	0.4702	0.6952	0.8338
No log	17.375	278	0.7412	0.4678	0.7412	0.8609
No log	17.5	280	0.6706	0.4672	0.6706	0.8189
No log	17.625	282	0.6264	0.5137	0.6264	0.7914
No log	17.75	284	0.6507	0.5310	0.6507	0.8066
No log	17.875	286	0.6350	0.5115	0.6350	0.7969
No log	18.0	288	0.6237	0.4985	0.6237	0.7898
No log	18.125	290	0.6511	0.4913	0.6511	0.8069
No log	18.25	292	0.6524	0.4854	0.6524	0.8077
No log	18.375	294	0.6286	0.5071	0.6286	0.7928
No log	18.5	296	0.6285	0.5282	0.6285	0.7928
No log	18.625	298	0.6267	0.5162	0.6267	0.7916
No log	18.75	300	0.6196	0.5276	0.6196	0.7871
No log	18.875	302	0.6222	0.5276	0.6222	0.7888
No log	19.0	304	0.6223	0.4858	0.6223	0.7888
No log	19.125	306	0.6185	0.4991	0.6185	0.7864
No log	19.25	308	0.6128	0.5005	0.6128	0.7828
No log	19.375	310	0.6109	0.4921	0.6109	0.7816
No log	19.5	312	0.6214	0.5240	0.6214	0.7883
No log	19.625	314	0.6313	0.5253	0.6313	0.7945
No log	19.75	316	0.6242	0.5321	0.6242	0.7901
No log	19.875	318	0.6249	0.5105	0.6249	0.7905
No log	20.0	320	0.6361	0.5150	0.6361	0.7975
No log	20.125	322	0.6614	0.4500	0.6614	0.8132
No log	20.25	324	0.6473	0.4794	0.6473	0.8045
No log	20.375	326	0.6260	0.4698	0.6260	0.7912
No log	20.5	328	0.6185	0.4709	0.6185	0.7865
No log	20.625	330	0.6234	0.4585	0.6234	0.7896
No log	20.75	332	0.6379	0.5111	0.6379	0.7987
No log	20.875	334	0.6348	0.4992	0.6348	0.7968
No log	21.0	336	0.6466	0.5258	0.6466	0.8041
No log	21.125	338	0.6793	0.5493	0.6793	0.8242
No log	21.25	340	0.7144	0.4823	0.7144	0.8452
No log	21.375	342	0.6969	0.4868	0.6969	0.8348
No log	21.5	344	0.6619	0.4927	0.6619	0.8136
No log	21.625	346	0.6172	0.4991	0.6172	0.7856
No log	21.75	348	0.6162	0.4818	0.6162	0.7850
No log	21.875	350	0.6091	0.4768	0.6091	0.7805
No log	22.0	352	0.6075	0.4859	0.6075	0.7794
No log	22.125	354	0.6253	0.5006	0.6253	0.7908
No log	22.25	356	0.6873	0.4696	0.6873	0.8290
No log	22.375	358	0.6915	0.5262	0.6915	0.8316
No log	22.5	360	0.6424	0.5079	0.6424	0.8015
No log	22.625	362	0.6336	0.5392	0.6336	0.7960
No log	22.75	364	0.6552	0.5481	0.6552	0.8094
No log	22.875	366	0.6476	0.5442	0.6476	0.8048
No log	23.0	368	0.6321	0.5176	0.6321	0.7951
No log	23.125	370	0.6208	0.5325	0.6208	0.7879
No log	23.25	372	0.6155	0.5191	0.6155	0.7846
No log	23.375	374	0.6493	0.5103	0.6493	0.8058
No log	23.5	376	0.6744	0.4915	0.6744	0.8212
No log	23.625	378	0.6964	0.4996	0.6964	0.8345
No log	23.75	380	0.6432	0.5229	0.6432	0.8020
No log	23.875	382	0.6116	0.5381	0.6116	0.7821
No log	24.0	384	0.6526	0.5485	0.6526	0.8078
No log	24.125	386	0.6499	0.5566	0.6499	0.8062
No log	24.25	388	0.6253	0.5513	0.6253	0.7907
No log	24.375	390	0.6187	0.5042	0.6187	0.7866
No log	24.5	392	0.6309	0.5245	0.6309	0.7943
No log	24.625	394	0.6226	0.5167	0.6226	0.7890
No log	24.75	396	0.6487	0.5571	0.6487	0.8054
No log	24.875	398	0.7193	0.5680	0.7193	0.8481
No log	25.0	400	0.8125	0.5202	0.8125	0.9014
No log	25.125	402	0.8191	0.5115	0.8191	0.9050
No log	25.25	404	0.7265	0.5433	0.7265	0.8523
No log	25.375	406	0.6465	0.5748	0.6465	0.8041
No log	25.5	408	0.6193	0.5284	0.6193	0.7869
No log	25.625	410	0.6125	0.5116	0.6125	0.7827
No log	25.75	412	0.5998	0.4738	0.5998	0.7745
No log	25.875	414	0.6146	0.4951	0.6146	0.7839
No log	26.0	416	0.6308	0.5087	0.6308	0.7942
No log	26.125	418	0.6345	0.5073	0.6345	0.7966
No log	26.25	420	0.6279	0.5504	0.6279	0.7924
No log	26.375	422	0.6210	0.5330	0.6210	0.7880
No log	26.5	424	0.6316	0.5295	0.6316	0.7947
No log	26.625	426	0.6280	0.5572	0.6280	0.7925
No log	26.75	428	0.6264	0.5657	0.6264	0.7915
No log	26.875	430	0.6396	0.5069	0.6396	0.7997
No log	27.0	432	0.7081	0.4528	0.7081	0.8415
No log	27.125	434	0.7989	0.4402	0.7989	0.8938
No log	27.25	436	0.7852	0.4350	0.7852	0.8861
No log	27.375	438	0.6867	0.4627	0.6867	0.8287
No log	27.5	440	0.6120	0.4978	0.6120	0.7823
No log	27.625	442	0.6106	0.5517	0.6106	0.7814
No log	27.75	444	0.6151	0.5777	0.6151	0.7843
No log	27.875	446	0.6205	0.5567	0.6205	0.7877
No log	28.0	448	0.6380	0.5565	0.6380	0.7988
No log	28.125	450	0.6570	0.5728	0.6570	0.8105
No log	28.25	452	0.6490	0.5657	0.6490	0.8056
No log	28.375	454	0.6313	0.5437	0.6313	0.7945
No log	28.5	456	0.6232	0.5312	0.6232	0.7895
No log	28.625	458	0.6230	0.5685	0.6230	0.7893
No log	28.75	460	0.6247	0.5685	0.6247	0.7904
No log	28.875	462	0.6149	0.5554	0.6149	0.7842
No log	29.0	464	0.6126	0.5415	0.6126	0.7827
No log	29.125	466	0.6111	0.5415	0.6111	0.7817
No log	29.25	468	0.6003	0.5410	0.6003	0.7748
No log	29.375	470	0.5914	0.5157	0.5914	0.7690
No log	29.5	472	0.5903	0.5334	0.5903	0.7683
No log	29.625	474	0.5982	0.5505	0.5982	0.7734
No log	29.75	476	0.6124	0.5611	0.6124	0.7825
No log	29.875	478	0.6200	0.5899	0.6200	0.7874
No log	30.0	480	0.6302	0.5525	0.6302	0.7938
No log	30.125	482	0.6296	0.5499	0.6296	0.7934
No log	30.25	484	0.6122	0.5677	0.6122	0.7824
No log	30.375	486	0.6004	0.5243	0.6004	0.7749
No log	30.5	488	0.5928	0.5282	0.5928	0.7699
No log	30.625	490	0.5851	0.5340	0.5851	0.7649
No log	30.75	492	0.5901	0.5191	0.5901	0.7682
No log	30.875	494	0.6145	0.5362	0.6145	0.7839
No log	31.0	496	0.6234	0.5318	0.6234	0.7896
No log	31.125	498	0.6330	0.5332	0.6330	0.7956
0.2975	31.25	500	0.6403	0.5309	0.6403	0.8002
0.2975	31.375	502	0.6401	0.5343	0.6401	0.8001
0.2975	31.5	504	0.6266	0.5438	0.6266	0.7916
0.2975	31.625	506	0.6127	0.5576	0.6127	0.7827
0.2975	31.75	508	0.6031	0.5735	0.6031	0.7766
0.2975	31.875	510	0.6001	0.5735	0.6001	0.7746
0.2975	32.0	512	0.6141	0.5450	0.6141	0.7836
0.2975	32.125	514	0.6304	0.5404	0.6304	0.7940
0.2975	32.25	516	0.6383	0.5668	0.6383	0.7989
0.2975	32.375	518	0.6469	0.5512	0.6469	0.8043
0.2975	32.5	520	0.6369	0.5610	0.6369	0.7981
0.2975	32.625	522	0.6160	0.5620	0.6160	0.7848
0.2975	32.75	524	0.6499	0.5432	0.6499	0.8061
0.2975	32.875	526	0.6512	0.5512	0.6512	0.8070
0.2975	33.0	528	0.6272	0.5642	0.6272	0.7920
0.2975	33.125	530	0.6658	0.5337	0.6658	0.8160
0.2975	33.25	532	0.7295	0.4625	0.7295	0.8541
0.2975	33.375	534	0.7420	0.4731	0.7420	0.8614
0.2975	33.5	536	0.6910	0.4982	0.6910	0.8313
0.2975	33.625	538	0.6375	0.5392	0.6375	0.7984
0.2975	33.75	540	0.6484	0.5668	0.6484	0.8052
0.2975	33.875	542	0.6642	0.5767	0.6642	0.8150
0.2975	34.0	544	0.6597	0.5633	0.6597	0.8122
0.2975	34.125	546	0.6465	0.5466	0.6465	0.8040
0.2975	34.25	548	0.6539	0.5461	0.6539	0.8087
0.2975	34.375	550	0.6877	0.5024	0.6877	0.8293
0.2975	34.5	552	0.7181	0.4777	0.7181	0.8474
0.2975	34.625	554	0.7077	0.4743	0.7077	0.8413
0.2975	34.75	556	0.6733	0.5025	0.6733	0.8206
0.2975	34.875	558	0.6339	0.5492	0.6339	0.7962
0.2975	35.0	560	0.6310	0.5699	0.6310	0.7944
0.2975	35.125	562	0.6454	0.5624	0.6454	0.8034
0.2975	35.25	564	0.6452	0.5624	0.6452	0.8032
0.2975	35.375	566	0.6366	0.5598	0.6366	0.7979
0.2975	35.5	568	0.6391	0.5497	0.6391	0.7995
0.2975	35.625	570	0.6415	0.5497	0.6415	0.8009
0.2975	35.75	572	0.6482	0.5501	0.6482	0.8051
0.2975	35.875	574	0.6435	0.5451	0.6435	0.8022
0.2975	36.0	576	0.6378	0.5427	0.6378	0.7986
0.2975	36.125	578	0.6277	0.5451	0.6277	0.7923
0.2975	36.25	580	0.6205	0.4897	0.6205	0.7877
0.2975	36.375	582	0.6119	0.5151	0.6119	0.7822
0.2975	36.5	584	0.5997	0.4539	0.5997	0.7744
0.2975	36.625	586	0.6155	0.4692	0.6155	0.7845
0.2975	36.75	588	0.6300	0.4672	0.6300	0.7937

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu118
• Datasets 2.21.0
• Tokenizers 0.19.1