ymoslem commited on
Commit
26aa258
·
verified ·
1 Parent(s): ca6f8c1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +55 -11
README.md CHANGED
@@ -37,34 +37,78 @@ datasets:
37
  - ymoslem/wmt-da-human-evaluation
38
  model-index:
39
  - name: Quality Estimation for Machine Translation
40
- results: []
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  ---
42
 
43
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
44
- should probably proofread and complete it, then remove this comment. -->
45
 
46
  # Quality Estimation for Machine Translation
47
 
48
- This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on the ymoslem/wmt-da-human-evaluation dataset.
49
  It achieves the following results on the evaluation set:
50
  - Loss: 0.0561
51
 
52
  ## Model description
53
 
54
- More information needed
55
-
56
- ## Intended uses & limitations
57
-
58
- More information needed
59
 
60
  ## Training and evaluation data
61
 
62
- More information needed
63
 
64
  ## Training procedure
65
 
66
  ### Training hyperparameters
67
 
 
 
 
 
 
 
 
68
  The following hyperparameters were used during training:
69
  - learning_rate: 0.0003
70
  - train_batch_size: 128
@@ -95,4 +139,4 @@ The following hyperparameters were used during training:
95
  - Transformers 4.48.0
96
  - Pytorch 2.4.1+cu124
97
  - Datasets 3.2.0
98
- - Tokenizers 0.21.0
 
37
  - ymoslem/wmt-da-human-evaluation
38
  model-index:
39
  - name: Quality Estimation for Machine Translation
40
+ results:
41
+ - task:
42
+ type: regression
43
+ dataset:
44
+ name: ymoslem/wmt-da-human-evaluation-long-context
45
+ type: QE
46
+ metrics:
47
+ - name: Pearson
48
+ type: Pearson Correlation
49
+ value: 0.2055
50
+ - name: MAE
51
+ type: Mean Absolute Error
52
+ value: 0.2004
53
+ - name: RMSE
54
+ type: Root Mean Squared Error
55
+ value: 0.2767
56
+ - name: R-R2
57
+ type: R-Squared
58
+ value: -1.6745
59
+ - task:
60
+ type: regression
61
+ dataset:
62
+ name: ymoslem/wmt-da-human-evaluation
63
+ type: QE
64
+ metrics:
65
+ - name: Pearson
66
+ type: Pearson Correlation
67
+ value: null
68
+ - name: MAE
69
+ type: Mean Absolute Error
70
+ value: null
71
+ - name: RMSE
72
+ type: Root Mean Squared Error
73
+ value: null
74
+ - name: R-R2
75
+ type: R-Squared
76
+ value: null
77
+ metrics:
78
+ - pearsonr
79
+ - mae
80
+ - r_squared
81
+ new_version: ymoslem/ModernBERT-base-qe-v1
82
  ---
83
 
 
 
84
 
85
  # Quality Estimation for Machine Translation
86
 
87
+ This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on the [ymoslem/wmt-da-human-evaluation](https://huggingface.co/datasets/ymoslem/wmt-da-human-evaluation-long-context) dataset.
88
  It achieves the following results on the evaluation set:
89
  - Loss: 0.0561
90
 
91
  ## Model description
92
 
93
+ This model is for reference-free, sentence level quality estimation (QE) of machine translation (MT) systems.
94
+ The long-context / document-level model can be found at: [ModernBERT-base-long-context-qe-v1](https://huggingface.co/ymoslem/ModernBERT-base-long-context-qe-v1),
95
+ which is trained on a long-context / document-level QE dataset [ymoslem/wmt-da-human-evaluation-long-context](https://huggingface.co/datasets/ymoslem/wmt-da-human-evaluation-long-context)
 
 
96
 
97
  ## Training and evaluation data
98
 
99
+ This model is trained on the sentence-level quality estimation dataset: [ymoslem/wmt-da-human-evaluation](https://huggingface.co/datasets/ymoslem/wmt-da-human-evaluation)
100
 
101
  ## Training procedure
102
 
103
  ### Training hyperparameters
104
 
105
+ This version of the model uses tokenizer.model_max_length=512.
106
+ The model with full length of 8192 can be found here [ymoslem/ModernBERT-base-qe-v1](https://huggingface.co/ymoslem/ModernBERT-base-qe-v1),
107
+ which is still trained on a sentence-level QE dataset [ymoslem/wmt-da-human-evaluation](https://huggingface.co/datasets/ymoslem/wmt-da-human-evaluation)
108
+
109
+ The long-context / document-level model can be found at: [ModernBERT-base-long-context-qe-v1](https://huggingface.co/ymoslem/ModernBERT-base-long-context-qe-v1),
110
+ which is trained on a long-context / document-level QE dataset [ymoslem/wmt-da-human-evaluation-long-context](https://huggingface.co/datasets/ymoslem/wmt-da-human-evaluation-long-context)
111
+
112
  The following hyperparameters were used during training:
113
  - learning_rate: 0.0003
114
  - train_batch_size: 128
 
139
  - Transformers 4.48.0
140
  - Pytorch 2.4.1+cu124
141
  - Datasets 3.2.0
142
+ - Tokenizers 0.21.0