Commit
·
5aab424
1
Parent(s):
c8d6e52
Adding usage example with pipelines.
Browse files
README.md
CHANGED
@@ -10,6 +10,17 @@ We utilize automatically generated samples from Wikipedia for training, where pa
|
|
10 |
We use the same articles as ([Koshorek et al., 2018](https://arxiv.org/abs/1803.09337)),
|
11 |
albeit from a 2021 dump of Wikpeida, and split at paragraph boundaries instead of the sentence level.
|
12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
13 |
## Training Setup
|
14 |
The model was trained for 3 epochs from `bert-base-uncased` on paragraph pairs (limited to 512 subwork with the `longest_first` truncation strategy).
|
15 |
We use a batch size of 24 wit 2 iterations gradient accumulation (effective batch size of 48), and a learning rate of 1e-4, with gradient clipping at 5.
|
|
|
10 |
We use the same articles as ([Koshorek et al., 2018](https://arxiv.org/abs/1803.09337)),
|
11 |
albeit from a 2021 dump of Wikpeida, and split at paragraph boundaries instead of the sentence level.
|
12 |
|
13 |
+
## Usage
|
14 |
+
Preferred usage is through `transformers.pipeline`:
|
15 |
+
|
16 |
+
```python
|
17 |
+
from transformers import pipeline
|
18 |
+
pipe = pipeline("text-classification", model="dennlinger/bert-wiki-paragraphs")
|
19 |
+
|
20 |
+
pipe("{First paragraph} [SEP] {Second paragraph}")
|
21 |
+
```
|
22 |
+
A predicted "1" means that paragraphs belong to the same topic, a "0" indicates a disconnect.
|
23 |
+
|
24 |
## Training Setup
|
25 |
The model was trained for 3 epochs from `bert-base-uncased` on paragraph pairs (limited to 512 subwork with the `longest_first` truncation strategy).
|
26 |
We use a batch size of 24 wit 2 iterations gradient accumulation (effective batch size of 48), and a learning rate of 1e-4, with gradient clipping at 5.
|