You set `add_prefix_space`. The tokenizer needs to be converted from the slow tokenizers
#215
by
d-hari
- opened
You set
add_prefix_space. The tokenizer needs to be converted from the slow tokenizers
I get this message when loading the pipeline. It seems to be because add_prefix_space is set to true in the json file tokenizer_3/tokenizer_config.json, so I assume there is little I can do about it (I don't want to go messing with the config unless I know what I am doing).
Can I ignore this message, or is it something that Stability AI need to fix?