OS
Linux
GPU Library
CUDA 12.x
Python version
3.12
Describe the bug
banned_strings=[] OK
banned_strings=["."] OK
banned_strings=[".."] OK
banned_strings=["..."] breaks
banned_strings=[".foo."] breaks
banned_strings=["---"] OK
It seems that any literal string of at least 3 characters which includes a . at in position 0 corrupts the model's output (usually just emitting the stop token, but sometimes gibberish or an incomplete message).
Reproduction steps
Pass banned_strings="..." for a simple reproduction.
I tested with EXL3 and Mistral-Small 24B.
Expected behavior
Expected behavior would be for literal strings to match verbatim instead of getting confused with regex or backtracking.
Logs
No response
Additional context
No response
Acknowledgements
OS
Linux
GPU Library
CUDA 12.x
Python version
3.12
Describe the bug
banned_strings=[] OK
banned_strings=["."] OK
banned_strings=[".."] OK
banned_strings=["..."] breaks
banned_strings=[".foo."] breaks
banned_strings=["---"] OK
It seems that any literal string of at least 3 characters which includes a
.at in position 0 corrupts the model's output (usually just emitting the stop token, but sometimes gibberish or an incomplete message).Reproduction steps
Pass
banned_strings="..."for a simple reproduction.I tested with EXL3 and Mistral-Small 24B.
Expected behavior
Expected behavior would be for literal strings to match verbatim instead of getting confused with regex or backtracking.
Logs
No response
Additional context
No response
Acknowledgements