There's a noticeable degree of prior knowledge degradation that makes certain characters present in base models less consistent than usual. Not really sure why, using additional tokens should fix it.