Can I 'protect' my 'reputation' if I change jobs?

It is against the law for him to do that. If he does, you can sue him. You may be over obsessing. Try to make this transition as easy as it can be - if you keep thinking about the worst, you will cause it to happen by focusing so much attention on it.