When writing up the short rules about levels and experience points, I mentioned that levels can go up and down, but XP can only go up. That was meant to refer to how I handle level drain from undead, but it made me think about something else. I've said several times that I interpret level as a combination of confidence and reputation... but that raises the question "what happens when you piss off the locals and your reputation is shot to hell?"
Why, handle it like level drain. Your level goes down. You don't forget anything, because levels aren't memory. But you're too flustered to operate at maximum effectiveness, and you can't regain all your luck (hit points.)
To regain a level of lost reputation, you need to go on an adventure locally and earn at least 1 XP. If you've lost several levels, maybe by committing multiple catastrophic blunders, you need to slowly regain your reputation, one adventure at a time. Your successful deeds eventually wipes that condescending smirk off the faces of those judgmental townies.
This also opens the door for bards or character assassins satirizing your exploits and deliberately ruining your reputation as a sort of level drain attack. Although unlike the undead, a bard can't completely destroy you; you will always be at least 1st level. In fact, I think a satire or character assassination should be limited as to how many levels below your max it can drop you. But I'll deal with that some other time.