I'm not as turned off by exception-based design in general, although it's something I'm wary of, mainly because of those three points. Funny thing is, I think exception-based design is OK if the exceptions follow general rules -- in other words, don't use exception-based design when designing exceptions. Keep it simple: X Resistance gives a simple bonus against attacks and effects that have something to do with X; X Invulnerability means X can't harm you; X Attack means you add an X feature to your attack. These are called production rules, and if you a small, well-designed set of production rules and the bonuses or mechanics remain the same for all of them, you don't need a list of feats; the name of the feat tells you everything you need to know.
I've used production rules here before: the psionic disciplines I suggested, for example, or backgrounds and labels, or using risks to design magic items (or even ordinary items.) Elsewhere, I've suggested defining some special feat-like abilities in terms of "ability score bonus to a type of action or in a particular situation", like "Int bonus to melee attacks" or "Wis bonus to woodland travel". Players can think up fluff names for these talents that fit into the setting, but the only thing that matters mechanically is what you add, and when. That particular production rule handles it all.
But while I can see ways of working with feats as exceptions, what I really have trouble with is exceptional feats. (Gasp! A double meaning to the title of this post!) A lot of 3e feats, and certainly 4e powers, feel more like superpowers. Cleave, for example, feels like a minor superpower, particularly if players have to "pay" for the ability, instead of any fighter having a chance to try attacking another opponent after a killing/subduing blow. Great Cleave is certainly superheroic. Power Attack is another example. Later additions to the 3e feats, especially those by third parties, get even more powerful. I, personally, would allow any fighter to attempt to Cleave by taking a risk of some kind, because Cleave doesn't seem that out of keeping with the style of fantasy combat; what I object to is it being some kind of special ability, available only to certain characters.
I also object to the "feat gravy train". When we're talking about the 3e feat system or 4e power system, we're not just talking about one or two powers to make a character stand out; we're talking about multiple powers at first level, followed by a steady increase in powers. If you look at the original classes, Fighting Men and Magic-Users each have one general power and one scaled power:
Fighting-Man: use any weapon, multiple attacks against 1HD creatures.They don't have any other powers. They can try anything, but there are no special skills, feats, or anything else. The six ability scores aren't even as powerful as modern D&D.
Magic-User: use magic (scrolls, wands, staves,) prepare and cast spells.
Clerics came later and kind of break that pattern, but you can interpret them as having the turn undead ability plus part of the fighter's "use any weapon" ability and part of the magic-user's "prepare and cast spells" ability. Still, they stick out enough that I've thought about ways of ditching clerics. Thieves break the pattern even more, partly by having a whole bunch of abilities that scale, partly by most of the abilities seeming trivial at 1st level compared to the other classes.
Later classes start becoming superhero packages: paladins with all fighter abilities, plus turn undead, laying on of hands, protection from evil, and dispel magic is just one example. Then D&D adds proficiencies, which turn into skills in 3e, adds feats, and finally refactors some feats as powers in 4e. The superheroic feel is just not that appealing to me.
And that, to me, is a bigger reason for hating feats.