"Where in any D&D or AD&D rulebooks does it place any restrictions at all on magic use (devices or castings) based on your attire (e.g. armor worn, baggy pants, shorts, types of shoes, etc)?It's a subtle point. The rules do restrict the armor a magic-user can wear, but don't say that magic itself is affected by armor. Men & Magic doesn't even say a magic-user can't wear armor, only that they can't use magic armor and weapons, which is why I rule that magic armor and weapons act like their mundane equivalents unless used by a fighter or cleric.
And if the answer to my question is "nowhere", where the heck do these misconceptions arise?"
The reason why some of us also make armor restrict magic is because we like the image of robed magicians and want to emphasize armor as a feature of fighters (and clerics.) Restrictions on magic use while wearing armor helps reinforce the stereotypes. I would say, though, that a magic wand is not restricted in the same way, as long as the magician isn't wearing gauntlets.