To begin elaborating how I would differentiate weapons if they all do the same 1d6 damage, let's start with the basics: weapon proficiencies. I'm against the later D&D-style proficiency slots; I prefer the original D&D distinction between fighters who can use any and all weapons versus magic-users who can only use very basic weapons that anyone could use (knife/dagger, club, staff, thrown rock.)
But (critics reply) what's the point of limiting magic-users to a handful of weapons or giving fighters the "power" of being ability to use absolutely any weapon if they all do the same damage? The thing is, not all weapons do damage in the same way. There are definitely some things a dagger can't do; it should take much longer to chop through a door using a dagger than it would with an ax, for example. Many of these differences are going to be the topic of individual posts.
The other objection usually raised is how unrealistic it is to say that a magic-user can't pick up and swing a sword. AD&D answered this objection with non-proficiency penalties, but I dislike those. My own opinion is that characters only get their level-based (or HD-based) attack bonus if they are using a weapon appropriate to their class. It's not that a magic-user can't use a sword or doesn't know how; it's that magic-users aren't any better than a common man-at-arms when using a sword.
Next: the most obvious difference between weapons, and how I'd use them in play.