I added a little variation to slimes and oozes, so I might as well do the same for molds...
The various molds, like slimes, act primarily as traps. There's no point in assigning them hit dice: just assume that a patch of mold up to about a yard or meter across has 1 hit point.
A basic mold is immune to attacks with melee weapons; if it has spores, melee attacks have a 50-50 chance of releasing a cloud in the area. For common molds, this has no real effect unless the victim is allergic (1 in 6 chance on first exposure;) allergic victims have to save vs. poison or Con save to avoid a 1-point penalty on attacks or actions for up to a full turn unless the victim takes steps to counteract (get away from the spore cloud, wash off, and so on.) These are all brown, black, or grey.
There's a 1 in 6 chance that an ordinary mold will be special, like the Lung Rot mentioned in the slimes and parasites. These have extras like parasitic infestation, contact poison, or even something innocuous like phosphorescent glow or edibility. Each color of ordinary mold has a vulnerability to an ordinary substance, handled as for slimes and oozes.
Extraordinary molds -- like the traditional yellow and brown molds -- grow in much more substantial patches. Each color of extraordinary mold has only one major vulnerability, like fire (yellow) or cold (brown) or explosive (red). They each have a contact or proximity attack, but are still non-mobile and mindless.