If a voltage regulator keeps a constant supply, why would a circuit need decoupling capacitors?
Regulators can't respond instantly to changes in load. The capacitors help stabilize the voltage rail and provide a current reserve when the load changes. They give time for the regulator to make adjustments.
For linear regulators, they're decoupling capacitors. For switching regulators, there is a decoupling capacitor (the input) and a filter capacitor (the output).
One mistake I often see, especially with linear regulators, is using capacitors that are far too large. Most manufacturers recommend less than 1µF for either in or out, yet people throw 470µF+.
I wrote on a blog post asking if 1µF could be too much for a decoupling capacitor.