The official reason for the delay: White iPhones are "more challenging to manufacture than we originally expected." This fits with previous reporting that the manufacturer responsible for producing the white iPhone 4 glass was struggling.
While there's likely a manufacturing glitch, there could be another reason for the delay: Apple is working on a fix for the antenna problem which it will add to the new white iPhones.
We first heard the idea on CNBC, but it's being kicked around in the MacRumors forums. At the antennagate press conference, Steve Jobs said he'd re-evaluate the free iPhone 4 case program on September 30th because, "I have no idea what solutions may come up," between now and then.
Apple doesn't want to give away iPhone 4 cases forever, and it's too late to radically overhaul the design of the phone. The most logical fix, from an outsider's perspective, is to add some sort of clear insulating material to the metal band. Apple can preserve the look and feel of the iPhone 4 and fix the antenna problem.
Rather than ship white iPhones and ship free cases, why not just ship them once with whatever fix Apple is planning?