The fact is that Combat Expertise isn't making or breaking such characters, and I don't see many casters at all running around with AC, mid-game or late. Suddenly making CE not totally suck won't spike those figures dramatically.
Meanwhile, it is punishing the characters who should be gaining a lot of use from the feat.