Displacement is too powerful to buff though 50% damage avoidance for a lasting duration at the click of a button is, just barely not OP raising that to 70% would be. If you can't see how unbalanced that would be you really don't understand the game mechanics well enough to make suggestions like this. Furthermore caster do cast it better use out of the spell then anyone else it has longer duration when they cast it and they can extend it if they chose to take that feat.
If you want to nerf it and make it work like you say where every caster level increases the concealment percentage by some amount we could talk but increasing the total concealment percentage is OP and ridiculous.