If you want a pure caster bard, make a pure caster bard. If you want a melee bard, make a melee bard. I'd advise against trying to combine the two, though; too easy to pull your toon in two opposite directions, particularly as there's no DPS synergies between CHA and staves.
If you were willing to go SWF instead, you could probably adapt the Count of MC into a more caster-focused configuration; e.g., drop a couple of melee feats like Shield Mastery, switch to orbs, put more APs into Spellsinger, and pick up a few more metamagics. But even that's kinda iffy due to feat shortage.