1. going into stealth costs you time. not much, but enough to interrupt your attack sequence.
2. stealth swings are always at your lowest attack bonus. not sure if it allows TWF if you have gTWF.
most importantly:
3. stealth makes you slow. 50% move speed w/o enhancements. hard to keep up with a target that is apparently not aggroed on you.
4. stealth is interrupted when you take damage. ie: to do the death attack you must have NO aggro, or a really good defense. Increases the benefit of Improved Feint and Bluff (or diplomacy, or Subtle Backstabbing, etc) for a WotA II build.