Here is my info on this. From my years of playing primarly as a bard. If the scroll is not connected to any arcane caster ASF is not bothered with. If it is a scroll a bard can cast normaly, ASF should only come to play from shields and medium+ armor. Now the rest of the spells, I found if it is just vocal no ASF but have to worry for the others.

Now so things have changed , I remeber when teleport had a pricey component and ASF was an issue for the scrolls. And the bug with dragonmarks CAN be at play here.