I can cope with this effect better than with hanging tapestries and plants, algae, you name it, which blocks user vision. But still, challenge should be created by chaarcter vision, not player vision. There is a a fine line but for example, I'd rather have more quests like Rainbow In The Dark than Shadowfell experiences.... I really like the idea and the immersion but after What Goes Up and an annoyingly confusing explorer area in Wheloon with random purple haze events it gets physical in Through A Mirror Darkly.
You know how to solve this issue? Just put an effect neutralizing feature on Shadowsight. That gave a trade-off for a complete different aspect: item slots. Or something like that silly dwarf in The Underdark, who gives you Goggles for free. Just something to prevent longterm effects.