Apart from single target only spells (self, undead foe, undead ally, friend, foe) theres bascially 3 types of spells.
AREA OF EFFECT (persistant and instantaneous) spells: (examples freezing sphere, cloud kill, firewall)
If you have SOMETHING targetted (any target (object/player/enemy) is valid as the spell only wants a direction) it will aim in that target. (you can for example target a friendly tank with the spell, though of course the detonation will not affect them)
If the spell/ability is missile like (acid blast, fireball) even if the target becomes visually blocked by the time the spell fires, the spell will still shoot in that direction.
If you have nothing targetted they should fire at the surface you have your reticule on with mouselook on, or in the direction your avatar is facing when not in mouselook mode.
LINE and RAY spells: (examples: lightning bolt, gust of wind, prismatic ray, scorching ray)
These spell start at the caster always, and heads in an X,Y,Z direction.
If you have SOMETHING targetted (any target is valid as the spell only wants a direction) it will aim in that direction, distance to target is NOT important.
If you have nothing targetted they should fire in the direction of your "reticule" with mouselook on, or in the direction your avatar is facing when not in mouselook mode.
A ray spell will fire till it hits a VALID target then stop regardless if the target is affected or not (due to save, spell resistance etc)
A line spell will go for the RANGE of the spell hitting all things in the rectangular selection, even if you have something targetted it will continue BEYOND the target till it's reached it's range.
CONE spells: (examples: Fear, cone of cold, waves of exhaustion, prismatic spray)
Identical to line/ray spells for targetting:
These spell start at the caster always, and head in an X,Y,Z direction.
If you have SOMETHING targetted (any target is valid as the spell only wants a direction) it will aim in that direction, distance to target is NOT important.
If you have nothing targetted they should fire in the direction of your "reticule" with mouselook on, or in the direction your avatar is facing when not in mouselook.