If somebody is ninja looting your soul gems, stop buffing them. Say that you will no longer do mass buffs either if said ninja looter is within a close proximity to the rest of the party.

If you do not know who is stealing your gems, then threaten to stop buffing the entire party if another gem is taken again.