I feel it IS rewarding skill and commitment. Remember the voice only adds 5% per quest. Not very substantial. On that note part of the cost of earning that extra 5% is either always wearing that trinket in lieu of any other trinket or remembering to switch it out. I think the system as it is, is perfectly fine.