The Eternal Wand of Ray of Frost's cast time isn't as fast as the other cantrip wands'.
I started using both alternatively on my newborn caster, and the speed is REALLY noticeable.
Yet Finger of Fire's base damage is higher than Ray of Frost's (1d4 vs 1d3)

I can understand the increased cast time on the wand of magic missile, a stronger spell, to be longer, I would expect the Ray of Frost one to be on par with the Acid Splash and Finger of Fire ones, that are already much faster.