No one is forcing you to click on the mushrooms, adventurer's packs, alchemy stations, etc. that litter the various dungeons. The collectible system is entirely optional.
I for one am glad for the healing elixirs that can be had from various collectible traders. It made being a barbarian so much easier. The randomization of the augment turn-in was a really bad idea, seeing as it was the most expensive thing that any collectible trader has, but it's not nearly as bad as you make it out to be. Good hyperbole, btw.