The snag is, in my experience that is not a minigame, that’s a game. If you change the source of a wizards power to being out in the world, the DM has to keep on providing a supply of that spell fuel, and the wizard has to keep prospecting. What is more, the DM will define what spells the player can cast, not just in general, but today, on the basis of what materials he makes available.
Another consideration is that it removes the “per day” pacing of magic user power. They can save it over weeks for a boss, they can use it all at once. If someone starts hoarding a certain spell component do you make it less frequent? That’s a negative feedback loop, as he could start hoarding the spell even more, because he wants to use it but is insuring he always has enough to do so.
So you give him enough fireball fuel for a week, then he blows it by chucking 21 fireballs in a single combat. Is that wrong? It’s a possible consequence given a resource-based magic system. Of course, depending on the game you run, you can just go hard luck and not give them any further resources of that kind for the next few days, but depending on the structure of your game that can cause friction.
One way to resolve that complexity is to just randomly generate sources of spells in a way that is fairly consistent and spread over all the spell types, but this means randomly generating monsters from certain types consistently through play. Alternatively you could give the player a constant requirement of money, by allowing them to buy such components on an open market, but this does risk situations where groups start chucking gold into the wizard in order to defeat a foe, meaning that treasure splitting becomes more complex, and player power more wildly varying.
Now this could be really cool, but it is a game of it’s own, with it’s own characteristics. In contrast vancian spell casting has a lighter footprint, because magic power is rationed out automatically by day. This still gives alpha-strike potential, and hoarding, but in a more limited fashion. There is also a relationship between spell casting and money, but again it is restrained (in almost all of the versions) by downtime requirements. You could replicate some of these balancing factors by requiring characters to process their spell components before use, and having the processed spell components go off, or by limiting the availability of components in the local area to a slowly growing stock with supply-demand elements but that adds it’s own complexities.