They aren't less toxic. The contribution from mining and the PM2.5 which results are on-par if not in excess of what coal mining and burning creates. Windmills are even worse, mostly due to the fiberglass dust that gets generated by manufacturing.
PM2.5 is far more dangerous than any other atmospheric emissions allowed, but of course we ignore it because the mining is 'exportable' to places where the general public doesn't see or feel it. Yay for 3rd worlds.