Junk in my new motor...

Grab the pickup close to the area where it enters the block with a smallish pipe wrench or sharp toothed vice grips, turn it counter clockwise to remove it.

If its really stubborn - Try tapping the wrench with a hammer to set up some vibrations, pipe thread responds pretty well to that kind of thing.

If its really stuck - heat up the pipe going into the block with a propane torch and spray it with penetrating oil after it gets hot. Smoke and fire are assured. But what you are doing is causing the oil to penetrate into the thread by capillary action accentuated by heating and cooling. It should loosen up after a couple of these type treatments. Allow it to cool down before trying to remove it.

By grabbing it close to cast iron you wont be putting your effort into twisting the pickup itself and will be putting the energy into the threaded portion.

I strongly recommend getting a replacement pickup. Its near impossible to get all the crap cleaned out of one of those things. A new pickup costs about 20$.

Let us know how things go.