I’m sure more than likely the issue is dirt because the previous owner had one of those clear plastic fuel filters in line and it did have a very small amount of sediment on the bottom.
The stock fuel system would have a filter/sock at the tank pickup, a canister filter after the fuel pump, and then another filter under the Carter AVS fuel inlet. I'm thinking the only good filter in your system is the one that you changed.