valve train noise?

I recently bolted a set of 360 j heads on to my 318 and now I have what sounds like slightly loose pushrods. When the engine is idling or cold it is not present. Iwas running the 302 swirl ports prior to the change and did not have any valve train noise. I checked pushrod length and i have 20-30 thousandths preload on the lifters. I tried putting in pushrods that were 30 thousandths longer and I get the same results. I am still using the stock non-adjustable rockers. I am going to be getting a set of stock 273 adjustable rockers to see if this will help. Am i heading down the right path to solve this? Will the adjustable rockers hold up well in a slightly modified 318 that does not go past 6000 rpm. I also converted to a 90 degree oil filter adapter to fit my spitfire headers in. I pulled the valve covers and checked for oiling issues but it seems to be getting good oil supply up top, oil pressure is good too.It does not seem to be hurting the performance much but it sure does worry me that I may be slowly killing my cam or worse. Any help with this would be greatly appreciated. (the j-heads are for future plans with a 360 swap)