Tranny questions?

Rebuilt 904 last year, put about 4600 miles on car this summer. My question is, using a cable kickdown set-up that is adjusted correctly. The car shifts fine when in drive, shift points are where they should be, BUT. When I shift manually which I do a lot because I like to bring the rpm up before shifting. When I bring rpm up from 1st to 2nd (around 4 to 4500rpm) when I shift to second it tends to shift slowly, I would like to have the firm shift. When I shift from 1st to 2nd at lower rpm it seems to shift right away. From 2nd to 3rd seems fine, shifts right away. Would that be a valve body issue or torque converter issue? Over all the car runs fine. I'm just trying to tweek a few things on it over this winter. thanks