Converting from Manual VB to stock?

Do you have an aftermarket shifter?
In any case, you have to synchronize the trans to the shifter. This is usually done in neutral, But I've had better luck using Drive or second.
But, the little shift lever that you install after the trans is buttoned up, comes in several shapes and sizes, and has to be matched to the shift indicator. If it is not the right length for your shifter, then it will not sync up correctly.
Same goes for the TV lever, it has to be the right length for your system, and then tuned for your driving requirements.
I get the idea that this what you have been trying, so Ima thinking we can move past this.

Ima thinking your governor is cranky.
But;
I would start by proving the trans actually works.
First;get the car up on safety stands with the rear wheels free to spin. Then
disconnect the cables from the levers. But before you do, figure out where second gear is. I think it's second from the rear, but I'm having a brainstall at the moment.
With the TV disconnected,the engine will have to be revved up some, not much, say 1200 to 1500 rpm. This will go better with a helper in the cab. Get your helper to increase the rpm up to the test zone and hold it steady there. You are gonna be underneath shifting from first to second to drive to neutral, then STOP! The wheels are gonna be spinning so make sure there is no gravel caught in the treads, that can fly out and hurt you. The speed-O needs to get up to about 30mph,exact number not important yet. Then get your helper to let off the gas pedal and gently brake the tires to a stop, and shut off the engine..
When it works right you will feel each shift. Without the TV working, the shifts should come early and be soft.Reach up and push the TV lever all the way to the rear; you should feel spring resistance, and it should spring back at least 1/8 revolution, or more, by itself.
If you didn't feel the shifts, repeat the test while dragging the brakes. The shifts should now shudder the rear end. Still no? increase the rpm to 1800 and repeat.
If still no, then it's time to test the governor pressure.

And the reason is that when installing a manual VB the installer will occasionally block the governor circuit. Or the governor itself will be modified to not shift until a very high rpm. And governors are known to just fly apart.

The thing is this;I'm confused by your saying that second works. Normally, with a stock VB, it is impossible to start in any other gear except first. This leads me to believe that fancy new VB is not working right, or not working right on your trans.
While the wheels are up, figure out your rear gear ratio; just maybe it's not what you think, or were told. Maybe the trans is starting in first, but the rear is a stinking 2.20 ratio,lol. That will feel like second alrightee.
This would then point to a non-functioning governor, as it is governor pressure which commands the shifts; with the TV delaying them with input from the throttle opening.