Only the lower pin is restrained from moving up & down by adjustment of the spacers and bearings with shims. The upper pin is not restrained at all, so whatever up/down movement there is in the lower pin the upper one will have exactly the same. That’s spacer “H” on your earlier diagram. If the pin is out already it should almost fall out. If it won’t move run a couple of circular rings of weld around the ID, that should shrink it enough to let it fall out.