When starting-up/turning-on your Joy-Con was the joystick being pressed in anyway? You can’t move the joysticks when staring the Switch or the Joy-Cons. This changes the controllers neutral point. Turn the Joy-Con off and on again, this should resolve issues with the joysticks moving characters in game without you pressing anything.