So it seems kind of confusing starting local CO-OP in this game for some reason, so here's a short guide for anyone else who struggled.

First you need to be in a game world, like this



Once you're in your world and have both controllers turned on (or keyboard + controller, yada yada), you need to press a button to bring up the co-op window.

Xbox Controller: View Button See here, number 3[support.xbox.com]]
Keyboard: = key
PS4 Controller: Share (I think)

The co-op section looks like this



Once you have that up on the controller you want player 2 to use, press the relevant button again to bring up a second popup



With this popup, click the right arrow to go to the second slot, then press A (or X).



Then press A (or X) again.



Another little sackboy will appear out of nowhere.

Video guide (kind of sucks sorry)

As of 28 Oct 2022, it is not possible to have two players from the same PC play online. Once you connect online, Player 2 will disconnect.

COOP also appears to be limited to 60 FPS.