It seems like the game really isn't glitching out on you, rather, you two have a bad connection. My friend and I never experienced any of these problems, maybe try having soeone else host it?
Stick together too, the game might have trouble rendering when you two are far apart. Hope you are able to get it!
688 days left to go!