To go into non-private browsing mode (aka normal mode), on the upper right hand corner, press on the icon which looks like two squares on top of each other. Then you'll see your main browser shrink, but you'll see on the upper right hand corner the "private" button. If the "private" button has the word "private" in black on a white background, it means you're in private browsing mode. To turn it off, hit the "private" button again and a new browser window will appear. Click on the new browser window, and you'll be on non-private mode again.
Personally, I find it to be a pain in the arse. I'd rather just open up Chrome in non-private mode and leave Safari in private mode.