I somehow must have written over the default grey cube with one of my own models. Now when I choose file..new I get my yellow modified thing every time.

How can I get the basic grey cube back as a starter item?

I found "load factory settings" which seems to have reset it.

However, if you have changed other settings to your personal preference, using "restore factory settings" will change those back, too. In that case, just make certain the 3D cursor is at the origin (or whatever other point you want it at), and choose "Add > mesh > cube".

ns