In the above example, to me, “oh” seems to suggest one should pronounce “o” as a short vowel, whereas “o”, seems to suggest one should pronounce “o” as a long vowel. In other words, I would expect it to be spelled as:
I would expect “oh” to be used in this example, meaning to express “oops”:
… or used an an expression of excited amazement, when watching beautiful fireworks for instance, like this:
So why is “Oh my!” spelled this way?
There is no difference between the pronunciation of oh and o; in both the cases, the pronunciation is /oʊ/.
Oh is used to express surprise, anger, disappointment, or joy; it is also used when reacting to something just said.
Oh, shut up.
O is also an archaic spelling of oh; it was also used before a name in direct address.
Give peace in our time, O Lord.