Look closely at the keyboard here.  You'll notice several things missing:

1 - No Enter/Return button.  Jumping down to the next line and returning the carriage back to the beginning was done manually, via the Type Bar.  Even the first electric typewriters didn't include one.  When they finally did come up with one, it was a 'Return' button, because it returned the carriage back to the beginning of the line.

2 - No 1 button.  I never could figure out why it took so long to give us a number one button.  They didn't start appearing until the 60's.  Before that, the lower case 'L' was used... which would severely limit the font styles.  This is across all brands.

3 - The exclamation point (now situated over the 1 button).  Aside from being a later development in the English alphabet system, it wasn't considered professional to use one until the late 80's.

The exclamation point is said to have developed out of the Latin word "oi" (joy).  Maybe some monk writes out a sentence and then denotes an exclamation of 'joy' at the end of it.  Or maybe we don't really know where ! came from, so people found the closest thing they could from a Latin phrase and claimed some piece of history that they really have no way of knowing.  I personally think that happens a whole lot.

Anyway, we see it start making appearances in the late 1800's.  And even then, it was used sparingly.  None of this "LOL!!!!" crap.  In fact, popular figures in literature were particularly hostile to the mark.  Fitzgerald was known to have said, “Cut out all those exclamation marks. An exclamation mark is like laughing at your own jokes.”

So it's no surprise that the early typewriter manufacturers didn't include the exclamation point.  But that doesn't mean people didn't type them...

There were two versions.  In one, you could type the backslash (beneath the question mark), backspace, then a period.  In the more popular version, you'd use the apostrophe (above the 8), backspace, then a period.

-OR- if you know your way around the keyboard, you can press and hold the spacebar, then press and release the period, then press and release the shift 8, then let go of the space bar.  The carriage moves AFTER you release a key, so holding down a spacebar allows you to repeatedly type letters without moving forward.  Incidentally, that's also how you would make bold letters.