I created a new font from the characters of a 1925 Remington Portable typewriter. I have encountered a buggy behavior when using within MS Office apps. It reverts to Times New Roman for the double and single quote characters. In non-Microsoft applications, this does not happen. I tried to copy/paste from the Windows Character Map tool, but it doesn’t fix anything. Anyone else with this problem? Solution? Thanks.
Does your font contain the required double and single quote glyphs in the General Punctuation character set?
Microsoft Word comes with an option (smart quotes, which is on by default) to automatically replace straight quotes ( " and ’ ) with left and right quotes ( “ ” and ‘ ’ ). I believe Photoshop also has such feature, and probably others have it too.
It is best to add those quotation marks to your font. In FontCreator select Insert from the main menu, and then click Insert Glyphs. From there on the first tab (By Name), enter:
quoteleft
quoteright
quotedblleft
quotedblright
This will add the missing glyphs along with their character code-points. All that is left, is to provide proper outlines. 
Thanks for that. But, I don’t understand how that gets mapped to the keyboard so that I can type it directly in an MS Office app.
And, yes, both the single and double quote characters are there. As I said, they type correctly in non-MS Office apps.
They do not need to be mapped to the keyboard. MS Word and other applications use Autocorrect to swap the double straight quote " to “open double quote before a quotation and to close double quote” after it; and the single straight quote (apostrophe) to ‘open single quote before a quotation and to close single quote’ after it.
If Autocorrect is disabled, no change is made; you just get what you type.