Smart quotes (also known as curly quotes) are fancy characters which make text look better compared to the straight apostrophe (') and straight quote or inches character (").
Either turn the “Smart Quotes” preference off in the application (In Word it’s under Tools → AutoCorrect) or make sure these mappings are available in the font:
$201C " Left Double Quotation Mark
$201D " Right Double Quotation Mark
$2018 ’ Left Single Quotation Mark
$2019 ’ Right Single Quotation Mark
Word also automatically changes 3 periods to an ellipsis ($2026). You can fix this two ways: add an ellipsis to the font you’re using, or you can turn off this “feature”. It’s under Tools → AutoCorrect.