I’m unhappy when I hear former Presidents and other ex-officials addressed as “Mister." Doesn't this belittle their importance? I was taught to address someone with the highest title or position they've reached in their career.
When addressing a former President of the United States in a formal setting, the correct form is “Mr. LastName.” (“President LastName” or “Mr. President” are terms reserved for the current head of state.) This is true for other ex-officials, as well.
When talking about the person to a third party, on the other hand, it’s appropriate to say, “former President LastName.” This holds for introductions, as well: A current state governor is introduced as “Governor Tom Smith,” while you’d introduce an ex-governor as “former Governor Jim Bell.”
Now, let’s look a little closer. In an informal setting (such as a private lunch), it’s acceptable to use the title the ex-official held. Here, you could refer to former President Jimmy Carter as either “President Carter” or “Mr. Carter.”
Finally, if the person you’re lunching with held more than one previous position—say, judge and ambassador—you’ll want to know which title he or she prefers.