I’m Retired. Should I Pay Off My Mortgage?

It’s 5 pm on a Tuesday, and you tune in to The Ramsey Show as you sit in gridlocked traffic. Dave Ramsey is going on about the best ways to pay down debt and why it’s imperative to be debt-free. You have two things working in your favor: (1) You have the money to do just that. (2) You only have to commute in rush-hour traffic for a few weeks longer, as you will retire at the end of the month.

The next day you start to do some research on paying off your mortgage, and you come across Ric Edelman, the founder of one of the largest personal finance companies in the country. His advice is just the opposite of Ramsey’s: You should stretch out a big mortgage for as long as possible, he maintains.

I’m guessing this leaves you a bit confused. The truth is that personal finance is just that: personal. The right answer for you won’t come from someone speaking to a million people and giving one answer.

If you have the money necessary to pay off your mortgage and you are retired, or nearly retired, this article will allow you to place yourself in one of three groups, to get closer to the right answer for you.

1. You have the money in cash because you are scared of the market

Should you pay your mortgage off? Yes. In this case you should pay it off.

why? There is a term we use in this profession: arbitration. Applied in this context, you have negative arbitrage. The bank is paying you 0.25% on your savings account (if you’re lucky) and charging you 3.75% on your mortgage. So, you are losing 3.5% every year you hang on to that loan. This is oversimplifying, of course, but you get the idea.

What’s the downside? First and foremost, you are losing liquidity. When you pay off a mortgage, you are essentially putting money into a piggy bank that you can’t get back out unless you sell the home or tap the equity. Second is the tax consideration. Paying off your mortgage may mean that you fall below the standard deduction threshold because you don’t have the mortgage interest to write off. This could raise your effective tax rate, but likely not significantly. Last, but especially relevant today, holding a loan is an inflation hedge. Because your principal and interest payment stay flat in a fixed-rate loan, your housing expense is likely to inflate much more slowly than CPI-W.

2. You have the money in a brokerage/taxable account

Should you pay your mortgage off? Probably not.

why? Same idea as above, but reversed. You now (historically) have positive arbitration. From 1991 to 2020, the S&P 500 returned 10.72% on average, annually. Every investment exam, class and disclosure will tell you that past performance is not indicative of future results. However, in that example, you would have lost (10.72% – 3.5%) 7.22% per year. There is also a tax consideration if the investment you hold has an unrealized gain. Depending on your taxable income and the size of the gain, you are likely to pay 15% or more of that gain to the Treasury before you pay off that loan.

What’s the downside? Stocks can always swing the other way. Historically, stocks go up about three quarters of the time. In order for you to make money by earning more than the interest rate on the loan, you have to be in that 75%. Picture a scenario, like now, when you were planning on making 10% in the brokerage account and paying 3.5% interest. Instead, you lost 20% in your brokerage account and paid 3.5% interest. You would have been better off paying off the loan before the drop. Unfortunately, no one has a crystal ball. My feeling is that you have to bet the odds that the market usually goes up and usually goes up by more than the current mortgage rates.

3. You have the money in a retirement account

Should you pay your mortgage off? no. You shouldn’t pay it off in this case.

why? I get this question all the time, but no one has ever asked me after actually cashing out a retirement account to pay off their mortgage. My guess is that the accompanying tax bill affirmed that it was a bad decision. In Scenario 2, it’s mostly an investment decision with a tax consideration. This answer is mostly tax based. When you pull funds from a pre-tax retirement account, those amounts are included in your taxable income and taxed at ordinary income rates. Therefore, if you take a large withdrawal, your bracket will jump, and you will see a significantly smaller amount come into your bank account before you pay off your loan.

What’s the downside? cost. There is comfort in living debt-free in retirement. Having a lower housing expense provides you more flexibility in your discretionary spending. But, in this case, is it worth the cost?

Neither Dave Ramsey nor Ric Edelman is wrong. They just give different reasons for their advice. Ramsey mostly uses behavioral reasoning. Essentially, he believes that people are not going to use discretionary income beyond their 30-year mortgage payment to invest, but rather to buy things they don’t need. Edelman’s reasoning is purely mathematical. He does not hypothesize about what people will do with excess income but points out that if you can earn more in an investment account than what you pay in mortgage interest, you come out on top.

The challenge for all of these talking heads is that they don’t know who they’re speaking to. Everyone has a money script. If your parents lived through the Depression and drilled lessons into you about the evils of borrowing money, you probably don’t care about the math behind my reasoning.

Here’s the good news: I’ve yet to find someone who regrets not having a mortgage in retirement.

Wealth Manager, Campbell Wealth Management

Evan Beach is a Certified Financial Planner™ professional and an Accredited Wealth Management Adviser. His knowledge is concentrated on the issues that arise in retirement and how to plan for them. Beach teaches retirement planning courses at several local universities and continuing education courses to CPAs. He has been quoted in and published by Yahoo Finance, CNBC, Credit.com, Fox Business, Bloomberg, and US News and World Report, among others.

Leave a Reply

Your email address will not be published.