Good for you for working towards paying off your mortgage, or at least paying it down. The only caveat I have is that you make reference to student loans. If you are still paying them off, that should be the priority since their interest rates are so much higher.

As to all the commenters advising to essentially arbitrage between the mortgage and the stock market - I would point out that given the volatility of the latter, it is important to diversify. And real estate (though also volatile) is a valid choice for some of that diversification.

And lastly, there is value in having peace of mind.