First, even a small rate cut can pay off quickly. That's because you can easily find mortgage companies willing to waive routine refinancing charges such as application, appraisal and legal fees (which can add up to $1,500 to $3,000). Of course, in exchange for low or no up-front costs, you'll have to be willing to accept a rate that's somewhat higher than the prevailing rock bottom.

Second, if you are planning to stay in your home for at least three to five years, it may make sense to pay "points" (a point equals 1% of the loan amount) and closing costs to get the lowest available rate.

And third, you can avoid laying out cash and still get a low rate by adding the points and closing costs to your new mortgage. Does that mean shouldering a lot of extra debt? Not necessarily. If you've had your current mortgage for at least three years, you've probably reduced your balance by several thousand dollars. So you may be able to tack your closing costs onto your new loan and still end up with a mortgage that's smaller than your original one -- plus, of course, a lower rate and lower monthly payment.