@leafis Generally yes, that is one factor to consider when selling. There are other factors (maybe your needs change, maybe you need more space because of a growing family, maybe you're downsizing, whatever) but a big penalty will impact the decision to sell.
If someone is thinking of selling, one of the first things to do is call your lender and ask for a current penalty quote. The exact figure can change but it will give a ballpark to work with.