So when I retire, and my oldest is out of the house to college, we are thinking we could rent that particular part (at a very reasonable rate to people we know). It is part of the house, so I can’t sell it separately.

If you don’t need that space, then you might as well sell it and let another family make use of it instead.

Yours is not a unique situation; a lot of older people downsize when their kids move out, and they have a lot of extra rooms and space they no longer need. Its the right decision anyway, as you’re now free to be more mobile, and get rid of all the years of accumulated junk.