Your home is far from air-tight. There are many areas where the air comes in and goes out, and this will affect the efficiency of your HVAC system. The greatest source of energy loss in most homes is around windows. Even if you have double-pane windows, there is usually plenty of heat exchange around them. One common issue is when there are cracks and holes in the frame of the window. If you want to spot them, using thermal imaging, or if you have to go more low-tech, having a friend with a flashlight shining while you spot areas of concern on the inside.
The most common issue that you will run into is wood rot, but this can be fixed.
It is important to ensure all the rot is removed before you attempt any kind of repair since it will just start back up again otherwise.
Any repairs that you make should be water-proof to ensure no other issues get started.