Exactly. If there is a temperature difference. To experience it, stand behind a window on a col..d black night and you will feel your body heat radiating out to the night sky.
I think the factor for surface color is called emissivity. I am going to check on that.