A little Difference
Here we see that with 2 1/2 inches of wood on the outside of a building and one inch of insulation on the inside, the variance in indoor/outdoor temperatures is still negligible.
Image from California Energy Commission - PASSIVE SOLAR HANDBOOK story