After sundown, things have just begun to cool off - hours later at sun-up will be about the coolest. Here in Arizona, where the saguaros are blooming and we're already seeing record heat around 110F, morning is really the only time to ride. As others have said, there will be fewer drunks on the road and there will be more light.
Mornings are definitely best in the desert. The large temperature swings means it is usually pretty cool just before and at sunrise (the coolest time of the day).