Cake batter is an emulsion, which means it's a blend of ingredients—wet and dry, fat and liquid—that ordinarily would not combine. And one of the reasons an emulsion might fail to come together is if its components are too cold.
Which makes sense: solid chunks of butter won't blend with anything. In batter, those chunks of butter will translate into a crumbly-textured cake that won't rise well. When it comes to a flaky biscuit or pie crust, big lumps of butter are exactly what you do want, but for a cake, the opposite is true.
Therefore, when you're baking a cake, let your butter, eggs, milk, and any other refrigerated ingredients come to room temperature before you start mixing. Half an hour on the counter is better than nothing at all, but you really should shoot for at least an hour.