Think layers, as if you were you going running on a cold wet day. It's much colder on the water, and windier, so wear a base layer, an insulating layer, and an outer layer. For the former, synthetic performance fabrics like Under Armor (or own-brand equivalents) are good, and certainly you should avoid cotton, because it does not wick moisture away. The key is to have clothing that will keep you warm even when wet and doesn't get in your way when you are rowing. Spandex leggings are superior to loose shorts, which can get caught in the slides under the moving seats. Loose tops can get caught in the oar handles, so avoid bulky jackets or sweatshirts.
Shameless plug: The Oakton boat jacket is an excellent outer layer: it blocks the wind but breathes, helping you retain warmth while not getting too hot. On sale now here, along with a bunch of other excellent crew clothing.
Hats, Sunglasses, and Sunscreen
A hat is obvious: you lose a lot of body heat through your head. And sunglasses and sunscreen would be natural on a hot day. But don't assume that because the temps are low you won't need them: there can be a lot of glare and reflected UV coming off the water even on cold days.
Hands and feet
You can wear gloves off the boat, but not on it: you need the tactile feel of the oar in your hands. (Besides, blisters and calluses are character forming.) Synthetic or wool socks help ensure that feet stay warm while wet. You should have second pair on land. Likewise, a pair of running shoes so that you have something to wear off the boat (when you are in the shell, you'll be wearing the shoes that it comes with).
And of course
Always bring a change of clothing. It happens: people sometimes end up in the water. Or it rains and rains and rains. You'll be glad to have dry clothes to change into when you get back on land.