Pretty and handsome are not the same, but they're unisex.

Britney Spears is pretty. So is Brad Pitt. What? You thought pretty only applied to girls? And handsome only applied to men? Girls and men -- get it? The words are less a function of sex as of age. When you're young and attractive, you're pretty. When you're older and still attractive, you're handsome. Think about the Eagles' song "Hotel California": "She's got a lot of pretty, pretty boys that she calls friends." Those boys are pretty because they're young. They're not handsome yet: that takes some aging and some living. Sean Connery is handsome. So is Meryl Streep. You wouldn't call her pretty, right? But she's very attractive. She's a handsome woman.
Let's stop sexing our words. Dame Judy Dench is handsome, as was Cary Grant. Pretty denotes youth and beauty; handsome denotes certain distinguishing features that come from a life lived by a formerly pretty person. Pretty is fleeting; handsome endures. In Harry Chapin's song, "Taxi", his former girlfriend "is acting happy inside her handsome home": we know right away what kind of home she lives in, which is not pretty but older and established but still attractive, as maybe she is, all these years later.