It looks like Tom Cruise‘s past is coming back to haunt him thanks to one of his former co-stars!

Curtis Armstrong, who appeared alongside the then 19-year-old actor in 1983’s Risky Business, writes in his autobiography, Revenge of the Nerd, that the A-lister was constantly hooking up with female fans while shooting the film.

EXCLUSIVE: Leah Remini Wants to Go Head-to-Head With Tom Cruise Over Scientology

“Late one night, I found three or four girls — late teens, I suspect — lined up in the hall outside of Tom’s room,” Curtis recalls. “Tom’s door opened and another girl came out, adjusting her hair and taking off down the hall, while the first girl in line slipped into Tom’s room.”

tom cruise getty images

Tom and Rebecca De Mornay in Risky Business.

Needless to say, the 54-year-old isn’t happy that tales from his early years in the biz are coming to light. “[Tom] is embarrassed about Curtis’ book exposing all these secrets,” an insider tells Life & Style. “He’s worked very hard to be portrayed a certain way in Hollywood, and this goes against everything he now stands for.”

And even though the book “actually makes Tom look human,” the insider notes, it’s making the father-of-three upset because he’s “all about controlling his image.”

The Oscar nominee already struggles with criticism over his relationship with daughter Suri, whose mom is his ex-wife Katie Holmes. A different source previously told Life & Style, “It’s been more than three years since Tom has seen Suri. And he hasn’t celebrated Thanksgiving or Christmas with her [in person] since she was six… He doesn’t talk about those days with Suri much anymore.”

But, an insider told In Touch, “Tom wants to marry for a fourth time and have Suri in his life again because family has become a key priority for him since his mother’s death [in February].”

For more on Tom Cruise’s mysterious life, pick up the latest issue of Life & Style, on newsstands now.