Simon Cowell
is no stranger to being in the spotlight, but the “X Factor” judge wants nothing to do with the bad publicity following him around after news broke about his alleged affair with his friend’s wife.

TMZ reports that Cowell is urging Lauren Silverman, the woman with whom he is expecting his first child, to quickly come to a divorce settlement with estranged husband Andrew Silverman to avoid the negative press.

An insider says the couple has a prenuptial agreement that will likely mean little settlement money for the mom-to-be, so he is urging her to speed up the process.

The source adds that the primary issue that could protract the divorce proceedings is the custody battle over the Silvermans’ 7-year-old son, also named Andrew,.

Though the expectant mother reportedly claims her husband knew their marriage was over, he has denied any knowledge of the affair.

Comment on this