The Times reports Irving Picard, the trustee in charge of recouping the money, has so far got back $11.5bn of the $17.5bn lost by investors. Madoff is currently service 150 years in jail for securities fraud.
The report said trustees are now seeking to recover a further $255m from the Madoff family, who are accused os using the Madoff securities firm as a "family piggy bank".
Picard wants $44.8m from Madoff's wife Ruth, as well as $27.5m from Stephanie Mack, the widow of Mark Madoff, one of Bernie Madoff's sons. He committed suicide two years ago.