George Soros, who called gold "the ultimate bubble," dumped almost his entire $800m stake in bullion in the first quarter.
Soros offloaded the stake well before a commodities slump blamed partly on reports he was liquidating his holdings, according to Reuters.
Gold bull John Paulson held his ground, but Soros was joined in the retreat by several other big names, including Eric Mindich and Paul Touradji, according to 13-F filings with the U.S. Securities and Exchange Commission.
Soros, who has been bullish on gold in the past several years, cut his holdings in the SPDR Gold Trust (GLD.P) to just $6.9m by the end of first quarter, compared with $655m in December.
He also liquidated a five million share stake in the iShares Gold Trust (IAU.P), the filings showed. His total holdings in gold-backed ETFs was $774m as of December.
Gold rose for a tenth consecutive quarter in the three months to March, hitting record highs above $1,400 an ounce, buoyed by political turmoil in the Middle East and North Africa and lingering worries about indebted European countries.
The gains accelerated in April, but peaked at the start of this month, reaching a record $1,575 an ounce on 2 May.
Prices have since fallen more than 5% amid the biggest commodities slump since late 2008, a move partly triggered by a Wall Street Journal report that Soros' $28bn fund was selling precious metals, and fears other big funds were also seeing a peak.
Paul Bruns and Elaine Parkes
3,000 left to transfer
Record numbers of people aged 90 plus
From 3 to 10 October