Banks paid out £81.2m last month to compensate businesses mis-sold complex interest rate hedging products.
The Financial Conduct Authority (FCA) said the speed of banks' progress in reviews and compensation was increasing.
It wrote the four bank chief executives last month to reiterate that redress should be delivered quickly.
Clive Adamson, director of supervision at the FCA said: "I welcome the fact that these figures show the pace of the banks’ reviews continuing to increase and more businesses and customers are starting to receive compensation payments, but we will keep the pressure on to ensure they continue to move as quickly as they can.
"The banks’ responses [to the letter] have been positive, with three of the four major banks saying to us they now expect to complete all initial redress determinations by May 2014.
"I am also pleased that eight out of the nine banks in the review have now agreed to split payments for initial redress and consequential loss. This change should simplify and speed up the process for paying basic redress to customers."
Claim from SocGen's global markets division
Third annual Hampton-Alexander review
European Commission yields to pressure
Numbers in Adviserland
Retirement sector trends