HM Revenue & Customs (HMRC) has been criticised by the National Audit Office (NAO) for failing to deal with its huge backlog of more than 18 million unresolved income tax cases.
The NAO said about half the cases might involve underpayment or overpayment of tax with as much as £4.4bn at stake, the BBC reports.
The Revenue acknowledged it had "a lot of work to do".
"Just like every government department we are going to have to do more with less and today's report will help us to focus on those areas of our business that need to improve," said an HMRC spokeswoman.
The accounts of the HMRC are scrutinised every year by the NAO, which looks at the accounts and spending of all government departments and some other public bodies.
"The department has not made enough progress in reducing the backlog of 18.2 million income tax cases where there is potentially overpaid or underpaid tax," said the comptroller and auditor general, Amyas Morse.
"It also needs to improve its collection rate for tax credits debt, which is substantially lower than that for tax debts," he added.
Staying invested could prove lucrative
Consider lasting powers of attorney
Less environment, more governance threatens to undermine firms' green credentials
Evidence your compliance