New figures from the National Audit Office reveal that one in ten phone calls to the HMRC helpline are not picked up.
Over 43 million people called HMRC's tax advice helpline over the past year; however, more than 10% of these were not answered. The National Audit Office (NAO) data also shows that 14% of calls rang for more than ten minutes before they were picked up by an adviser.
What's more, the audit of calls did not include those tax-payers who got an engaged tone when they called the helpline.
Meg Hillier, chair of Parliament's Public Accounts Committee, told the Telegraph: "There are a lot of people who are self-employed, who do not have a raft of tax advisers and who rely on HMRC for help. Hanging on the phone during the day is difficult to do for a lot of people."
An HMRC spokesman said: "We have improved our customer service standards enormously over the last two years. Our phone call handling has got better, with the average response time falling from 12 minutes in 2015-16 to below five minutes for the past two years.
"Time in the automated telephony system is valuable because it ensures customers are directed to the right person to deal with their question, or get their question answered through the automated system."
However, the figures show that the number of people who can't get through has actually doubled in the past year - from one in 20 last year to one in ten this year.
HMRC has been accused of cutting back on staff and resources in order to encourage more tax-payers to submit their tax returns online as part of the move to Making Tax Digital.