Criminals used data that was stolen to get access to prior tax returns for over 100,000 people through an application on the website of the Internal Revenue Service, said the agency on Tuesday.

Using social security numbers, birth dates, street addresses and other information that was obtained elsewhere, the hackers completed a authentication process and then requested the prior tax returns and others types of filings, said the IRS.

Information taken off those forms was used to file returns that were fraudulent, said the IRS and the agency sent close to $50 million in tax refunds before detecting the scheme.

The agency opened an internal investigation into this breach and temporarily shut down its Get Transcript app, which was the one used to gain access to all the personal data.

Past tax returns sometimes are needed for applying for loans or a mortgage and taxpayers are able to request those records through the mail.

Over 200,000 attempts to see the past returns using the stolen information were made between February and mid-May, and close to half, turned out successful.

It is not clear whether the cyberattackers were within the U.S. or not.

Dealing with tax claims that are fraudulent has been a big challenge for the IRS as there has been a big growth in online crime that has become quite sophisticated of late. The tax revenue service said it paid over $5.8 billion in claims that were false in 2013.

Eighty percent of cases of identity theft dealt with refund fraud related to worldwide organized crime.

The IRS said attackers exploited data such as password and email addresses gleaned from the banks’ other branches to answer the most basic of authentication question such birth data and family member names.

After recent data breaches at Home Depot and Anthem, security experts noted that personal data from users is available to hackers, who can purchase it from criminal websites.