Several tea party and conservative groups had complained for years that the IRS had requested onerous amounts of documentation, including donors lists, from organizations during the process of applying for tax-exempt status.
But watchdog groups who had called on the IRS to scrutinize further tax-exempt groups say the news came as a surprise.
"It had seemed if anything, that the IRS was very inactive in this arena," Lisa Gilbert, director of Public Citizen's Congress Watch, said today. "We raised alarm bells around that in the past."
Gilbert added that blowback from missteps by low-level employees should not prevent the IRS from cracking down on illegal political activity by these tax-exempt "social welfare" organizations.
"We're not supportive of any kind of biased methodology that looks at tax-exempt organizations. But we are very supportive of the IRS expanding their scrutiny of 501(c)4 organizations," she said.
Though Democrats had been on the front lines of the push to pressure the IRS to investigate further the political activities of these tax-exempt groups, they have now joined with Republicans and conservatives in condemning the IRS in the wake of a revelation that threatens to derail advocacy for tougher campaign finance regulations.
In statements today, both House Minority Leader Nancy Pelosi, D-Calif., and Senate Majority Leader Harry Reid, D-Nev., echoed Obama's comments from earlier in the day.
"Those who engaged in this behavior were wrong and must be held accountable for their actions," Pelosi said in a written statement. "Regardless of political affiliation or bias, there is no place for this type of activity by the IRS or its employees."
Reid called the reports of misconduct at the IRS a "terrible breach of the public's trust," but cautioned that no one should "jump to conclusions" before the IG report has been released.
"We should all rest assured as soon as we have the inspector general's report, the Senate will swiftly take appropriate action," Reid said on the Senate floor.
The IG report suggests that the practice might have begun after the landmark 2010 Citizen's United vs. Federal Election Commission Supreme Court case, which was viewed as expanding the ability for nonprofit organizations to become involved in political elections.
David Bossie, executive director of Citizen's United, the organization that brought the case against the FEC, said the new information should prompt a full congressional investigation.
"The IRS clearly saw the Citizens United case as a catalyst to go after conservative groups in the lead-up to President Obama's re-election," Bossie said in a statement.
"The Obama Administration's obsession of win-at-all-costs has brought Chicago style politics to the IRS. A full and complete congressional investigation must commence immediately and those who targeted fellow Americans for pure political purposes must be held accountable under the fullest extent of the law."
ABC News' Sunlen Miller contributed to this story.