An appeal is certain, but should U.S. District Court Judge Rosemary Collyer’s ruling be upheld, it could spark the economic “death spiral” Republicans have predicted and Democrats feared would doom the 2010 Affordable Care Act.

But the ruling has implications far beyond Obamacare, signaling that federal courts may begin to play a more active role in reeling in executive powers that many legal experts say have grown far beyond what the country’s founders intended.

Judge Collyer, presiding in Washington, said the administration violated the Constitution when it made “cost-sharing” payments to Obamacare insurers, over the objections of Congress, which had zeroed out the funding.

“Authorization and appropriation by Congress are nonnegotiable prerequisites to government spending,” she wrote.

Judge Collyer said it was illegal for the administration to continue making the payments. But she stayed her own decision to give Mr. Obama a chance to appeal her ruling.

The White House was stunned by the ruling and railed against the House for bringing the fight to the courts in the first place.

“This suit represents the first time in our nation’s history that Congress has been permitted to sue the executive branch over a disagreement about how to interpret a statute,” White House press secretary Josh Earnest said.

The cost-sharing program was written into Obamacare to make it more attractive for poor people without insurance to buy plans on the new health exchanges. In addition to tax subsidies, those with incomes just above the poverty line were supposed to have some of their costs paid directly by the government to insurers. The Affordable Care Act authorized the payments, but Congress and the Obama administration have feuded over whether Capitol Hill needed to take the next step and appropriate the billions of dollars each year. Initially the administration seemed to think it did need an appropriation and requested the money in its budget. But after Congress refused, Mr. Obama changed tune and said he felt he could spend the money even without a new OK.

In court the administration argued that it wouldn’t have made sense for Congress to approve the program but not come up with the money. Judge Collyer rejected that, saying Congress authorizes programs all the time but never finds the money to carry them out. She spanked the secretaries of the Treasury and Health and Human Services departments for trying to spend the money anyway.

“Such an appropriation cannot be inferred,” she wrote. “None of the secretaries’ extra-textual arguments — whether based on economics, ‘unintended’ results, or legislative history — is persuasive.”

Millions of Obamacare customers with incomes between 100 percent and 250 percent of poverty rely on the payments, and health plans are required to reduce their out-of-pocket costs whether they’re reimbursed or not, so they’d likely increase rates to balance their ledgers.

House Republicans’ decision to sue Mr. Obama in 2014 was unusual, though then-Speaker John A. Boehner said he had no choice after the executive branch doled out cost-sharing payments and twice delayed the part of Obamacare that requires large employers to provide health coverage to employees. Judge Collyer shocked Democrats in September by saying House Republicans had legal standing to pursue their central claim: that the administration injured Congress as an institution by usurping its power.

House lawmakers said Thursday’s ruling on the merits offered further vindication.

“The court ruled that the administration overreached by spending taxpayer money without approval from the people’s representatives. Here the executive branch is being held accountable to ‘We the People,’ and that’s why this decision is very good news,” House Speaker Paul D. Ryan, Wisconsin Republican, said.

Analysts have said the constitutional and political considerations behind the case may outweigh the economic ones. Without the reimbursements, insurers may hike the price of their benchmark Obamacare plans. But those higher costs could end up being covered by the federal government anyway through the tax subsidies paid directly to Obamacare customers to help them buy insurance on the exchanges.

“Federal payments to insurers for cost-sharing subsidies would end, but would essentially be replaced by larger federal premium subsidy payments to insurers,” the Center on Budget and Policy Priorities said in a September study.

For now, the ruling offers a timely jolt to congressional Republicans who want to repeal and replace Mr. Obama’s signature domestic achievement in 2017, and who say its existence is proof the president has stepped out of bounds in using his powers.

“Here’s what the court just said on Obamacare: America still has three branches of government, and the president cannot rewrite the law — even if it is his namesake,” Sen. Ben Sasse, Nebraska Republican, said.

For his part, Mr. Obama has been hoping to shore up his health legacy — roughly 20 million have gained coverage under his reforms — instead of fending off repeated legal challenges to reforms. Mr. Obama had urged his opponents to move beyond legal challenges to Obamacare last year, when the Supreme Court turned back a second major challenge that would have gutted his reforms. Now he will have to fend off one more challenge.

“It’s unfortunate that Republicans have resorted to a taxpayer-funded lawsuit to refight a political fight that they keep losing,” Mr. Earnest. “They’ve been losing this fight for six years. And they’ll lose it again.”

⦁ Dave Boyer contributed to this report.