“What a world, where Hillary Clinton isn’t in jail but Kim Davis is,” Huckabee said after the clerk was taken into federal custody on Sept. 3.

U.S. District Court Judge David Bunning held Davis, who is the Rowan County court clerk, in contempt for refusing to issue same-sex marriage licenses after Kentucky Gov. Steve Beshear issued a mandate to issue the licenses. Davis is represented by Liberty Counsel in the case. Mat Saver, founder and chairman of the law group known for addressing religious discrimination issues, will join Huckabee at the rally.

“This is a reckless, appalling, out-of-control decision that undermines the Constitution of the United States and our fundamental right to religious liberty. Having Kim Davis in federal custody removes all doubt of the criminalization of Christianity in our country. We must defend religious liberty and never surrender to judicial tyranny,” Huckabee said.

Huckabee isn’t the only presidential candidate to support Davis. Kentucky Sen. Rand Paul said putting the clerk in jail wasn’t a good idea. “I think it’s absurd to put someone in jail for exercising their religious liberty,” Paul said in an interview after the decision. “I think it’s a real mistake and even those on the other side of the issue, I think it sets their movement back.” Paul said the jailing will harden people’s opinions on same-sex marriage and will encourage local courts across the country to “opt out” of issuing licenses altogether.

Other candidates, including Louisiana Gov. Bobby Jindal and Wisconsin Gov. Scott Walker, said the federal judge went too far. “We are seeing government today discriminate against whether it’s clerks, florists, musicians or others. I think that’s wrong. I think you should be able to keep your job and follow your conscience,” Jindal said.

Walker said religious freedom is a fundamental right. “I read that the Constitution is very clear that people have freedom of religion — you have the freedom to practice religious beliefs out there, it’s a fundamental right,” the Wisconsin governor said.

Huckabee said the sticking point in the case is that Davis actually didn’t break any laws. The United States Supreme Court did not make a law when justices issued their same-sex marriage ruling earlier this summer; they simply ruled on the constitutionality of bans states had in place, according to Huckabee. Only Congress can actually make a law, he said. The Kentucky legislature, likewise, has not created a law allowing for same-sex marriage.

Huckabee said that the U.S. Supreme Court had no legal precedent when it ruled on the case, and that their ruling is not backed up by the Constitution. The presidential candidate tweeted that if the refusal to implement federal law should be punished with jail time, there are others across the country who should face the same fate.

“#KimDavis in jail for not issuing same-sex wedding licenses. Why aren’t San Fran officials in jail for ignoring immigration law?” Huckabee said in the tweet.