New IRS data suggests otherwise.

According to a Fiscal Policy Institute analysis of IRS Statistics of Income data, the states with the most progressive tax systems have the highest concentration of millionaires per filer in the country. The number of millionaires within those states (California, Connecticut, Massachusetts, New Jersey, New York, and Washington, D.C) has also grown since 2010.

New York’s millionaire share rose again in 2023, the most recent year of data available, even after the state raised its rate on incomes over $1 million in 2021. Florida saw a notable decline in its millionaire share that same year.

“We’re just really not seeing any declines in the millionaire population in New York State,” Emily Eisner, executive director of the Fiscal Policy Institute, told Fortune. She said millionaire households “simply just aren’t that sensitive to the tax rates in the state” because, for them, “the taxes aren’t really high enough for them to be sensitive to them.”

Where it does fall, the tax burden lands hardest on families earning between $100,000 and $500,000 a year. “It’s tough living in New York City,” Eisner said. “You’re making a lot of money relative to the population of the United States, but in New York City, you don’t feel rich at all. You still feel squeezed.”

In 2016, New York state’s Department of Taxation and Finance counted 57,126 million-dollar earners, but that grew to 68,068 by 2019, according to an FPI fact sheet. Then in 2021, when the state raised taxes on those earning over $1 million, the millionaire count jumped 21% to 84,366. That same year, the number of millionaires who left the state fell, to 1,453 from nearly 2,000 the year before. (To be sure, New York’s share of the nation’s total millionaires slipped even as its raw count grows, since other states are adding them faster).

It wasn’t just income tax. In 2019, New York expanded its “mansion tax,” a transfer tax on New York City home sales over $1 million, adding a progressive scale that rises to 3.9% on sales above $25 million. New York’s millionaire count and its share of the nation’s millionaires both kept climbing since 2019, even as the tax on high-end property sales went up.

Flat tax debate

Still, threats to leave are the predictable response whenever a state floats taxing the wealthy. Ahead of California’s 2012 vote to raise its top rate to 13.3%, opponents said the rich were already packing. Instead, a Stanford-led study found about 138 high-net-worth Californians left in the year after the tax passed, just 0.04% of the 312,000 subject to it. The FPI report shows the state’s millionaire population has kept growing overall.

It seems to be a pattern: in New York this year after the city unveiled a pied-à-terre tax, Ken Griffin’s Citadel initially hinted it might abandon its planned $6 billion Park Avenue office tower in response to video from the city’s mayor calling out Griffin’s apartment. By August, Citadel confirmed it was staying, with demolition already underway at the tower site.

Still, some folks are arguing for an alternative to a graduated tax system. A Cato Institute briefing paper published this month found that states converting from graduated to flat income tax systems saw roughly 1 percentage point faster per capita income and GDP growth for five to seven years after the reform, or nearly $4,000 more income per resident. (The results are insignificant after year 10, so the boost is only seen for at most six years). “States tend to grow about a percentage point faster for five, six years, and then they return to trend,” the paper’s author, Adam N. Michel, told Fortune.

Colorado, the first state to make the switch in 1987, ended up with per capita income about 5% higher a decade later than comparable states. This year, Prop 87 will allow Colorado voters to vote on returning back to the graduated tax system. Eisner, when asked to comment on the Cato paper, told Fortune that “the timing of these changes is nowhere close to random,” and that it was timed with Prop 87. She also pointed out the data used in the paper didn’t match up: “At the state level, most of this is a zero-sum game where states are competing with each other. It’s not so much ‘we’re growing the pie for everyone.'”

Michel acknowledged that the data used was at the federal level save for the state. On the FPI paper, he said there are reasons why some stay: they’ve grown roots in that state and will stay despite the tax increase.

“Not every millionaire leaves, or even most millionaires leave, when you raise top marginal rates,” Michel said. “There are some wealthy people that leave. They tend to be the ones that are least connected in the state—after you sell your business and you’re retired, or you’re a superstar scientist or an athlete, someone that has a lot of options.”