With rates falling as they have over the past few years, a lot of people refinanced their homes and investment properties. And yet, rates keep falling to historic lows. Does it make sense to refinance again?

If you own a second home in New York but don’t work there, you may soon be taxed like those who do, even on income made outside of the state. I love New York City and would be there every other week if I could. But I doubt I’d ever own property there, or could afford to.¬† After a recent series of court rulings, this feeling could spread. A New Canaan, Ct. resident who works in New York City and owns a second vacation home on Long Island has been told by an appeals court he must pay income taxes in New York City and state — among the highest in the nation — on income generated outside of the state even though he and his wife live in Connecticut. The reason? They own a year ’round habitable vacation home on Long Island.

John Barker, an investments manager for Neuberger Berman in New York, bought a home (with his wife, Laura) in Napeague, Long Island, between Montauk and Amagansett, (Johnny Cash’s daughter, Roseann, lives there), for $260,000 in 1997. Mr. Barker works in New York City. From 2002 to 2004, the period that was assessed for back income taxes, the Barkers said they spent only a few days a year at the Long Island home, usually during the summer. They have a permanent residence in New Canaan.

The new ruling apparently applies only to people who spend more than 183 days in New York, which naturally includes anyone who commutes to the city. (Lawyer question: what if you commute but do not spend the night in the city?) New York law requires people who work in New York to pay taxes on any income they make in this state. But they generally haven’t had to pay New York taxes on income they make outside of the state, or on their spouses’ income, if the spouse works elsewhere.

This could all change with this court ruling. And I have first hand knowledge this is sending many celebrity second home owners packing.

The recent ruling, an appeal of a 2009 decision, effectively re-interprets what counts as a permanent residence and says if you own any year round habitable property here, you are a New York state resident.

New York tax code specifically excludes “a mere camp or cottage, which is suitable and used only for vacations,” as a permanent home abode. Now New York tax experts say this interpretation/ruling is saying summer homes count as permanent residences. Long Island is loaded with summer homes. And who doesn’t want to own a NYC pied a tierre?

The appeals court that upheld the ruling says who cares how the homeowners intend to live in the home. The point is the home could be used all year long. The Barkers’ Long Island summer home is about 1,122 square feet with heat, electricity, and internet service, which the court says “makes it very habitable and comfortable year round.” The fact that Mrs. Barker let her parents stay at the home during winter months proved it was a year ’round residence, said the court.

The Barkers say  they use the home only a few days a year, and the refrigerator is even always empty.

I want to know more: did the Barkers ever lease out the house as income producing property, and if so, would that have changed the court’s ruling? Secondly, are second home owners in places like the Hamptons who live out of state going to rush in and yank out heating systems in an effort to avoid being taxes as a year ’round residence? Does having an out house qualify a home as a “camp”?

The extra taxes will cost the Barkers $1.06 million, and I’ll bet his attorney bill is almost half that. Those attorneys also say they don’t think the decision can be overturned.