If I have a single member LLC in one state and then move to another state, while leaving the single member LLC business behind, would I just owe taxes in the state in which I am living?
Income passes through the LLC and is paid by each member personally. Therefore, your personal state taxes would be determined by the state of your residence, not the state the LLC is formed in.
However, the state that you left might claim that you were actually earning a living in their state by virtue of your LLC being formed there–even if you were physically located in state #2. With the current economic crisis facing state budgets, I wouldn’t be surprised to see some states take this very aggressive (and unjustified, IMO) position.
This happens particularly often with California. If you owned a California LLC, I would highly advise that if you plan on moving to become a resident of another state, you look into selling the assets of the CA LLC to an LLC formed in your new state. There are might be tax implications of such a sale, so see a tax adviser, however, it is almost certain California will want to claim taxes on your California LLC (at the minimum, they will seek annual franchise taxes).
If your former state gives you trouble, you could always form a new LLC in your new state, and then transfer the assets from the old to the new LLC.
However, you would need to consult a tax adviser or accountant or attorney before doing any sales of assets or stock between the LLCs.