Yes, you can do it. No, it's not an exploit. It's an unintentional, unofficial side effect of multiple houses.
Think of it as in the same catagory as a breakout house. They don't mind if we do it. But if something goes horribly wrong (such as a lost spec), don't expect any help.
Also, as you swap them around, you *will* go mad trying to keep track of which spec is in which house mirror at any given time.