Sleeping with the boss has almost never ended well for anyone - not the boss, and not the person sleeping with the boss. And that's not even counting the chaos that ensues for the rest of the workplace when this happens! In other words, this is a potential professional minefield, and you're right to be concerned about possible repercussions.
Before you go, could you use your tenured position to advocate for others at the institution who have less power? Graduate students organizing a union could always use more support from faculty.