Had a "relationship" with my straight boss, and...?

Why not start something discreet with him again; Harmless fun. I think you should tell him that you are leaving and that will turn him on. Just remember though, his wife and children come before you. Just keep it discreet.