Answer

If a Haven is removed from a hex during the Nemesis Phase (for example, by an enemy effect or the placement of a Curse), the following applies:

• Player Units remain: If there are player Units on the hex when the Haven is removed, those Units remain on the hex. This is explicitly stated in the context of Curses:

  "If there are player Units here, they remain and may be Commanded off of it;"
  (Page 49)

• Commanding Units: You may still use the Command action to move (Command) your Units off of the hex in subsequent turns, even if the Haven is gone or the hex is Cursed.
  (Page 49: "may be Commanded off of it")

• Building on the Hex:

  • If the hex is Cursed, you cannot use the Haven action there ("You can not use the Haven Action in this hex" — Page 49), and you cannot build a Wall or Tower without a Haven (Page 23).
  • If the hex is not Cursed and meets the requirements (explored, no enemy Units, no existing Haven, not The Capital or Ice Wastes), you may use the Haven action to build a new Haven there in a future turn (Page 23).

Summary

• Units remain on the hex after a Haven is removed.
• Units can be Commanded off the hex in future turns.
• You cannot build (Haven, Wall, or Tower) on a Cursed hex, but if the Curse is removed and requirements are met, you may build again.

Sources:
Page 49 (Curses and removing Havens), Page 23 (Haven action requirements)

Source: Core Rulebook (pp. 11, 21, 23, 27, 49)