Based on the rulebook context:

• At the end of the game, scoring for player factions is based on the number of Havens they have on the map, any hex VP bonuses for Havens, and any VP gained by spending resources (page 30).
• The Empire and Chaos score VP for the number of player units in their respective Graveyards (page 30):
  • The Empire: "2 VP for each faction in the Imperial Graveyard (so if there was 1 Krowh Unit and 2 Druwhn Units, The Empire would score 4 VP)"
  • Chaos: "2 VP for each Faction in the Chaos Graveyard"

There is no mention in the provided context that a player is "eliminated" or loses points for having units in the Chaos or Empire Graveyard. The only effect is that The Empire and Chaos score VP for each faction represented in their respective Graveyards.

Player scoring is determined by:

• 2 VP for each Haven on the map (only their own)
• 1 VP for each hex bonus where they have a Haven
• VP for spending resources (5 resources = 1 VP, any number of times, usually in the last Chapter)

Summary

• Only Havens, hex VP, and resource conversion matter for player scoring.
• Player units in the Chaos or Empire Graveyard do not affect the player's own VP, but do give VP to The Empire or Chaos.
• There is no rule in the context that says a player is "eliminated" for having units in a Graveyard.

Source: page 30, page 31

Source: Core Rulebook (pp. 30, 31, 34, 36, 57)