Yes, you can choose to Forget a Power Card that is currently in play when you gain a Major Power. The rulebook states:

WHAT POWER CARDS CAN I FORGET? Any of your cards! From your hand, your discard, or (if you somehow gain a Major Power mid-turn) from your cards in play. It can even be the Major Power you just chose. If you Forget a Power Card from play, you immediately lose the Elements it provided, and if you haven’t used it yet, you don’t get to. If you already used it, its effects continue until end of turn as usual. (page 18)

Summary

• You may Forget a Power Card from your hand, discard, or in play.
• If you Forget a Power Card from play:
  • You immediately lose the Elements it provided.
  • If you have not yet used the card, you do not get to use its effects.
  • If you have already used it, its effects continue until the end of the turn as usual.

Source: page 18.

Source: Core Rulebook (pp. 14, 17, 18, 20, 31)