Only you have the authority to cancel your Power of Attorney.
Although an attorney-in-fact can decline to act for any reason, it’s not technically a "revocation". If they decline to act, the other attorney-in-fact or the substitute (if you named any) typically takes over. Otherwise, you’ll need to appoint a new agent.
Further, an attorney-in-fact cannot revoke the Power of Attorney. This is true even if one attorney-in-fact has the authority to make legal decisions without consulting the other (i.e. in a joint and independent situation). As the principal, you’ve given someone authority to act on your behalf. An attorney-in-fact cannot invalidate your wishes.
It’s also important to note that courts have the authority to revoke a Power of Attorney in certain situations (e.g., when a principal with a Durable Power of Attorney and two attorneys-in-fact are all incapacitated). Family members can file the court application but can’t revoke any powers themselves.
Can a Power of Attorney be irrevocable?
An irrevocable Power of Attorney is uncommon. You can rescind your Power of Attorney at any time, as long as you are competent and notify your attorney-in-fact. This is true even if the Power of Attorney has a specified end date.
However, it’s important to note that a Durable Power of Attorney remains in effect even if you’re mentally incapacitated. At that point, you’ll no longer be able to revoke its powers.