The biggest drawback to a power of attorney is that an agent may act in a way that the principal would disapprove of. This may be unintentional if they are ignorant of the principal’s wishes, or it may be intentional because they’re acting in bad faith. As POAs don’t have court oversight, they can be susceptible to abuse or exploitation.
POA agreements may also not be honored, largely due to the lack of court oversight. Many banks, for example, ask you to sign their forms rather than accepting a POA. Lastly, the principal must be competent to execute a POA, which can be a disadvantage if it’s not set up before they become incapacitated.