lose favor

• fall from grace, lose popularity, fall from a position of high respectfulness and esteem