owe one a spite

• hold animosity, hold a grudge against someone