have a guilty conscience

• feel guilt, feel self-reproach, feel regret or remorse