drive to desperation

• exasperate someone, drive someone crazy; causing someone to feel despair or hopelessness