You feared you were being hard-hearted, but refusing to yield shows that you truly care. Tough love can be tough sometimes.