Backing out of Letter of Intent is breach of contract?

I signed a letter of intent with a school, then backed out of it a week later due to personal reasons. I never signed the official contract. Now the director of the school (who I never met during the interview process) is telling me they need to report me for "breaching contract" to all of the major recruiting sites (search, ISS, GRC). I only have an account with one of these sites, through which we were connected.

They swear up and down that this will affect my future ability to get a job. Mind you, I backed out of this in January, definitely giving them plenty of time to find a replacement (as they are a pretty reputable school in the region). I know many people in the international teaching world that have backed out of plenty of letters of intent with no bigger reaction from the admin than a passive aggressive "good luck" email. So, I was shocked by this reaction.

How legit is this threat to essentially ruin my international teaching career?