Resignation: Terminating Employment Contract Before Start Date

My question involves labor and employment law for the state of: NJ

I signed a employment contract with a at will employer "with whom I can end the relationship with or without cause and with or without notice" with the start date in one week. But due to some personal reasons I decided not join the employer.

Not the employer is threatening me that there could be financial responsibilities as I already signed the contract. And I must join this contract.
I want to know if these claim has any base as this is at-will contract

Also, I am on H1B and employer has already started my visa transfer process, I took the drug test and background check is done. So can they sue me or demand money from me?
====================================
companyX is an employer at will. This means that either you or companyX can end the employment relationship at
any time, for any reason, with or without cause, unless prohibited by law. This also means that companyX can change
the terms of your employment, including pay rates, with or without notice, unless notice is required by law. This
Confirmation is not a contract. A vital component of companyX mission and culture is promoting workforce diversity
within our ranks, with our suppliers and with our clients. We strive to create inclusive environments that embrace
those with diverse characteristics, including gender, race, ethnicity, religion, disability, age, sexual orientation, work
experience, social class, educational background and any other status protected by law. The Employment
Agreement Terms and Conditions that you signed with CompanyX, and not this document, govern the terms of your
employment with CompanyX.
=========================================

Resignation: Terminating Employment Contract Before Start Date