If you are unable to delete SAML Single Sign-On Settings in Salesforce, check the below before deleting it.

1. Go to Domain Management –> My Domain. Check whether the SAML Settings is disabled in “Authentication Configuration” section. If not, please disable it.

2. Please check the Portal Login and Registration Settings and Community Login and Registration Settings and do the same.

Cheers!!!