Self-Signed Certificate:
A self-signed certificate generated from the Setup is signed by Salesforce.
Process is very simple and straight forward. With just few clicks, the certificate will be generated and can be used.
CA Certificate:
A CA-signed certificate generated from the Setup is signed by an external Certificate Authority(CA).
Process is easy but not as easy as Self-Signed Certificate. Certificate will be generated and it will be send to an external Certificate Authority(CA) to get it signed. Later, the signed Certificate will be imported into Salesforce for use.
Please check the following for the Salesforce Supported Certificates if you are going to use an external Certificate Authority(CA).