The problem was that I want only secure connections to my sendmail server, using SSL with a single, privately distributed, password protected, certificate signed by my own (made up) signing authority, and allowing no mail to be sent without using the certificate.
The solution was creating the self-signed certificate, configuring sendmail to accept requests to send mail only when the cert is present, and then configuring all MUA's (Mail User Agents) to use the cert.
I'm not going to get into how to create self-signed certs since it is well documented online and the same with configuring sendmail (though maybe I'll document that part later cuz sendmail is a bear :). Once the cert has been created and sendmail configured, the public, password protected, portion of the cert (a PKCS12 file, *.p12) must be distributed to anyone wishing to send mail using the server, and imported into their MUAs. The method for this varies with each MUA, but with Thunderbird, in which the process is similar to most others, you go to Tools->Options->Advanced->Encryption then "View Certificates" and "Import". Import the cert file and enter the certificate password when prompted. Users will then need to "trust" the made up signing authority used to sign the cert, but this can be trusted forever by most systems which also allows you to use other certs signed by the same authority for other purposes (eg web certs).
One caveat is that if you're using an SSL connection, you don't want or need to check "Use Secure Authentication" in any server settings because the whole connection is secure and separately secured authentication is unavailable and superfluous.
Posted by Mike at September 13, 2007 10:58 PM