Hey Tony,
I wrote up how to set it up with Postfix (for my own benefit, since I'm busy adding it to the installer right now). I won't be identical for Sendmail, but probably some parts are similar.
http://www.virtualmin.com/bug-tracker/bug?bug%5fnumber=147Virtualmin itself doesn't care how SMTP authentication happens, so any docs that are correct for your platform and MTA should work fine with Virtualmin. Though, I guess I should point out that you'll need to be authenticating to a user db supported by Virtualmin and your MTA (i.e. PAM, shadow, LDAP for some MTAs, etc.).