I was given an opportunity to setup an email system for a friend that is similar to mine, so I figured I’d document this a bit better than what i’ve been documenting before.
As a primer, this is how we use sendmail, dovecot, and php to get email at a host and have it automatically move messages between email inboxes. Fairly easy 🙂
This should be a bit better than my previous partial writeups, specifically email.heick.email, sendmail & dovecot, how do you work…, and Dovecot IMAP (part 1).
Table of Contents:
- Base System Setup
- Software Installation
- Dovecot IMAP Configuration
- Sendmail Configuration
- Service Wrap-up
- PHP Automation
Base System Setup
To make sure I don’t futz anything up I decided to do a kick-off session in VirtualBox with a Centos 6.9 base. I used the minimal installation media, setup the domain, activated the networking with DHCP, set the root password, and got things underway.
After installation was done and the OS was rebooted, I proceeded to create myself a user, visudo that user for godness, set selinux to permissive, and reboot. Some standard maintenance before we proceed with awesomeness.
The commands are pretty much copypasta. As root:
# adduser matt # passwd matt Changing password for user matt. New password: BAD PASSWORD: it is based on a dictionary word Retype new password: passwd: all authentication tokens updated successfully. # visudo ... ## Allow root to run any commands anywhere root ALL=(ALL) ALL matt ALL=(ALL) ALL # shutdown -r now
As myself:
$ uname -a Linux niffynoo.vm 2.6.32-696.el6.x86_64 #1 SMP Tue Mar 21 19:29:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo vi /etc/selinux/config SELINUX=permissive $ sudo yum update ...tons of yum update stuff $ sudo shutdown -r now ...shutdown+restart $ uname -a Linux niffynoo.vm 2.6.32-696.23.1.el6.x86_64 #1 SMP Tue Mar 13 22:44:18 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Software Installation
We know from previous write-ups that we have a base of software to install that will get us our mail server capabilities.
$ sudo yum install dovecot sendmail cyrus-sasl sendmail-cf cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-md5 cyrus-sasl-plain ...yum installation stuff $ sudo service sendmail status sendmail is stopped sm-client is stopped $ sudo service dovecot status dovecot is stopped $ sudo service saslauthd status saslauthd is stopped
Dovecot IMAP Configuration
Dovecots configuration for what we need it to do is fairly straightforward, as we want the *@domain.com setup. All email going to the inbox will eventually be scripted out and delivered to a specific IMAP folder for easy sorting.
Users
Firstly, we’ll need a system-level user that has a home folder. This specific user is simply for permissions and resource segregation. Since we’re on this whole “magic”-imap deal, we’ll keep with the imapic setup. In a real-world scenario
$ sudo adduser imapic $ sudo passwd imapic Changing password for user imapic. New password: password BAD PASSWORD: it is based on a dictionary word Retype new password: password passwd: all authentication tokens updated successfully.
Secondly, we’re gonna go the cheap route and create a passwd file that allows the user to login with a plaintext username/password and associate themselves with the system user. It is important in this case to duplicate the uid/gid from the system user we’ve created in the previous example. We’ll manufacture the file in /etc/dovecot as users
$ id imapic
uid=501(imapic) gid=501(imapic) groups=501(imapic)
$ sudo touch /etc/dovecot/users
$ sudo su
# echo "imapic:{PLAIN}derp:501:501::/home/imapic" > /etc/dovecot/users
So, now there is a system user named imapic with the password password, and a dovecot user with the same name but a password of derp.
We now need a configuration for dovecot to read the users mail as their inbox.
Configuration
All configurations for dovecot is in /etc/dovecot/conf.d. We’ll just copy/paste our defacto config into there.
For the sake of config loading and naming, we’ll call this 99-awesome.conf
# This enables imap(143) and imaps(993) protocols = imap pop3 # Setup where we get and store mail at mail_location = mbox:~/mail:INBOX=/var/mail/%u # related to authentication disable_plaintext_auth = no auth_mechanisms = plain # Using a passwordfile !include auth-passwdfile.conf.ext # Set so we can use this from home from more than one device mail_max_userip_connections = 50
The config is actually very easy to read:
- Setup protocols to read mail
- Setup where we store IMAP folders (in the users home folder under ~/mail), and what file we determine as the main INBOX (where mail is delivered to)
- Setup how we want to authenticate, and include the standard “passwordfile” auth configuration
- Make it so that we can have multiple connections to the server from the same IP Address
Kickoff
I always like to make sure I’ve got a message waiting for me, so I’ll send a dumb email to myself:
$ echo "imapic" | sendmail imapic
..and i’ll start dovecot
$ sudo service dovecot start
and see if all is okay by talking directly to dovecot:
$ telnet localhost 143 Trying ::1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready. A login imapic derp A OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in B select INBOX B NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2018-05-05 17:14:59] C logout * BYE Logging out C OK Logout completed. Connection closed by foreign host.
Debugging
Ah, seems we have an error! It recommends to check error logs located at /var/log/maillog:
$ sudo cat /var/log/maillog May 5 17:13:13 dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled) May 5 17:14:51 dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=1609, secured May 5 17:14:59 dovecot: imap(imapic): Error: chown(/home/imapic/mail/.imap/INBOX, -1, 12(mail)) failed: Operation not permitted (egid=501(imapic), group based on /var/mail/imapic) May 5 17:14:59 dovecot: imap(imapic): Error: mkdir(/home/imapic/mail/.imap/INBOX) failed: Operation not permitted May 5 17:15:09 dovecot: imap(imapic): Disconnected: Logged out bytes=24/435
Our mail file is owned by our user, but it needs to also be a member of the same group for dovecot to even think about reading it.
$ ll /var/mail/imapic -rw-rw----. 1 imapic mail 599 May 5 17:10 /var/mail/imapic
So, lets fix that up:
$ sudo usermod -G mail imapic $ id imapic uid=501(imapic) gid=501(imapic) groups=501(imapic),12(mail)
and we try again:
$ telnet localhost 143 Trying ::1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready. A login imapic derp A OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in B select INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1525555276] UIDs valid * OK [UIDNEXT 2] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest B OK [READ-WRITE] Select completed. C logout * BYE Logging out C OK Logout completed. Connection closed by foreign host.
Perfect! We have access to our inbox and we have 1 mail waiting for us.
Sendmail Configuration
Sendmail already works like a charm. We just need to tell it about our “new domain”, and what to do with messages destined to it.
For all this, we’re gonna work as root and in /etc/mail
local-host-names
This file contains all of the alternate host names of the server (i.e. domain-name.com). Sendmail will not accept mail for a domain unless it is permitted to do so by the contents of this file.
With all that written, we’ll add our domains in this file.
virtusertable
This file is heavily documented, but it comes down to adding the rule that all mail to the domain goes to the system account that we created back in the dovecot days.
... @domain.com imapic
If we wanted mail from derp@domain.com to go to a different system account and not be dropped into the imapic “public” box, we’d add that rule above the one we’d created.
sendmail.mc
This, by far, is just something that I don’t personally understand. Items would need to be commented or changed to allow specific actions:
Add all the following to the bottom of the file:
Authentication rules:
define(`confAUTH_OPTIONS', `A')dnl define(`confAUTH_MECHANISMS', `LOGIN PLAIN')dnl TRUST_AUTH_MECH(`LOGIN PLAIN')dnl
Allow connections through a primary and secondary port for sending email messages:
DAEMON_OPTIONS(`Port=25,Name=MTA')dnl DAEMON_OPTIONS(`Port=587, Name=MSA, M=E')dnl
To save a headache later on, delete any lines in the sendmail.mc that contain any configurations above.
Once you’ve saved everything, compile:
# /etc/mail/make
Kickoff
As simple as starting/restarting the services:
# service sendmail start Starting sendmail: [ OK ] Starting sm-client: [ OK ] # service saslauthd start Starting saslauthd: [ OK ]
Also, you might want to check if postfix has 25. Stop that if necessary.
Testing
Once you’ve started everything necessary, test to make sure you can connect and authenticate with the server. It will use system username/passwords to login, so you can create a test user before telnetting and testing:
$ sudo adduser test $ sudo passwd test ... # telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 hostname ESMTP Sendmail 8.14.4/8.14.4; Sat, 5 May 2018 21:12:03 -0400 HELO derp 250 hostname Hello localhost [127.0.0.1], pleased to meet you AUTH LOGIN dGVzdA== 334 UGFzc3dvcmQ6 dGVzdA== 235 2.0.0 OK Authenticated QUIT 221 2.0.0 hostname closing connection Connection closed by foreign host.
dGVzdA==, in base64, is “test”. It’s both the username and password for this specific test account.
Service Wrap-up
Finally, after all this testing and configuration, we need to make sure we don’t have to do much more maintanance in case of server restart:
$ sudo chkconfig --level 345 dovecot on $ sudo chkconfig --level 345 saslauthd on
PHP Automation
Part of the magic of all this is to now use the power of scripting to automatically “move” all incoming email to specific mailboxes.
So, we’ll need to install some additional software:
$ yum install php php-imap
And, as the all-mighty root, we setup imap-mail-mover.php
<?php
/**
* The whole purpose of this script is to perform the following:
* 1) Open an IMAP connection to an INBOX
* 2) Look through all the messages
* 3) Grab all messages and look for the first "To: " header in each message
* 4) If the person in the "To: " is in the allowed domain
* - We grab the user
* - We check to see if their is a mailbox for that user, and move the messge there
* - We delete the message
* 5) If the person is not in the allowed domain
* - We move the message to a default folder
*/
function get_imap_folders($resource, $config)
{
// Get a list of mailboxes
$original_folders = imap_listmailbox($resource, "{" . $config['server'] . ":" . $config['port'] . "}", "*");
// these come through as {server:port}mailbox, so we just clean them up a bit
$new_folders = array();
$to_remove = "{" . $config['server'] . ":" . $config['port'] . "}";
$folders = str_replace($to_remove, "", $original_folders);
return $folders;
}
$config = array(
'server' => 'localhost',
'port' => '143',
'username' => 'redacted',
'password' => 'redacted',
'folder' => 'INBOX',
'spam' => 'SPAM',
'debug' => true,
);
$debug_message = "";
$res = imap_open("{" . $config['server'] . ":" . $config['port'] . "/service=imap/novalidate-cert" . "}" . $config['folder'], $config['username'], $config['password']);
if (!$res)
{
if ($config['debug'])
{
$debug_message = "IMAP Stream Failure";
}
die($debug_message);
}
$folders = get_imap_folders($res, $config);
// Lets get all the mail messages in the $config['folder']
$mbox = imap_check($res);
$number_messages = $mbox->Nmsgs;
if ($number_messages == 0)
{
if ($config['debug'])
{
$debug_message = "No Messages";
}
die($debug_message);
}
$range = "1:" . $number_messages;
// now, we'll get the messages
$messages = imap_fetch_overview($res, $range);
foreach ($messages as $msg)
{
$msgno = $msg->msgno;
$to = $msg->to;
echo "Message: " . $msg->subject . "\n";
if (strpos($to, "@"))
{
$array_to = explode("@", $to);
$to = $array_to[0];
}
// do we need to create a folder to move this message into?
$destination_mbox = "{" . $config['server'] . ":" . $config['port'] . "}" . $to;
if (!in_array($to, $folders))
{
if (imap_createmailbox($res, $destination_mbox))
{
echo "> Created folder [$to]\n";
}
else
{
echo "> Failed to create folder [$to]\n";
}
}
$folders = get_imap_folders($res, $config);
if (imap_mail_move($res, $msgno, $to))
{
echo "+ Moved successfully\n";
}
else
{
echo "- Failed to move message\n";
}
}
imap_expunge($res);
imap_close($res);
?>
and, finally, we set some cron job to execute it:
$ crontab -e # Create folder based on incoming message */1 * * * * /usr/bin/php -f /root/imap-mail-mover.php 2>/dev/null 1>&2