This page looks best with JavaScript enabled

Milter SMTP in Python

 ·  🎃 kr0m

Intro

An SMTP milter is a software that allows us to perform certain operations on an incoming email such as reading or modifying its headers and content, checking blacklists according to the sender, responding in one way or another according to certain parameters of the email or any other action that we can think of.

This article will assume that we already have an email system working on Sendmail, if not, we can follow this previous guide

NOTE: If we do not use Sendmail as MTA, there is no problem in following the article since the milter is completely independent of the MTA used.

Pymilter

Our milter will be a Python script and for this we will use the pymilter library.

Pymilter provides several callback functions that are called in each phase of sending an email:

connect: Called for each connection to the MTA.
hello: Called when the SMTP client says HELO.
envfrom: Called when the SMTP client says MAIL FROM.
envrcpt: Called when the SMTP client says RCPT TO.
data: Called when the SMTP client says DATA.
header: Called for each header field in the message body.
eoh: Called at the blank line that terminates the header fields.
body: Called to supply the body of the message to the Milter by chunks.
unknown: Called when the SMTP client issues an unknown command.
eom: Called at the end of the message body.
abort: Called when the connection is abnormally terminated.
close: Called when the connection is closed.

All callback functions must return one of the following constants: CONTINUE, TEMPFAIL, REJECT, ACCEPT, DISCARD, SKIP.
Pymilter can use decorators to modify the behavior of callback functions, but this depends on whether the MTA allows negotiation:

  • @Milter.noreply: The callback function does not generate a response if the MTA allows negotiation, if it does not allow it, it will respond with a CONTINUE.
  • @Milter.nocallback: The callback function is disabled if the MTA allows negotiation, if it does not allow it, it will respond with a CONTINUE.

In addition to callback functions, it also provides us with functions for email manipulation:

log: Defined by subclasses to write log messages.
protocol_mask: Return mask of SMFIP_N* protocol option bits to clear for this class The @nocallback and @noreply decorators set the milter_protocol function attribute to the protocol mask bit to pass to libmilter, causing that callback or its reply to be skipped.
negotiate: Negotiate milter protocol options.
getsymval: Return the value of an MTA macro.
setreply: Set the SMTP reply code and message.
setsymlist: Tell the MTA which macro names will be used.
addheader: Add a mail header field.
chgheader: Change the value of a mail header field.
addrcpt: Add a recipient to the message.
delrcpt: Delete a recipient from the message.
replacebody: Replace the message body.
chgfrom: Change the SMTP envelope sender address.
quarantine: Quarantine the message.
progress: Tell the MTA to wait a bit longer.

NOTE: Some functions can only be called from certain callback functions, this information is in the pymilter documentation.

Milter

In our script we will use the pymilter and configparser libraries so we install them using pip:

pip install pymilter configparser

vi kr0mMilter.py
#!/usr/bin/env python
# pip install pymilter configparser
# https://pythonhosted.org/pymilter/classMilter_1_1Base.html

import Milter
import time
import email
import sys
import re
import os
import logging
import logging.handlers
import io
from io import StringIO

from Milter.utils import parse_addr
from socket import AF_INET6
from configparser import ConfigParser

## Config see ./config.ini
__version__ = '0.3b'  # version

CONFIG = os.path.join(os.path.dirname(__file__),"config.ini")

# get the configuration items
if os.path.isfile(CONFIG):
	config = ConfigParser()
	config.read(CONFIG)
	SOCKET = config.get('Milter', 'SOCKET')
	try:
		UMASK = int(config.get('Milter', 'UMASK'), base=0)
	except:
		UMASK = 0o0077
	TIMEOUT = config.getint('Milter', 'TIMEOUT')
	MAX_FILESIZE = config.getint('Milter', 'MAX_FILESIZE')

	LOGFILE_DIR = config.get('Logging', 'LOGFILE_DIR')
	LOGFILE_NAME = config.get('Logging', 'LOGFILE_NAME')
	LOGLEVEL = config.getint('Logging', 'LOGLEVEL')
else:
	sys.exit("Please check the config file! Config path: %s" % CONFIG)

# =============================================================================

LOGFILE_PATH = os.path.join(LOGFILE_DIR, LOGFILE_NAME)

# Set up a specific logger with our desired output level
log = logging.getLogger('kr0mMilter')

# disable logging by default - enable it in main app:
log.setLevel(logging.CRITICAL+1)

class kr0mMilter(Milter.Base):

	def __init__(self):  # A new instance with each new connection.
		self.id = Milter.uniqueID()	# Integer incremented with each call.

	# Called for each connection to the MTA.
	def connect(self, IPname, family, hostaddr):
		log.debug("[%d] Connect from Reverse: %s IP: %s Port: %s IPFamily: %s" % (self.id, IPname, hostaddr[0], hostaddr[1], family))
		return Milter.CONTINUE

	# Called when the SMTP client says HELO.
	def hello(self,	hostname):
		log.debug("[%d] HELO Received: %s" % (self.id, hostname))
		return Milter.CONTINUE

	# Called when the SMTP client says MAIL FROM.
	def envfrom(self, mailfrom, *str):
		log.debug("[%d] MAIL FROM Received: %s" % (self.id, mailfrom))
		return Milter.CONTINUE

	# Called when the SMTP client says RCPT TO.
	def envrcpt(self, to, *str):
		log.debug("[%d] RCPT TO Received: %s" % (self.id, to))
		return Milter.CONTINUE
	
	# Called when the SMTP client says DATA.
	def data(self):
		log.debug("[%d] DATA command received" % (self.id))
		return Milter.CONTINUE

	# Called for each header field in the message body(all the headers are shown when email message is ended with . command).
	def header(self, header_field, header_field_value):
		log.debug("[%d] New HEADER in message body: %s -> %s" % (self.id, header_field, header_field_value))
		return Milter.CONTINUE

	# Called at the blank line that terminates the header fields.
	def eoh(self):
		log.debug("[%d] END of headers detected" % (self.id))
		return Milter.CONTINUE

	# Called to supply the body of the message to the Milter by chunks.
	def body(self, chunk):
		log.debug("[%d] BODY chunk: %s" % (self.id, chunk))
		return Milter.CONTINUE

	# Called when the SMTP client issues an unknown command.
	def unknown(self, cmd):
		log.debug("[%d] UNKNOWN command: %s" % (self.id, cmd))
		return Milter.CONTINUE

	# Called at the end of the message body.
	def eom(self):
		log.debug("[%d] END OF MESSAGE BODY detected" % (self.id))
		return Milter.CONTINUE

	# Called when the connection is abnormally terminated.
	def abort(self):
		log.debug("[%d] ABORT connection abnormally terminated" % (self.id))
		return Milter.CONTINUE

	# Called when the connection is closed.
	def close(self):
		log.debug("[%d] CLOSE connection closed" % (self.id))
		return Milter.CONTINUE

# =============================================================================

def main():
	
	# make sure the log directory exists:
	try:
		os.makedirs(LOGFILE_DIR,0o0027)
	except:
		pass

	# Add the log message handler to the logger
	oldumask = os.umask(0o0026)
	handler = logging.handlers.WatchedFileHandler(LOGFILE_PATH, encoding='utf8')
	# create formatter and add it to the handlers
	formatter = logging.Formatter('%(asctime)s - %(levelname)8s: %(message)s')
	handler.setFormatter(formatter)
	log.addHandler(handler)
	os.umask(oldumask)

	# Loglevels are: 1 = Debug, 2 = Info, 3 = Error

	if LOGLEVEL == 2:
		log.setLevel(logging.INFO)
	elif LOGLEVEL == 3:
		log.setLevel(logging.WARNING)
	else:
		log.setLevel(logging.DEBUG)

	# Register to have the Milter factory create instances of the class:
	Milter.factory = kr0mMilter

	# Define wich actions will be allowed to our milter
	flags = Milter.CHGBODY
	flags += Milter.CHGHDRS
	flags += Milter.ADDHDRS
	flags += Milter.ADDRCPT
	flags += Milter.DELRCPT
	Milter.set_flags(flags)  # tell Sendmail which features we use

	# start milter processing
	print("%s kr0mMilter startup - Version %s" % (time.strftime('%Y-%m-%d %H:%M:%S'), __version__ ))
	print('logging to file %s' % LOGFILE_PATH)

	log.info('Starting kr0mMilter v%s - listening on %s' % (__version__, SOCKET))
	log.debug('Python version: %s' % sys.version)
	sys.stdout.flush()

	# ensure desired permissions on unix socket
	os.umask(UMASK)

	# set the "last" fall back to ACCEPT if exception occur
	Milter.set_exception_policy(Milter.ACCEPT)

	# start the milter
	Milter.runmilter("kr0mMilter", SOCKET, TIMEOUT)

	print("%s kr0mMilter shutdown" % time.strftime('%Y-%m-%d %H:%M:%S'))

if __name__ == "__main__":
	main()

MTA Configuration

Depending on the MTA we use, we must generate the Unix socket in one path or another, for example Postfix is ​​by default chrooted to /var/spool/postfix:

/var/run/kr0mMilter.sock

/var/spool/postfix/var/run/kr0mMilter.sock

We generate the milter configuration file, which will be read at startup:

vi config.ini

[Milter]
# bind to unix or tcp socket "inet:port@ip" or "/<path>/<to>/<something>.sock"
SOCKET = /var/run/kr0mMilter.sock
# Set umask for unix socket, e.g. 0077 for group writable
UMASK = 0077
# Milter timout in seconds
TIMEOUT = 30
# Define the max size for each message in bytes (~50MB)
MAX_FILESIZE = 50000000

[Logging]
LOGFILE_DIR = /var/log/kr0mMilter
LOGFILE_NAME = kr0mMilter.log
# Loglevels are: 1 = Debug (default) , 2 = Info, 3 = Warning/Error
LOGLEVEL = 1

MTA: Sendmail

Now that we have our milter, we need to configure Sendmail.

cd /etc/mail

We define the milter and add it to the confINPUT_MAIL_FILTERS list:

vi DrWho.alfaexploit.com.mc

MAIL_FILTER(`kr0mMilter', `S=/var/run/kr0mMilter.sock, F=T, T=R:2m')
define(`confINPUT_MAIL_FILTERS', `spamassassin, dkim-filter, kr0mMilter')

We compile the new configuration to M4:

make

We update the Sendmail configuration with ours:

cp DrWho.alfaexploit.com.cf sendmail.cf

We restart the service:

service sendmail restart

MTA: Postfix

We edit its configuration as follows:

vi /etc/postfix/main.cf

smtpd_milters = unix:/var/run/kr0mMilter.sock

We restart the service:

service postfix restart

Milter testing

We start our milter:

python kr0mMilter.py

2021-09-03 16:35:56 kr0mMilter startup - Version 0.3b
logging to file /var/log/kr0mMilter/kr0mMilter.log

In another console, we open the logs:

tail -f /var/log/kr0mMilter/kr0mMilter.log

2021-09-05 09:33:36,224 -     INFO: Starting kr0mMilter v0.3b - listening on /var/run/kr0mMilter.sock
2021-09-05 09:33:36,224 -    DEBUG: Python version: 3.8.10 (default, Jul  4 2021, 01:12:00) [Clang 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c

We perform a test to verify that the milter is working correctly. In my case, it is sending an email from the RainLoop web interface to GMail:

2021-09-05 11:41:40,154 -     INFO: Starting kr0mMilter v0.3b - listening on /var/run/kr0mMilter.sock
2021-09-05 11:41:40,154 -    DEBUG: Python version: 3.8.10 (default, Jul  4 2021, 01:12:00) 
[Clang 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c
2021-09-05 11:42:00,478 -    DEBUG: [1] Connect from Reverse: [192.168.69.6] IP: 192.168.69.6 Port: 50717 IPFamily: 2
2021-09-05 11:42:00,478 -    DEBUG: [1] HELO Received: mail.alfaexploit.com
2021-09-05 11:42:00,518 -    DEBUG: [1] MAIL FROM Received: <kr0m@alfaexploit.com>
2021-09-05 11:42:00,559 -    DEBUG: [1] RCPT TO Received: <jjivarspoquet@gmail.com>
2021-09-05 11:42:00,559 -    DEBUG: [1] DATA command received
2021-09-05 11:42:00,841 -    DEBUG: [1] New HEADER in message body: DKIM-Signature -> v=1; a=rsa-sha256; c=simple/simple; d=alfaexploit.com;
	s=smtp; t=1630834920;
	bh=uITeNK7M/REy9bj3VrPs3Ye/r1/XrjtgJqzuPdU1YBA=;
	h=Date:From:Subject:To;
	b=iBNrLJEfYLhSAQ5Pruz5A7Jx2pYuoFsLTuP1dwlRiORP4l+PC4/o6ZcPHCQIMHVu0
	 sKzOyFDAPq3+rPnao9cX9MPAFCzXsPYNFv5F5MxV0F2qou0QnnkrNmsu31SuvahwR4
	 iWF5GPUGKVYhKkxL7RE5TnId0ScfWC/OMaB0RVA4=
2021-09-05 11:42:00,841 -    DEBUG: [1] New HEADER in message body: MIME-Version -> 1.0
2021-09-05 11:42:00,842 -    DEBUG: [1] New HEADER in message body: Date -> Sun, 05 Sep 2021 09:42:00 +0000
2021-09-05 11:42:00,842 -    DEBUG: [1] New HEADER in message body: Content-Type -> multipart/alternative;
 boundary="--=_RainLoop_336_866536040.1630834920"
2021-09-05 11:42:00,842 -    DEBUG: [1] New HEADER in message body: X-Mailer -> RainLoop/1.15.0
2021-09-05 11:42:00,843 -    DEBUG: [1] New HEADER in message body: From -> kr0m@alfaexploit.com
2021-09-05 11:42:00,843 -    DEBUG: [1] New HEADER in message body: Message-ID -> <f2fe9a15d5aa35de261a63fc9aed35b9@alfaexploit.com>
2021-09-05 11:42:00,843 -    DEBUG: [1] New HEADER in message body: Subject -> test envio con milter
2021-09-05 11:42:00,843 -    DEBUG: [1] New HEADER in message body: To -> jjivarspoquet@gmail.com
2021-09-05 11:42:00,850 -    DEBUG: [1] New HEADER in message body: X-Spam-Status -> No, score=-1.0 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE
	autolearn=unavailable autolearn_force=no version=3.4.5
2021-09-05 11:42:00,851 -    DEBUG: [1] New HEADER in message body: X-Spam-Checker-Version -> SpamAssassin 3.4.5 (2021-03-20) on
	DrWho.alfaexploit.com
2021-09-05 11:42:00,851 -    DEBUG: [1] END of headers detected
2021-09-05 11:42:00,852 -    DEBUG: [1] BODY chunk: b'\r\n----=_RainLoop_336_866536040.1630834920\r\nContent-Type: text/plain; charset="utf-8"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n1234\r\n\r\n----=_RainLoop_336_866536040.1630834920\r\nContent-Type: text/html; charset="utf-8"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n<!DOCTYPE html><html><head><meta http-equiv=3D"Content-Type" content=3D"t=\r\next/html; charset=3Dutf-8" /></head><body><div data-html-editor-font-wrap=\r\nper=3D"true" style=3D"font-family: arial, sans-serif; font-size: 13px;"><=\r\nbr>1234<signature></signature></div></body></html>\r\n\r\n----=_RainLoop_336_866536040.1630834920--\r\n'
2021-09-05 11:42:00,852 -    DEBUG: [1] END OF MESSAGE BODY detected
2021-09-05 11:42:00,886 -    DEBUG: [1] CLOSE connection closed

As you can see, milters can be used to debug both the sending and receiving of emails. But let’s go a step further and make it so that when we send an email with a specific subject, a copy of that email is sent to a recipient.

Snooping via milter

We will need to modify some code. First, we will add some fields to the milter instance to store and check these values in different callback functions:

	def __init__(self):  # A new instance with each new connection.
		self.id = Milter.uniqueID()	# Integer incremented with each call.
		self.additional_rcpts = []	# Additional rcpts when secret subject detected 
		self.source = ''			# Source variable to check if source is kr0m
		self.destination = ''		# Destination variable to check if destination is kr0m

When analyzing the email, we save its source and destination to be able to access this value in the header callback function. This way, we can snoop on all incoming and outgoing emails on the server, except those originating/destined for kr0m@alfaexploit.com :

	def envfrom(self, mailfrom, *str):
		log.debug("[%d] MAIL FROM Received: %s" % (self.id, mailfrom))
		self.source = mailfrom
		return Milter.CONTINUE

	def envrcpt(self, to, *str):
		log.debug("[%d] RCPT TO Received: %s" % (self.id, to))
		self.destination = to
		return Milter.CONTINUE

If the subject contains the word “secret”, we add an rcpt to the additional_rcpts array with one exception: self.source or self.destination is kr0m@alfaexploit.com . This way, we don’t snoop on that account:

	def header(self, header_field, header_field_value):
		log.debug("[%d] New HEADER in message body: %s -> %s" % (self.id, header_field, header_field_value))
		if header_field == "Subject":
			log.debug("[%d] Checking secret regexp under Subject" % (self.id))
			pattern = '.*secret.*'
			secret_document = re.match(pattern, header_field_value, re.IGNORECASE)
			log.debug("[%d] Source: %s" % (self.id, self.source))
			log.debug("[%d] Destination: %s" % (self.id, self.destination))
			if secret_document:
				if self.source != '<kr0m@alfaexploit.com>' and self.destination != '<kr0m@alfaexploit.com>':
					log.debug("[%d] Secret document detected" % (self.id))
					self.additional_rcpts.append('kr0m@alfaexploit.com')
				
		return Milter.CONTINUE

Finally, before allowing the email to be sent, we check if any additional rcpt to needs to be added and act accordingly:

	def eom(self):
		log.debug("[%d] END OF MESSAGE BODY detected" % (self.id))
		if len(self.additional_rcpts) > 0:
			for additional_rcpt in self.additional_rcpts:
				log.debug("[%d] Adding additional RCPT TO: %s" % (self.id, additional_rcpt))
				self.addrcpt(additional_rcpt)

		return Milter.CONTINUE

When someone sends an email with the subject: ‘.secret.’ (re.IGNORECASE), we will receive a copy at kr0m@alfaexploit.com .

Similarly, since we have done this, we could make any type of modification to the emails, add/remove content/headers, or analyze attachments for viruses and remove those attachments.

As a final note, it is worth mentioning that milters in both SendMail and Postfix seem to execute simultaneously. Therefore, we cannot chain milters that depend on the execution of a previous one. In these cases, we must program a single milter that performs all the necessary operations.

If you liked the article, you can treat me to a RedBull here