Esta pagina se ve mejor con JavaScript habilitado

Milter SMTP en Python

 ·  🎃 kr0m

Intro

Un milter SMTP es un software que nos permitirá realizar ciertas operaciones sobre un email entrante como leer o modificar sus cabeceras y contenido, comprobar blacklists según remitente, responder de un modo u otro según ciertos parámetros del email o cualquier otra acción que se nos ocurra.

Este artículo va a suponer que ya tenemos un sistema de email funcionando sobre Sendmail, de no ser así podemos seguir esta guía anterior

NOTA: Si no utilizamos Sendmail como MTA no hay problema con seguir el artículo ya que el milter es completamente independiente del MTA utilizado.

Pymilter

Nuestro milter será un script en Python y para ello utilizaremos la librería pymilter

Pymilter proporciona varias funciones de callback que son llamadas en cada fase del envío de un email:

connect: Called for each connection to the MTA.
hello: Called when the SMTP client says HELO.
envfrom: Called when the SMTP client says MAIL FROM.
envrcpt: Called when the SMTP client says RCPT TO.
data: Called when the SMTP client says DATA.
header: Called for each header field in the message body.
eoh: Called at the blank line that terminates the header fields.
body: Called to supply the body of the message to the Milter by chunks.
unknown: Called when the SMTP client issues an unknown command.
eom: Called at the end of the message body.
abort: Called when the connection is abnormally terminated.
close: Called when the connection is closed.

Todas las funciones de callback deben retornar alguna de las siguientes constantes: CONTINUE, TEMPFAIL, REJECT, ACCEPT, DISCARD, SKIP.
Pymilter puede utilizar decorators para modificar el comportamiento de las funciones de callback, pero esto depende de si la MTA permite la negociación:

  • @Milter.noreply: La función de callback no genera respuesta si la MTA permite la negociación, en caso de no permitirla responderá con un CONTINUE.
  • @Milter.nocallback: La función de callback queda deshabilitada si la MTA permite la negociación, en caso de no permitirla responderá con un CONTINUE.

Además de las funciones de callback también nos proporciona funciones funciones de manipulación del email:

log: Defined by subclasses to write log messages.
protocol_mask: Return mask of SMFIP_N* protocol option bits to clear for this class The @nocallback and @noreply decorators set the milter_protocol function attribute to the protocol mask bit to pass to libmilter, causing that callback or its reply to be skipped.
negotiate: Negotiate milter protocol options.
getsymval: Return the value of an MTA macro.
setreply: Set the SMTP reply code and message.
setsymlist: Tell the MTA which macro names will be used.
addheader: Add a mail header field.
chgheader: Change the value of a mail header field.
addrcpt: Add a recipient to the message.
delrcpt: Delete a recipient from the message.
replacebody: Replace the message body.
chgfrom: Change the SMTP envelope sender address.
quarantine: Quarantine the message.
progress: Tell the MTA to wait a bit longer.

NOTA: Algunas funciones solo pueden ser llamadas desde determinadas funciones de callback, esta información está en la documentación de pymilter.

Milter

En nuestro script vamos a emplear las librerías pymilter y configparser así que las instalamos mediante pip:

pip install pymilter configparser

vi kr0mMilter.py
#!/usr/bin/env python
# pip install pymilter configparser
# https://pythonhosted.org/pymilter/classMilter_1_1Base.html

import Milter
import time
import email
import sys
import re
import os
import logging
import logging.handlers
import io
from io import StringIO

from Milter.utils import parse_addr
from socket import AF_INET6
from configparser import ConfigParser

## Config see ./config.ini
__version__ = '0.3b'  # version

CONFIG = os.path.join(os.path.dirname(__file__),"config.ini")

# get the configuration items
if os.path.isfile(CONFIG):
	config = ConfigParser()
	config.read(CONFIG)
	SOCKET = config.get('Milter', 'SOCKET')
	try:
		UMASK = int(config.get('Milter', 'UMASK'), base=0)
	except:
		UMASK = 0o0077
	TIMEOUT = config.getint('Milter', 'TIMEOUT')
	MAX_FILESIZE = config.getint('Milter', 'MAX_FILESIZE')

	LOGFILE_DIR = config.get('Logging', 'LOGFILE_DIR')
	LOGFILE_NAME = config.get('Logging', 'LOGFILE_NAME')
	LOGLEVEL = config.getint('Logging', 'LOGLEVEL')
else:
	sys.exit("Please check the config file! Config path: %s" % CONFIG)

# =============================================================================

LOGFILE_PATH = os.path.join(LOGFILE_DIR, LOGFILE_NAME)

# Set up a specific logger with our desired output level
log = logging.getLogger('kr0mMilter')

# disable logging by default - enable it in main app:
log.setLevel(logging.CRITICAL+1)

class kr0mMilter(Milter.Base):

	def __init__(self):  # A new instance with each new connection.
		self.id = Milter.uniqueID()	# Integer incremented with each call.

	# Called for each connection to the MTA.
	def connect(self, IPname, family, hostaddr):
		log.debug("[%d] Connect from Reverse: %s IP: %s Port: %s IPFamily: %s" % (self.id, IPname, hostaddr[0], hostaddr[1], family))
		return Milter.CONTINUE

	# Called when the SMTP client says HELO.
	def hello(self,	hostname):
		log.debug("[%d] HELO Received: %s" % (self.id, hostname))
		return Milter.CONTINUE

	# Called when the SMTP client says MAIL FROM.
	def envfrom(self, mailfrom, *str):
		log.debug("[%d] MAIL FROM Received: %s" % (self.id, mailfrom))
		return Milter.CONTINUE

	# Called when the SMTP client says RCPT TO.
	def envrcpt(self, to, *str):
		log.debug("[%d] RCPT TO Received: %s" % (self.id, to))
		return Milter.CONTINUE
	
	# Called when the SMTP client says DATA.
	def data(self):
		log.debug("[%d] DATA command received" % (self.id))
		return Milter.CONTINUE

	# Called for each header field in the message body(all the headers are shown when email message is ended with . command).
	def header(self, header_field, header_field_value):
		log.debug("[%d] New HEADER in message body: %s -> %s" % (self.id, header_field, header_field_value))
		return Milter.CONTINUE

	# Called at the blank line that terminates the header fields.
	def eoh(self):
		log.debug("[%d] END of headers detected" % (self.id))
		return Milter.CONTINUE

	# Called to supply the body of the message to the Milter by chunks.
	def body(self, chunk):
		log.debug("[%d] BODY chunk: %s" % (self.id, chunk))
		return Milter.CONTINUE

	# Called when the SMTP client issues an unknown command.
	def unknown(self, cmd):
		log.debug("[%d] UNKNOWN command: %s" % (self.id, cmd))
		return Milter.CONTINUE

	# Called at the end of the message body.
	def eom(self):
		log.debug("[%d] END OF MESSAGE BODY detected" % (self.id))
		return Milter.CONTINUE

	# Called when the connection is abnormally terminated.
	def abort(self):
		log.debug("[%d] ABORT connection abnormally terminated" % (self.id))
		return Milter.CONTINUE

	# Called when the connection is closed.
	def close(self):
		log.debug("[%d] CLOSE connection closed" % (self.id))
		return Milter.CONTINUE

# =============================================================================

def main():
	
	# make sure the log directory exists:
	try:
		os.makedirs(LOGFILE_DIR,0o0027)
	except:
		pass

	# Add the log message handler to the logger
	oldumask = os.umask(0o0026)
	handler = logging.handlers.WatchedFileHandler(LOGFILE_PATH, encoding='utf8')
	# create formatter and add it to the handlers
	formatter = logging.Formatter('%(asctime)s - %(levelname)8s: %(message)s')
	handler.setFormatter(formatter)
	log.addHandler(handler)
	os.umask(oldumask)

	# Loglevels are: 1 = Debug, 2 = Info, 3 = Error

	if LOGLEVEL == 2:
		log.setLevel(logging.INFO)
	elif LOGLEVEL == 3:
		log.setLevel(logging.WARNING)
	else:
		log.setLevel(logging.DEBUG)

	# Register to have the Milter factory create instances of the class:
	Milter.factory = kr0mMilter

	# Define wich actions will be allowed to our milter
	flags = Milter.CHGBODY
	flags += Milter.CHGHDRS
	flags += Milter.ADDHDRS
	flags += Milter.ADDRCPT
	flags += Milter.DELRCPT
	Milter.set_flags(flags)  # tell Sendmail which features we use

	# start milter processing
	print("%s kr0mMilter startup - Version %s" % (time.strftime('%Y-%m-%d %H:%M:%S'), __version__ ))
	print('logging to file %s' % LOGFILE_PATH)

	log.info('Starting kr0mMilter v%s - listening on %s' % (__version__, SOCKET))
	log.debug('Python version: %s' % sys.version)
	sys.stdout.flush()

	# ensure desired permissions on unix socket
	os.umask(UMASK)

	# set the "last" fall back to ACCEPT if exception occur
	Milter.set_exception_policy(Milter.ACCEPT)

	# start the milter
	Milter.runmilter("kr0mMilter", SOCKET, TIMEOUT)

	print("%s kr0mMilter shutdown" % time.strftime('%Y-%m-%d %H:%M:%S'))

if __name__ == "__main__":
	main()

Configuración MTA

Según el MTA que utilicemos debemos generar el socket unix en un path u otro, por ejemplo Postfix por defecto está chrooteado a /var/spool/postfix:

/var/run/kr0mMilter.sock

/var/spool/postfix/var/run/kr0mMilter.sock

Generamos el fichero de configuración del milter, este será leído en el arranque:

vi config.ini

[Milter]
# bind to unix or tcp socket "inet:port@ip" or "/<path>/<to>/<something>.sock"
SOCKET = /var/run/kr0mMilter.sock
# Set umask for unix socket, e.g. 0077 for group writable
UMASK = 0077
# Milter timout in seconds
TIMEOUT = 30
# Define the max size for each message in bytes (~50MB)
MAX_FILESIZE = 50000000

[Logging]
LOGFILE_DIR = /var/log/kr0mMilter
LOGFILE_NAME = kr0mMilter.log
# Loglevels are: 1 = Debug (default) , 2 = Info, 3 = Warning/Error
LOGLEVEL = 1

MTA: Sendmail

Ahora que ya tenemos nuestro milter debemos configurar Sendmail.

cd /etc/mail

Definimos el milter y lo añadimos a la lista confINPUT_MAIL_FILTERS:

vi DrWho.alfaexploit.com.mc

MAIL_FILTER(`kr0mMilter', `S=/var/run/kr0mMilter.sock, F=T, T=R:2m')
define(`confINPUT_MAIL_FILTERS', `spamassassin, dkim-filter, kr0mMilter')

Compilamos a M4 la nueva configuración:

make

Actualizamos la configuración de Sendmail con la nuestra:

cp DrWho.alfaexploit.com.cf sendmail.cf

Reiniciamos el servicio:

service sendmail restart

MTA: Postfix

Editamos su configuración del siguiente modo:

vi /etc/postfix/main.cf

smtpd_milters = unix:/var/run/kr0mMilter.sock

Reiniciamos el servicio:

service postfix restart

Pruebas milter

Arrancamos nuestro milter:

python kr0mMilter.py

2021-09-03 16:35:56 kr0mMilter startup - Version 0.3b
logging to file /var/log/kr0mMilter/kr0mMilter.log

En otra consola abrimos los logs:

tail -f /var/log/kr0mMilter/kr0mMilter.log

2021-09-05 09:33:36,224 -     INFO: Starting kr0mMilter v0.3b - listening on /var/run/kr0mMilter.sock
2021-09-05 09:33:36,224 -    DEBUG: Python version: 3.8.10 (default, Jul  4 2021, 01:12:00) [Clang 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c

Realizamos una prueba para comprobar que el milter funciona correctamente, en mi caso se trata de un envío de email desde la interfaz web RainLoop a GMail:

2021-09-05 11:41:40,154 -     INFO: Starting kr0mMilter v0.3b - listening on /var/run/kr0mMilter.sock
2021-09-05 11:41:40,154 -    DEBUG: Python version: 3.8.10 (default, Jul  4 2021, 01:12:00) 
[Clang 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c
2021-09-05 11:42:00,478 -    DEBUG: [1] Connect from Reverse: [192.168.69.6] IP: 192.168.69.6 Port: 50717 IPFamily: 2
2021-09-05 11:42:00,478 -    DEBUG: [1] HELO Received: mail.alfaexploit.com
2021-09-05 11:42:00,518 -    DEBUG: [1] MAIL FROM Received: <kr0m@alfaexploit.com>
2021-09-05 11:42:00,559 -    DEBUG: [1] RCPT TO Received: <jjivarspoquet@gmail.com>
2021-09-05 11:42:00,559 -    DEBUG: [1] DATA command received
2021-09-05 11:42:00,841 -    DEBUG: [1] New HEADER in message body: DKIM-Signature -> v=1; a=rsa-sha256; c=simple/simple; d=alfaexploit.com;
	s=smtp; t=1630834920;
	bh=uITeNK7M/REy9bj3VrPs3Ye/r1/XrjtgJqzuPdU1YBA=;
	h=Date:From:Subject:To;
	b=iBNrLJEfYLhSAQ5Pruz5A7Jx2pYuoFsLTuP1dwlRiORP4l+PC4/o6ZcPHCQIMHVu0
	 sKzOyFDAPq3+rPnao9cX9MPAFCzXsPYNFv5F5MxV0F2qou0QnnkrNmsu31SuvahwR4
	 iWF5GPUGKVYhKkxL7RE5TnId0ScfWC/OMaB0RVA4=
2021-09-05 11:42:00,841 -    DEBUG: [1] New HEADER in message body: MIME-Version -> 1.0
2021-09-05 11:42:00,842 -    DEBUG: [1] New HEADER in message body: Date -> Sun, 05 Sep 2021 09:42:00 +0000
2021-09-05 11:42:00,842 -    DEBUG: [1] New HEADER in message body: Content-Type -> multipart/alternative;
 boundary="--=_RainLoop_336_866536040.1630834920"
2021-09-05 11:42:00,842 -    DEBUG: [1] New HEADER in message body: X-Mailer -> RainLoop/1.15.0
2021-09-05 11:42:00,843 -    DEBUG: [1] New HEADER in message body: From -> kr0m@alfaexploit.com
2021-09-05 11:42:00,843 -    DEBUG: [1] New HEADER in message body: Message-ID -> <f2fe9a15d5aa35de261a63fc9aed35b9@alfaexploit.com>
2021-09-05 11:42:00,843 -    DEBUG: [1] New HEADER in message body: Subject -> test envio con milter
2021-09-05 11:42:00,843 -    DEBUG: [1] New HEADER in message body: To -> jjivarspoquet@gmail.com
2021-09-05 11:42:00,850 -    DEBUG: [1] New HEADER in message body: X-Spam-Status -> No, score=-1.0 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE
	autolearn=unavailable autolearn_force=no version=3.4.5
2021-09-05 11:42:00,851 -    DEBUG: [1] New HEADER in message body: X-Spam-Checker-Version -> SpamAssassin 3.4.5 (2021-03-20) on
	DrWho.alfaexploit.com
2021-09-05 11:42:00,851 -    DEBUG: [1] END of headers detected
2021-09-05 11:42:00,852 -    DEBUG: [1] BODY chunk: b'\r\n----=_RainLoop_336_866536040.1630834920\r\nContent-Type: text/plain; charset="utf-8"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n1234\r\n\r\n----=_RainLoop_336_866536040.1630834920\r\nContent-Type: text/html; charset="utf-8"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n<!DOCTYPE html><html><head><meta http-equiv=3D"Content-Type" content=3D"t=\r\next/html; charset=3Dutf-8" /></head><body><div data-html-editor-font-wrap=\r\nper=3D"true" style=3D"font-family: arial, sans-serif; font-size: 13px;"><=\r\nbr>1234<signature></signature></div></body></html>\r\n\r\n----=_RainLoop_336_866536040.1630834920--\r\n'
2021-09-05 11:42:00,852 -    DEBUG: [1] END OF MESSAGE BODY detected
2021-09-05 11:42:00,886 -    DEBUG: [1] CLOSE connection closed

Como véis los milters pueden servir para debugear tanto el envío como la recepción de emails, pero vamos a ir un paso mas allá y vamos a hacer que cuando enviemos un email con un asunto en concreto se realice una copia de dicho email a un destinatario.

Snooping mediante milter

Tendremos que modificar algo de código, lo primero será añadir algunos campos a la instancia del milter para poder guardar y comprobar dichos valores en las distintas funciones de callback:

	def __init__(self):  # A new instance with each new connection.
		self.id = Milter.uniqueID()	# Integer incremented with each call.
		self.additional_rcpts = []	# Additional rcpts when secret subject detected 
		self.source = ''			# Source variable to check if source is kr0m
		self.destination = ''		# Destination variable to check if destination is kr0m

Cuando analizamos el mail guardamos su origen y destino para poder consultar dicho valor en la función de callback header, de este modo logramos espiar todos los mails de entrada y salida del servidor con la excepción de los originados/destinados a kr0m@alfaexploit.com :

	def envfrom(self, mailfrom, *str):
		log.debug("[%d] MAIL FROM Received: %s" % (self.id, mailfrom))
		self.source = mailfrom
		return Milter.CONTINUE

	def envrcpt(self, to, *str):
		log.debug("[%d] RCPT TO Received: %s" % (self.id, to))
		self.destination = to
		return Milter.CONTINUE

Si el asunto lleva la palabra secret, añadimos un rcpt to al array additional_rcpts con una excepción, que self.source o self.destination sea kr0m@alfaexploit.com de este modo no espiamos dicha cuenta:

	def header(self, header_field, header_field_value):
		log.debug("[%d] New HEADER in message body: %s -> %s" % (self.id, header_field, header_field_value))
		if header_field == "Subject":
			log.debug("[%d] Checking secret regexp under Subject" % (self.id))
			pattern = '.*secret.*'
			secret_document = re.match(pattern, header_field_value, re.IGNORECASE)
			log.debug("[%d] Source: %s" % (self.id, self.source))
			log.debug("[%d] Destination: %s" % (self.id, self.destination))
			if secret_document:
				if self.source != '<kr0m@alfaexploit.com>' and self.destination != '<kr0m@alfaexploit.com>':
					log.debug("[%d] Secret document detected" % (self.id))
					self.additional_rcpts.append('kr0m@alfaexploit.com')
				
		return Milter.CONTINUE

Finalmente antes de dejar salir el email comprobamos si hay que añadir algún rcpt to adicional y actuamos en consecuencia:

	def eom(self):
		log.debug("[%d] END OF MESSAGE BODY detected" % (self.id))
		if len(self.additional_rcpts) > 0:
			for additional_rcpt in self.additional_rcpts:
				log.debug("[%d] Adding additional RCPT TO: %s" % (self.id, additional_rcpt))
				self.addrcpt(additional_rcpt)

		return Milter.CONTINUE

Cuando alguien envie un mail con asunto: ‘.secret.’(re.IGNORECASE), recibiremos una copia en kr0m@alfaexploit.com

Igualmente como hemos hecho esto podríamos realizar cualquier tipo de modificación sobre los mails, añadir/eliminar contenido/cabeceras o añalizar adjuntos en búsqueda de virus y eliminar dichos adjuntos.

Como apunte final destacar que los milters tanto en SendMail como en Postfix parecen ejecutarse de forma simultánea, por lo tanto no podemos encadenar milters que dependan de la ejecución de otro previo, en estos casos debemos programar un único milter que haga todas las operaciones que necesitemos.

Si te ha gustado el artículo puedes invitarme a un RedBull aquí