distro_tracker.mail

Email interface of Distro Tracker.

distro_tracker.mail.control

Module implementing the processing of email control messages.

distro_tracker.mail.control.send_response(original_message, message_text, recipient_email, cc=None)

Helper function which sends an email message in response to a control message.

Parameters

• original_message (email.message.Message or an object with an equivalent interface) – The received control message.

• message_text – The text which should be included in the body of the response.

• cc – A list of emails which should receive a CC of the response.

distro_tracker.mail.control.send_plain_text_warning(original_message, logdata)

Sends an email warning the user that the control message could not be decoded due to not being a text/plain message.

Parameters

original_message (email.message.Message or an object with an equivalent interface) – The received control message.

class distro_tracker.mail.control.ConfirmationSet

Bases: object

A class which keeps track of all confirmations which are required during a single control process run. This is necessary in order to send the emails asking for confirmation only when all commands are processed.

add_command(email, command_text, confirmation_message)

Adds a command to the list of all commands which need to be confirmed.

Parameters

• email – The email of the user the command references.

• command_text – The text of the command which needs to be confirmed.

• confirmation_message – An extra message to be included in the email when asking for confirmation of this command. This is usually an explanation of what the effect of the command is.

ask_confirmation_all()

Sends a confirmation mail to all users which have been registered by using add_command().

get_emails()

Returns

A unique list of emails which will receive a confirmation mail since there exists at least one command which references this user’s email.

distro_tracker.mail.control.process(msg)

The function which actually processes a received command email message.

Parameters

msg (email.message.Message) – The received command email message.

distro_tracker.mail.control.extract_command_from_subject(message)

Returns a command found in the subject of the email.

Parameters

message (email.message.Message or an object with an equivalent interface) – An email message.

distro_tracker.mail.dispatch

Implements the processing of received package messages in order to dispatch them to subscribers.

exception distro_tracker.mail.dispatch.SkipMessage

Bases: Exception

This exception can be raised by the vendor provided classify_message() to tell the dispatch code to skip processing this message being processed. The mail is then silently dropped.

distro_tracker.mail.dispatch.process(msg, package=None, keyword=None)

Dispatches received messages by identifying where they should be sent and then by forwarding them.

Parameters

• msg (email.message.Message) – The received message

• package (str) – The package to which the message was sent.

• keyword (str) – The keyword under which the message must be dispatched.

distro_tracker.mail.dispatch.forward(msg, package, keyword)

Forwards a received message to the various subscribers of the given package/keyword combination.

Parameters

• msg (email.message.Message) – The received message

• package (str) – The package name.

• keyword (str) – The keyword under which the message must be forwarded.

distro_tracker.mail.dispatch.process_for_team(msg, team_slug)

Dispatch a message sent to a team.

distro_tracker.mail.dispatch.forward_to_team(msg, team)

Forward a message to a team, adding headers as required.

distro_tracker.mail.dispatch.classify_message(msg, package=None, keyword=None)

Analyzes a message to identify what package it is about and what keyword is appropriate.

Parameters

• msg (email.message.Message) – The received message

• package (str) – The suggested package name.

• keyword (str) – The suggested keyword under which the message can be forwarded.

distro_tracker.mail.dispatch.approved_default(msg)

The function checks whether a message tagged with the default keyword should be approved, meaning that it gets forwarded to subscribers.

Parameters

msg (email.message.Message or an equivalent interface object) – The received package message

distro_tracker.mail.dispatch.add_new_headers(received_message, package_name=None, keyword=None, team=None)

The function adds new distro-tracker specific headers to the received message. This is used before forwarding the message to subscribers.

The headers added by this function are used regardless whether the message is forwarded due to direct package subscriptions or a team subscription.

Parameters

• received_message (email.message.Message or an equivalent interface object) – The received package message

• package_name (string) – The name of the package for which this message was intended.

• keyword (string) – The keyword with which the message should be tagged

distro_tracker.mail.dispatch.add_direct_subscription_headers(received_message, package_name, keyword)

The function adds headers to the received message which are specific for messages to be sent to users that are directly subscribed to the package.

distro_tracker.mail.dispatch.add_team_membership_headers(received_message, keyword, team)

The function adds headers to the received message which are specific for messages to be sent to users that are members of a team.

distro_tracker.mail.dispatch.send_to_teams(received_message, package_name, keyword)

Sends the given email message to all members of each team that has the given package.

The message is only sent to those users who have not muted the team and have the given keyword in teir set of keywords for the team membership.

Parameters

• received_message (email.message.Message or an equivalent interface object) – The modified received package message to be sent to the subscribers.

• package_name (string) – The name of the package for which this message was intended.

• keyword (string) – The keyword with which the message should be tagged

distro_tracker.mail.dispatch.send_to_team(received_message, team, keyword, package_name=None)

Send a message to a team.

distro_tracker.mail.dispatch.send_to_subscribers(received_message, package_name, keyword)

Sends the given email message to all subscribers of the package with the given name and those that accept messages tagged with the given keyword.

Parameters

• received_message (email.message.Message or an equivalent interface object) – The modified received package message to be sent to the subscribers.

• package_name (string) – The name of the package for which this message was intended.

• keyword (string) – The keyword with which the message should be tagged

distro_tracker.mail.dispatch.send_messages(messages_to_send, date)

Sends all the given email messages over a single SMTP connection.

distro_tracker.mail.dispatch.prepare_message(received_message, to_email, date)

Converts a message which is to be sent to a subscriber to a CustomEmailMessage so that it can be sent out using Django’s API. It also sets the required evelope-to value in order to track the bounce for the message.

Parameters

• received_message (email.message.Message or an equivalent interface object) – The modified received package message to be sent to the subscribers.

• to_email (string) – The email of the subscriber to whom the message is to be sent

• date (datetime.datetime) – The date which should be used as the message’s sent date.

distro_tracker.mail.dispatch.bounce_is_for_spam(message)

Return True if the bounce has been generated by spam, False otherwise.

distro_tracker.mail.dispatch.handle_bounces(sent_to_address, message)

Handles a received bounce message.

Parameters

sent_to_address (string) – The envelope-to (return path) address to which the bounced email was returned.

distro_tracker.mail.mail_news

Module implementing the processing of received emails which could be turned into news items.

distro_tracker.mail.mail_news.create_news(message, package, create_package=False, **kwargs)

Create a news item from the given message.

The created news parameters are:

• title - the Subject of the message

• content - the message content itself

• content_type - message/rfc822

Parameters

• message (email.message.Message) – A message which should be turned into a news item.

• package (distro_tracker.core.models.PackageName or a string.) – The package for which this news item should be created.

Returns

The created news item

Return type

distro_tracker.core.models.News

distro_tracker.mail.models

Defines models specific for the distro_tracker.mail app.

class distro_tracker.mail.models.CommandConfirmationManager(*args, **kwargs)

Bases: distro_tracker.core.models.ConfirmationManager

A custom manager for the CommandConfirmation model.

create_for_commands(commands)

Creates a CommandConfirmation object for the given commands.

Parameters

commands – An iterable of commands for which a confirmation is requested.

Raises

distro_tracker.mail.models.CommandConfirmationException – If it is unable to generate a unique key.

class distro_tracker.mail.models.CommandConfirmation(*args, **kwargs)

Bases: distro_tracker.core.models.Confirmation

A model representing pending confirmations for email interface commands.

commands

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <distro_tracker.mail.models.CommandConfirmationManager object>

property command_list

Returns

A list of strings representing commands which are confirmed by this instance.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

get_next_by_date_created(*, field=<django.db.models.fields.DateTimeField: date_created>, is_next=True, **kwargs)

get_previous_by_date_created(*, field=<django.db.models.fields.DateTimeField: date_created>, is_next=False, **kwargs)

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class distro_tracker.mail.models.UserEmailBounceStatsManager(*args, **kwargs)

Bases: django_email_accounts.models.UserEmailManager

A custom Manager for the UserEmailBounceStats model.

get_bounce_stats(email, date)

Gets the UserEmailBounceStats instance for the given UserEmail on the given date

Parameters

• email (string) – The email of the UserEmail

• date (datetime.datetime) – The date of the required stats

add_bounce_for_user(email, date)

Registers a bounced email for a given UserEmail

Parameters

• email (string) – The email of the UserEmail for which a bounce will be logged

• date (datetime.datetime) – The date of the bounce

add_sent_for_user(email, date)

Registers a sent email for a given UserEmail

Parameters

• email (string) – The email of the UserEmail for which a sent email will be logged

• date (datetime.datetime) – The date of the sent email

limit_bounce_information(email)

Makes sure not to keep more records than the number of days set by DISTRO_TRACKER_MAX_DAYS_TOLERATE_BOUNCE

class distro_tracker.mail.models.UserEmailBounceStats(*args, **kwargs)

Bases: django_email_accounts.models.UserEmail

A proxy model for the UserEmail model. It is defined in order to implement additional bounce stats-related methods without needlessly adding them to the public interface of UserEmail when only the distro_tracker.mail.dispatch app should use them.

objects = <distro_tracker.mail.models.UserEmailBounceStatsManager object>

has_too_many_bounces()

Checks if the user has too many bounces.

exception DoesNotExist

Bases: django_email_accounts.models.UserEmail.DoesNotExist

exception MultipleObjectsReturned

Bases: django_email_accounts.models.UserEmail.MultipleObjectsReturned

class distro_tracker.mail.models.BounceStats(*args, **kwargs)

Bases: django.db.models.base.Model

A model representing a user’s bounce statistics.

It stores the number of sent and bounced mails for a particular date.

user_email

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

mails_sent

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

mails_bounced

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

date

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

get_next_by_date(*, field=<django.db.models.fields.DateField: date>, is_next=True, **kwargs)

get_previous_by_date(*, field=<django.db.models.fields.DateField: date>, is_next=False, **kwargs)

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>

user_email_id

distro_tracker.mail.processor

Module implementing the processing of incoming email messages.

exception distro_tracker.mail.processor.MailProcessorException

Bases: Exception

exception distro_tracker.mail.processor.ConflictingDeliveryAddresses

Bases: distro_tracker.mail.processor.MailProcessorException

The message contained multiple headers with possible delivery addresses for the domain defined in settings.DISTRO_TRACKER_FQDN.

exception distro_tracker.mail.processor.MissingDeliveryAddress

Bases: distro_tracker.mail.processor.MailProcessorException

The message contained no header with a delivery address for the domain defined in settings.DISTRO_TRACKER_FQDN.

exception distro_tracker.mail.processor.InvalidDeliveryAddress

Bases: distro_tracker.mail.processor.MailProcessorException

The message contained a delivery address for the domain defined in settings.DISTRO_TRACKER_FQDN but it did not match any known Distro Tracker service.

class distro_tracker.mail.processor.MailProcessor(message_or_filename)

Bases: object

Takes an incoming email and do something useful out of it.

To this end, it must find out where the email was sent and adjust the processing depending on the role of the target address.

load_mail_from_file(filename)

Load the mail to process from a file.

Parameters

filename (str) – Path of the file to parse as mail.

static find_delivery_address(message)

Identify the email address the message was delivered to.

The message headers Delivered-To, Envelope-To, X-Original-To, and X-Envelope-To are scanned to find out an email that matches the FQDN of the Distro Tracker setup.

static identify_service(address)

Identify service associated to target email and extract optional args.

The address has the generic form <service>+<details>@<fqdn>.

static do_nothing(self)

Just used by unit tests to disable process()

process()

Process the message stored in self.message.

Find out the delivery address and identify the associated service. Then defer to handle_*() for service-specific processing. Can raise MissingDeliveryAddress and UnknownService

static build_delivery_address(service, details)

handle_control()

handle_bounces(details)

handle_dispatch(package=None, keyword=None)

handle_team(team)

distro_tracker.mail.processor.run_mail_processor(mail_path, log_failure=False)

Run a MailProcessor on a stored email.

Parameters

• mail_path (str) – path of the email

• log_failure (bool) – indicates whether to log any failure

class distro_tracker.mail.processor.MailQueue

Bases: object

A queue of mails to process. The mails are identified by their filename within DISTRO_TRACKER_MAILDIR_DIRECTORY.

MAX_WORKERS = 4

The maximum number of sub-process used to process the mail queue

SLEEP_TIMEOUT_EMPTY = 30.0

SLEEP_TIMEOUT_TASK_RUNNING = 0.01

SLEEP_TIMEOUT_TASK_FINISHED = 0.0

SLEEP_TIMEOUT_TASK_RUNNABLE = 0.0

add(identifier)

Add a new mail in the queue.

Parameters

identifiername (str) – Filename identifying the mail.

remove(identifier)

Remove a mail from the queue. This does not unlink the file.

Parameters

identifier (str) – Filename identifying the mail.

initialize()

Scan the Maildir and fill the queue with the mails in it.

property pool

close_pool()

Wait until all worker processes are finished and destroy the pool

process_queue()

Iterate over messages in the queue and do whateever is appropriate.

sleep_timeout()

Return the maximum delay we can sleep before we process the queue again.

process_loop(stop_after=None, ready_cb=None)

Process all messages as they are delivered. Also processes pre-existing messages. This method never returns.

Parameters

• stop_after (int) – Stop the loop after having processed the given number of messages. Used mainly by unit tests.

• ready_cb – a callback executed after setup of filesystem monitoring and initial scan of the mail queue, but before the start of the loop.

class distro_tracker.mail.processor.MailQueueEntry(queue, identifier)

Bases: object

An entry in a :py:class:MailQueue.

Contains the following public attributes:

The parent :py:class:MailQueue.

The entry identifier, it’s the name of the file within the directory of the MailQueue. Used to uniquely identify the entry in the MailQueue.

The full path to the mail file.

set_data(key, value)

get_data(key)

move_to_subfolder(folder)

Move an entry from the mailqueue to the given subfolder.

start_processing_task()

Create a MailProcessor and schedule its execution in the worker pool.

processing_task_started()

Returns True when the entry has been fed to workers doing mail processing. Returns False otherwise.

Returns

an indication whether the mail processing is on-going.

Return type

bool

processing_task_finished()

Returns True when the worker processing the mail has finished its work. Returns False otherwise, notably when the entry has not been fed to any worker yet.

Returns

an indication whether the mail processing has finished.

Return type

bool

handle_processing_task_result()

Called with mails that have been pushed to workers but that are still in the queue. The task likely failed and we need to handle the failure smartly.

Mails whose task raised an exception derived from :py:class:MailProcessorException are directly moved to a “broken” subfolder and the corresponding entry is dropped from the queue.

Mails whose task raised other exceptions are kept around for multiple retries and after some time they are moved to a “failed” subfolder and the corresponding entry is dropped from the queue.

schedule_next_try()

When the mail processing failed, schedule a new try for later. Progressively increase the delay between two tries. After 5 tries, refuse to schedule a new try and return False.

Returns

True if a new try has been scheduled, False otherwise.

class distro_tracker.mail.processor.MailQueueWatcher(queue)

Bases: object

Watch a mail queue and add entries as they appear on the filesystem

class EventHandler(pevent=None, **kargs)

Bases: pyinotify.ProcessEvent

my_init(queue=None)

This method is called from ProcessEvent.__init__(). This method is empty here and must be redefined to be useful. In effect, if you need to specifically initialize your subclass’ instance then you just have to override this method in your subclass. Then all the keyworded arguments passed to ProcessEvent.__init__() will be transmitted as parameters to this method. Beware you MUST pass keyword arguments though.

@param kargs: optional delegated arguments from __init__(). @type kargs: dict

process_IN_CREATE(event)

process_IN_MOVED_TO(event)

start()

Start watching the directory of the mail queue.

process_events(timeout=0, count=1)

Process all pending events since last call of the function.

Parameters

• timeout (float) – Maximum time to wait for an event to happen.

• count (int) – Number of processing loops to do.