E-Mail Transfer in Clusters

Incoming SMTP connections are received with a TCP Load Balancer and sent to Cluster Frontends. A Frontend Server receives a message in the same way as it does in the single-server mode, but it may contact the Backend Servers (via CLI) when it needs to:

• authenticate the sender
• verify the sender and recipient addresses
• check the recipient status
• retrieve the sender limits

Received messages are enqueued. If a message is directed to an external address, it can be relayed by the same Frontend:

A message directed to a local recipient can be enqueued on a "wrong" Server, i.e. on a Server that cannot open the target Account and deliver the message to that Account.

This situation can happen when a message is enqueued on a Frontend Server (Frontend Servers cannot directly open any Account in Shared Domains), or a message is enqueued on a Backend Server that either does not host the target Account (in a Static Cluster), or it cannot open it, because the Account is opened by some other Backend Server (in a Dynamic Cluster).

To solve the problem, the Local Delivery module uses a Delivery channel connection to the proper Backend Server and passes the message there. The receiving Backend immediately opens the target Account, applies its Account-Level Rule and stores the transferred message. This Backend does not enqueue the message.

If there is a temporary problem or a delivery failure, the receiving Backend Server reports the error back, and the message is either delayed in Queue or it is removed from the Queue (with error report messages generated).

WebUser Interface, XIMSS, MAPI sessions, Rules, and other modules and components can generated E-mail messages on Backend Servers.

Backend Server often do not have direct access to the Internet, in this case they cannot deliver generated messages to remote systems. To solve this problem, the Backend Servers can be configured to relay all messages to the Frontend Servers first, using the * symbol as the SMTP Forwarding Server name.

In this case the message is submitted to the Backend Queue, where it is processed using the Server-wide and Cluster-wide Rules, and if it is not directed to a local recipient, it is directed to the SMTP module which sends it to one of the Frontend Servers:

In this setup, each messages generated on a Backend Server is processed twice. If Cluster Rules invoke content management Plugins, double-processing can create a substantial overhead.

To avoid these problems and the inevitable overhead, the Remote Queue Processing method should be used.

Most of the Queue processing takes place on the Frontend Servers. Frontend Servers accept incoming SMTP E-mail Messages and either relay them to remote locations or deliver them to local Accounts via Backends, using the special inter-cluster protocol, without placing messages into the Backend Server Queue.

A certain amount of E-mail Messages can be generated directly on the Backend Servers.

They include messages:

• composed with WebUser, XIMSS, AirSync, CalDAV sessions;
• submitted using the MAPI connector and XTND XMIT POP3 mechanism;
• created with Account-level Rules;
• retrieved using the RPOP module;
• submitted using the PIPE module.

You may want to avoid processing Message Queues on Backends for various reasons, including:

• lack of Internet connectivity for Backends;
• requirement to use anti-spam and/or anti-virus Plugins installed on Frontends only;
• shortage of processing power and/or disk space on Backend Servers.

You may also want to process Message Queues on some Frontends only.

To specify Queue processing options, open the Settings WebAdmin realm and select the Cluster link on the General page. Find the Queue Processing panel:

Queue Processing

Submit Messages:

AutoRemotelyLocallyLocally for Others

Remote Submit Log:

Crashes OnlyFailuresMajor & FailuresProblemsLow LevelAll Info

Submit Messages

This setting specifies how the composed or received E-mail messages are submitted to the Enqueuer component for delivery.

Locally

messages are submitted to the Enqueuer component on the same Server (this is the "regular", single-server processing mode).

Locally for Others

messages are submitted to the Enqueuer component on the same Server.
The Dynamic Cluster Controller is informed that this Server can accept (enqueue) E-mail messages composed or received with the other Cluster members.
The Dynamic Cluster Controller collects and distributes information about all active Cluster members that have this option selected.

Remotely

messages are transferred to some Cluster member that has this setting set to Locally for Others. The temporary message file content (the message envelope and the message itself) is sent to that other Cluster member via a special protocol that uses the SMTP port. If the remote message submission does not work (the Server failed to connect to the Cluster member(s) or the message file transfer fails), the message is submitted to the Server own Queue to ensure that no message is lost:

Auto

• if this Server is not a Dynamic Cluster member, same as Locally
• if this Server is a Dynamic Cluster frontend, same as Locally for Others
• if this Server is a Dynamic Cluster backend, same as Remotely if there are other Dynamic Cluster members configured as Locally for Others, if there are none - same as Locally

Remote Submit Log

Use this setting to specify what kind of information is stored in the Server Log when a message is being submitted remotely (to a different Server).
These records are marked with the SUBMIT tag.