Troubleshooting Exchange’s Built-In Anti-Spam Technologies: Pt. 3 Content FilteringWritten by Casper Manes on August 14, 2012
One of the anti-spam technologies built-in to the Edge Transport role in Exchange 2010 is Content Filtering. Content Filtering scans all inbound messages and uses a variety of inputs to assign a Spam Confidence Level (SCL) to each message. Depending upon the inputs, Content Filtering may also choose to either immediately block or immediately pass the message. Content filtering is not magic-it operates primary using lists, and in its default configuration, applies a one size fits all approach to whether to block or permit messages in to users.
We covered how Content Filtering works in this post which you may want to review before proceeding.
If you are going to use the content filtering built into Exchange 2010, there are some important things to know about how it operates, which may save you significant time troubleshooting issues.
- Content filtering does not even attempt to scan messages over 11 MB. Large attachments are going to pass right through, unless of course they exceed your maximum allowed message size.
- Messages sent to distribution lists do not take individual user Spam Confidence Level settings into account. If you have a user that you have configured a particularly low SCL for, but that user is a part of a distribution list that can receive emails from outside the organization, that user may find themselves receiving messages with a higher score than they want.
- Content filtering only scans inbound mail. If you want to use content filtering to scan outbound mail, run this command in the Exchange Management Shell. Set-ContentFilterConfig -InternalMailEnabled $true
- If a word or phrase on the allowed list is contained in the message, it will be passed no matter how spammy it is. Choose the words you add to the whitelist carefully.
- Content filtering logging (and all other anti-spam agent logging) is off by default. If you are trying to determine why Exchange is bouncing a message, see below.
Enabling Agent Logging
If you are troubleshooting bounced mail, make sure that your system is really the one bouncing the mail. Get a copy of the NDR from the sender and look for the 5.7.1 to make sure it was bounced by your server. If it was, then you want to look at the agent logs. This may be turned off on you server, so if C:Program FilesMicrosoftExchange ServerV14TransportRolesLogsAgentLog is empty, enable logging as follows.
- Open an administrative command prompt.
- Execute this command, adjusting the path to account for your install location. notepad “C:Program FilesMicrosoftExchange ServerBinEdgeTransport.exe.config”
- Find the <appSettings> section and change the AgentLogEnabled to true, like this <add key=”AgentLogEnabled” value=”TRUE” />
- Save the file.
- Restart the Microsoft Exchange Transport service.
Reviewing Agent Logging
The agent logs are CSV files stored in C:Program FilesMicrosoftExchange ServerV14TransportRolesLogsAgentLog. The work with them, open the CSV files in Excel. You will see columns for Timestamp, sender and receiver, SMTP response, and more. The diagnostics in the last column tells the full story. Here’s a table from this Microsoft page that tells you the full story on the diagnostics data.
|SID||The Sender ID (SID) stamp is based on the sender policy framework (SPF) that authorizes the use of domains in e-mail. The SPF is displayed in the message envelope as Received-SPF. The Sender ID evaluation process generates a Sender ID status for the message. This status can be returned as one of the following values:
The Sender ID stamp is displayed as an X-Header in the message envelope as follows:
For more information about Sender ID, see Understanding Sender ID.
|DV||The DAT version (DV) stamp indicates the version of the spam definition file that was used when scanning the message.|
|SA||The signature action (SA) stamp indicates that the message was either recovered or deleted because of a signature that was found in the message.|
|SV||The signature DAT version (SV) stamp indicates the version of the signature file that was used when scanning the message.|
|PCL||The phishing confidence level (PCL) stamp displays the rating of the message based on its content and is applied when the message is processed by the Content Filter agent. This status can be returned as one of the following values:
The PCL value can range from 1 through 8. A PCL rating from 1 through 3 returns a status of Neutral. This means that the message’s content isn’t likely to be phishing. A PCL rating from 4 through 8 returns a status of Suspicious. This means that the message is likely to be phishing.
The values are used to determine what action Outlook takes on messages. Outlook uses the PCL stamp to block the content of suspicious messages.
The PCL stamp is displayed as an X-header in the message envelope as follows:
|SCL||The spam confidence level (SCL) stamp of the message displays the rating of the message based on its content. The Content Filter agent uses Microsoft SmartScreen technology to assess the contents of a message and to assign an SCL rating to each message. The SCL value is from 0 through 9, where 0 is considered less likely to be spam, and 9 is considered more likely to be spam. The actions that Exchange and Outlook take depend on your SCL threshold settings.The SCL stamp is displayed as an X-header in the message envelope as follows:CopyX-MS-Exchange-Organization-SCL:<status>For more information about SCL thresholds and actions, see Understanding Spam Confidence Level Threshold.|
|CW||The custom weight (CW) stamp of a message indicates that the message contains an unapproved word or phrase and that the SCL value, or weight, of that unapproved word or phrase was applied to the final SCL score:
For more information about how to add approved and unapproved words or phrases to the Content Filtering agent, see Configure Content Filtering Properties.
|PP||The presolved puzzle (PP) stamp indicates that if a sender’s message contains a valid, solved computational postmark, based on Outlook E-mail Postmark validation functionality, it’s unlikely that the sender is a malicious sender. In this case, the Content Filter agent would reduce the SCL rating.The Content Filter agent doesn’t change the SCL rating if the E-mail Postmark validation feature is enabled and either of the following conditions is true:
For more information about the postmark validation feature, see Configure Content Filtering Properties.
|TIME:TimeBasedFeatures||The TIME stamp indicates that there was a significant time delay between the time that the message was sent and the time that the message was received. The TIME stamp is used to determine the final SCL rating for the message.|
|MIME:MIMECompliance||The MIME stamp indicates that the e-mail message isn’t MIME compliant.|
|P100:PhishingBlock||The P100 stamp indicates that the message contains a URL that’s present in a phishing definition file.|
|IPOnAllowList||The IPOnAllowList stamp indicates that the sender’s IP address is on the IP Allow list. For more information about the IP Allow list, see Understanding Connection Filtering.|
|MessageSecurityAntispamBypass||The MessageSecurityAntispamBypass stamp indicates that the message wasn’t filtered for content and that the sender has been granted permission to bypass the anti-spam filters.|
|SenderBypassed||The SenderBypassed stamp indicates that the Content Filter agent doesn’t process any content filtering for messages that are received from this sender. For more information, see Configure Content Filtering Properties.|
|AllRecipientsBypassed||The AllRecipientsBypassed stamp indicates that one of the following conditions was met for all recipients listed in the message:
You can also check the configuration of your content filtering with the command
get-contentfilterconfig | fl
This will give you a list of the basic settings so you can see if someone is on the bypassed or blocked senders, as well as organization-wide SCL settings.
Remember that you can always add a user to the “bypassed senders” list to ensure critical business communications are not blocked. You can add multiple entries using a comma separated list.
Once you have determined that it is content filtering that is blocking a message, you can focus on the configuration of the filtering, and if it’s not there, the key word lists to determine what is wrong.