Troubleshooting Exchange’s Built-In Anti-Spam Technologies: Pt. 9 Anti-Spam Agent Log OutputWritten by Casper Manes on October 3, 2012
Welcome back to our series on troubleshooting Exchange’s built-in anti-spam technologies. In this post, we are going to take a look at managing the log files our anti-spam agents create. We’ll see where those are located, and what options we have for tweaking them.
Remember that our anti-spam agents (typically) run on the Edge Transport server. Their execution and behavior is controlled by the edgetransport.exe process. Note I said typically in the opening sentence of this paragraph. You can run many of the anti-spam agents on a Hub Transport server if you want to. When you run the Install-AntispamAgents.ps1 script on a Hub Transport server, you not only get the anti-spam agents, you get the EdgeTransport.exe service. And that is what creates our logs.
By default, the EdgeTransport.exe service will log all anti-spam agent activity to the %programfiles%MicrosoftExchange ServerV14TransportRolesLogsAgentLog directory. Unless you tweak the settings, here’s what you get in that directory.
- No more than 250 MB of logs
- No single log file larger than 10 MB (so on a busy day, you will have lots of logs for the same day!)
- No more than 30 days’ worth of logs.
If you want more you are going to need to modify the EdgeTransport.exe.config file. I think 30 days worth of logs is more than enough, but on a really busy server that could be more than 250 MB worth. Also, while I prefer to either have one log file per day, or one per hour, but not several of exactly the same size covering different timeframes, unfortunately we can only specify size, not timespan. So on a production server, here’s what we can do to modify the logging behaviour.
- Find %programfiles%MicrosoftExchange ServerV14BinEdgeTransport.exe.config.
- Make a copy of it, you know, just in case.
- Open the file in your favourite text editor. It’s just an XML file.
- Find the <appSettings> section. There will be a ton of <add key=blah blah blah/> entries here.
- Edit to suit your needs. See below for the specifics.
- Save the file.
- Restart the EdgeTransport.exe service.
What you can add to the <appSettings> section is defined in the following table. The only value that is present by default and relevant to anti-spam agents is “AgentLogEnabled.” Note this is a boolean, and it is set to true, as in <add key=”AgentLogEnabled” value=”true” />.
|AgentLogEnabled||System.Boolean||Valid values for this key are true or false. The default value is true.|
|AgentLogMaxDirectorySize||System.Int32||The value of this key specifies the maximum size, in bytes, of the AgentLog directory. When this value is exceeded, the oldest log file in the directory is deleted and a new log file is created. If this key isn’t specified, the default value is 250 MB, or 262144000 bytes, which is determined as follows: 250×1,024×1,024.|
|AgentLogMaxFileSize||System.Int32||The value of this key specifies the maximum size, in bytes, of each log file in the directory. When a log file reaches the size specified, a new log file is created.If this key isn’t specified, the default is 10 MB, or 10485760 bytes, which is determined as follows: 10×1024×1024.|
|AgentLogMaxAge||System.TimeSpan||The value of this key specifies the maximum age limit of a specified log file. When a log file exceeds the age limit, it’s deleted.This key is of system type TimeSpan. The value of this key can be represented as a string in the format d.hh:mm:ss.ff where d is days, hh is hours, mm is minutes, ss is seconds, and ff is fractions of a second.If this key isn’t specified, the default value is 30 days, or 30.00:00:00.00|
A practical example
Suppose you wanted to keep only a maximum fourteen days’ worth of logs, you wanted to allocate no more than 1GB to the log files, and you never wanted to deal a single file larger than 5MB. Here is what you would add just under the AgentLogEnabled line.
<add key=”AgentLogMaxDirectorySize” value=”1073741824″ />
<add key=”AgentLogMaxFileSize” value=”5242880″ />
<add key=”AgentLogMaxAge” value=”14.00:00:00.00″ />
The only thing worse than no logging is too much logging, and a 10MB file is a lot of lines to parse. Setting a smaller value means that your favourite text editor can open a file more quickly, and it will take longer for you to go cross-eyed while reviewing the logs.
Logs are recorded as *.log files, but they are really just CSV files. You may find it easier to deal with if you open them in Excel, but any good text editor will do. You may also want to check out the free tool from Microsoft, LogParser. This is a great tool for crunching through logs from all sorts of sources, though it is command line based.
Here’s two examples; one of an accepted message and one of an entry that was rejected. First, here’s the headers of the log file.
Timestamp,SessionId,LocalEndpoint,RemoteEndpoint,EnteredOrgFromIP,MessageId,P1FromAddress, P2FromAddresses,Recipient,NumRecipients,Agent,Event,Action,SmtpResponse,Reason,ReasonData, Diagnostics
Now, here is a message that was rejected because the source address does not exist. The Sender ID agent rejected this message.
2012-09-21T03:04:00.474Z,08CF5F9CA50FE8BD,192.168.0.6:25,18.104.22.168:32925, 22.214.171.124,,email@example.com,"""Home Energy Saver"" firstname.lastname@example.org;",email@example.com,1,Sender Id Agent, OnEndOfHeaders,RejectMessage,550 5.7.1 Missing purported responsible address,MissingPRA,No valid PRA
Here’s another one, where the Content Filter agent rejected it as spam.
2012-09-21T12:06:39.275Z ,08CF5F9CA51026B0,192.168.0.6:25,126.96.36.199:41366,188.8.131.52, firstname.lastname@example.org, InstantCheckmateBackgroundChecks@longbasketball.com, InstantCheckmateBackgroundChecks@longbasketball.com;email@example.com,1, Content Filter Agent OnEndOfData RejectMessage "550 5.7.1 Your message smells like spam. No 250 for you. Come back, one year!" SclAtOrAboveRejectThreshold 7 DV:3.3.11717.454;SID:SenderIDStatus Neutral
As you can see, if you are troubleshooting anti-spam agents and you are parsing the logs, if you have the approximate time the message was sent and at least on of the following: the sender ip.addr, the sender email address, or the recipient email address, you can quickly find the entry in the log file that explains why a message was accepted or rejected and the agent responsible. Then you can use one of our earlier posts to help troubleshoot and tweak the specific agent involved.