IMPORTANT: This document is meant for SYSADMIN (org admins or channel admins).
The spam filtering engines used in all filtering solutions aren't perfect. All spam filtering vendors including Vircom (modusGate) and Proofpoint Essentials use a "kitchen sink" approach to spam filtering. Namely, we use a variety of means to determine if a message is good or not.
1) IP Reputation: We look at where the email came from. If the IP Address the Email came from has a bad reputation for instance, there's a much higher chance that the message will go to quarantine and in some cases, be outright rejected at the front door (ie: blocked by a 550 error, your email is not wanted here). Reputation is determined by networks of machines deployed internally by us (spamtraps & honeypots) and third parties (ex: CloudMark, spamhaus, many others ...). If those honeypots get hit by spam, the IP is recorded and the more hits from the same IP, the worse is the reputation. Reputation systems also have aging mechanims whereas if there have been no hits for a certain amount of time, the reputation slowly drifts back towards a "neutral" state.
2) Bad Practices: We look at obvious bad practices used by certain senders. Not having declared a reverse DNS record (PTR record) for the IP they are sending mail from for instance. Or if the PTR record doesn't match what's in the EHLO/HELO statement.
3) Artificial Intelligence engines: We use various Artificial Intelligence engines to look at the content of the Email for "spamminess". Usually these AI engines are trained by providing them a large corpus of "known good" and "known bad" emails, and this forms an information "cloud" whereas new messages are ranked by how close to "goodness" or "badness" they are. They have fancy names like "bayesian filtering" or "support vector machines" but in all cases, these engines need constant feeding of new samples to maintain accuracy. That's why Proofpoint (and Vircom) operate honeypots or spamtraps to get these samples to keep training the engines.
4) Heuristic Rules: Other Heuristic approaches are used. Word-matching, pattern-matching and obvious obfuscation attempts are accounted for and detected.
5) Bad/No Authentication: If a domain doesn't provide any authentication methods (SPF, DKIM, DMARC), that also has an influence on the spam score. Domains that provide no verification at all usually have a harder time insuring deliverability.
So when you add all these things together, false-positives happen.
What can I do to reduce false-positives?
Well - the answer is "it depends"
Our experience with FPs shows that most FPs come from badly configured sending MTAs (mail transfer agents or mail servers). Since often these are External senders trying to mail YOU, there's not that many things you can do to prevent them other than encouraging the senders to adopt better policies or fix their broken policies. For instance, if a sender is sending Emails signed with a DKIM key but their email afterwards transits through a custom signature tool that adds a standardized signature at the bottom of each Email AFTER the message was signed internally with DKIM, then all the emails they will be sending out will be marked as DKIM Failed.
So really, the only option for you is to add the sender's Email address to your trusted senders list.
- End users can release the message and add the message to their trusted senders / allowed list. This can be done directly from the Quarantine digest by "Releasing and Approving"
- You and your end users can do the same thing from the message log.
Don't forget, we also have our SpamReporter tool that you can deploy in Outlook and Office365.
Nonetheless, there are some cases where things ARE under your control
INCOMING FALSE-POSITIVES
Here are some cases we see daily that clients contact us about fixing.
1) Web Forms submitted from a website that the client owns are getting caught inbound in quarantine.
Most of our clients operate websites that send mail back to their employees with a FROM: address matching their domain.
Example:
Company widget.com has an information request form on their website @ www.widget.com.
Those forms have a from: address of "info@widget.com" and is sent to internal employees @widget.com.
All those emails wind up in quarantine.
Reason:
Client widget.com omitted to put the IP Address of the web server in proofpoint's DOMAIN settings under "Sending Servers". When you put an IP there, it tells proofpoint that this IP is a legit IP that is allowed to send mail on my company's behalf. So adding the IP there would fix the FP issues.
IMPORTANT: If you do not do any outgoing filtering, you might want to add the IP address in your global Allowed Sender list or create a filter rule to allow it. The filter rules kick before the Allowed Sender List
if header contains (ip of the server) and sender address is info@widget.com then allow.Now in some cases, it's possible that the webhoster uses a cloud-based mail deliver system so the IP addresses change all the time. In those cases, because the address changes constantly, it's better to use a custom filter. For instance, if we examine the header of one of these FPs, we might see something like this:
Received: from webhoster.someformservice.com ([X.X.X.X] ident=nobody)
by us5.ppe-hosted.com with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <info@widget.com>) id 1j5DCt-0007yp-9f
for jimbob@localdomain.com; Fri, 21 Feb 2020 18:39:24 +0000Since the IP X.X.X.X can change, it's easier to make a rule that looks for "webhoster.somesformservice.com"
Basically the logic of the rule would be:
if sender address is info@widget.com and
header contains "webhoster.someformservice.com" then
allow
This is what the rule would need to look like in Proofpoint Essentials:
2) Inbound Emails from marketing efforts using services like MailChimp, Constant contact, etc ...
This problem is similar to the web form issue whereas the sender is using a cloud-service to send mail from the website to the local domain. This is exacerbated by the Antispoofing measure in proofpoint. Basically Proofpoint's ANTISPOOFING measure shown below is very aggressive. It will tag anything with FROM: yourdomain.com in the from field that isn't coming from an authorized IP as a spoof. So if the IP is not listed under Domains or is not an IP the actual domain is configured to deliver mail to, it'll be tagged as a spoofing message.
This is irregardless if the you have proper SPF setup from MailChimp, Constant Contact, Salesforce or whatever other cloud service you may use that sends mail on your behalf.
So the obvious question is -- shouldn't I turn off this feature? Answer is a resounding NO. The number of newsletter / external services you use is finite. You just need to determine what they are and make a rule similar as in issue #1 above for each of them that is winding up in quarantine.
It's better to simply create a rule. For instance, in the received headers of messages coming from Constant Contact, you will often found something like "ccsend.constantcontact.com" or similar entry. So you just make a constant contact rule.
3) Inbound Email that is coming FROM your domain to your domain (this applies if you're using Exclaimer with Office365)
Normally, when two people Email each other on the same tenant on office365, the Email should never leave Office365. However there is a case whereas, if a client uses the Exclaimer tool (Exclaimer is a professional Signature Management system), that tool breaks this internal mail flow ... the Emails are sent out to the internet back to the MX record so the emails are coming INBOUND instead of staying on the tenant. This has on occasion created false positives. Normally, you shouldn't even see in the message log inter-user emails within the same org if they are in Office365.
Basically, to counter this you need to create a filter rule that allows anything FROM your local domain(s) inbound if it comes from Office365. Since Office365 has a huge number of IP addresses, it's better to look for typical information found in the header of Emails typically sent FROM office365.
So we can build around along certain tags in the header.
Example (snip of a much bigger header):
X-Virus-Scanned: Proofpoint Essentials engine
Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2049.outbound.protection.outlook.com [104.47.66.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 1A73BB4005F for <yveslacombe@gmail.com>; Mon, 24 Feb 2020 16:21:33 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tripoli-quebec.org; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0pZ3/u+EmyxX+oS/9SsHgYcDoetxYInE4nijBFrTDVk=; b=ZFdGsE1LyPnezzsmF9twxBNL2KAZTadmoiKGv2at2PBKfaHvm7c8jiKdm8ya6LjMKW6GATIPt0Xi4+37bvpRyfCClfHkcBvXuNN8PcaTK9STNp+/tNRcRURUyTxN3+5EAz50+O/X9AIxyFL++G0bcRUHBda1tuDKRerNshQnrUM=
Received: from SN6PR05MB4415.namprd05.prod.outlook.com (2603:10b6:805:3a::13) by SN6PR05MB4736.namprd05.prod.outlook.com (2603:10b6:805:92::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2772.11; Mon, 24 Feb 2020 16:21:30 +0000
Received: from SN6PR05MB4415.namprd05.prod.outlook.com ([fe80::a455:2f63:bad2:334a]) by SN6PR05MB4415.namprd05.prod.outlook.com ([fe80::a455:2f63:bad2:334a%6]) with mapi id 15.20.2772.009; Mon, 24 Feb 2020 16:21:30 +0000
From: Yves Lacombe <yves@tripoli-quebec.org>
To: "yveslacombe@gmail.com" <yveslacombe@gmail.com>
Subject: test 123 test
Thread-Topic: test 123 test
Thread-Index: AQHV6y546S5KWeCbXEeBcQseGnkMTw==
Date: Mon, 24 Feb 2020 16:21:29 +0000
Message-ID: <SN6PR05MB4415E1FBF7C85337DAE1A403EAEC0@SN6PR05MB4415.namprd05.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=yves@tripoli-quebec.org;
So in the example above. we'd allow anything FROM *@tripoli-quebec.org if in the header we see prod.outlook.com and outbound.protection.outlook.com. We obviously don't want to do a blanket allow anything from my domain due to spoofing.
OUTGOING FALSE-POSITIVES
Outgoing FPs are generally caused by the AI portion of our antispam engines that is misclassifying the Email incorrectly. In those cases, it's better to do the following steps:
1) Report the FP through the interface the Proofpoint Essentials interface.
See this article here: https://vircomhelp.freshdesk.com/support/solutions/articles/48000960925-reporting-spam-and-false-positives-to-proofpoint
IMPORTANT: when you submit an FP using the interface, Vircom will NOT see those emails. Proofpoint does not provide feedback on those submissions either.
2) If you want Vircom to have a look at them to see if something can be done pre-emptively to prevent them you should email support@vircom.com with the following information:
a) The permalink of the Email submitted (example):
b) if possible, once you released the email, give us a copy of the original Email. DO NOT FORWARD the email AS-IS, we lose all header information. Instead, attach the email to a NEW email and send us the original that way.
Send it to support@vircom.com
What will Vircom do?
Usually, we will implement a temporary outgoing filter rule to allow any emails sent from the particular user to go out temporarily while Proofpoint fixes the false positive and keep track of the ticket until closure.
Is there anything I can do to reduce the chance of this happening?
Yes -- there's a trick you can do, what we call an "open-sesame" rule. For those who don't know where the expression "open sesame" comes from, it's a phrase used in the children's fable of Ali Baba and the thousand knights. Basically, most companies have standardized signature. For instance, this is the author's personal signature put at the bottom of every Email:
Yves Lacombe | Support Director @ Vircom
Visit our status page at https://vircom.statusy.co
Phone: 514-845-1666 ext. 300 | Cell: 514-808-3457
Yves.lacombe@vircom.com | vircom.com
Nothing prevents you to add a catch phrase in the signature that you could use in a rule that would prevent signed messages from getting caught on the outbound leg. Example:
Yves Lacombe | Support Director @ Vircom
Visit our status page at https://vircom.statusy.co
Cogito Ergo Sum (I think, therefore I am)
Phone: 514-845-1666 ext. 300 | Cell: 514-808-3457
Yves.lacombe@vircom.com | vircom.com
Then, all you need to do is make an outgoing rule to allow anything with this catch phrase.
Conclusion
False Positives are a fact of life. Dealing with them doesn't need to be a huge chore. We provide you with all the tools you need. However if you need our help, don't hesitate to contact us, we'll be happy to lend you a hand.