メールフィルタリング ゲートウェイ

From Gentoo Wiki
Jump to:navigation Jump to:search
This page is a translated version of the page Mailfiltering Gateway and the translation is 33% complete.
Other languages:
This article has been flagged for not conforming to the wiki guidelines (Use of 2nd person pronouns). Please help Gentoo out by starting fixing things.
Some of the information in this article may have drifted out of sync with current practices. Please help out by checking over the content (how to get started).

このガイドでは、Postfix のスパム対策技術のインストール手順を段階的に説明しています。そのうち、Amavisd-new は Spamassassin と ClamAV を使用し、greylisting と SPF を使用しています。

いくつかの機能を備えたメールサーバを構成するための完全なガイドについては、完全な仮想メールサーバの記事を参照してください。

はじめに

このガイドでは、スパムおよびウイルスフィルタリングメールゲートウェイをインストールする方法を段階的に説明します。 単一のサーバーソリューションにこれを採用するのは非常に簡単です。

概観

このドキュメントでは、複数のドメインを持つスパムフィルタリングメールゲートウェイを設定する方法について説明します。 このサーバーは、実際にメールアカウント、すなわちMicrosoft ExchangeまたはLotus Notesを保持しているメールサーバーの前で実行されます。

この設定では、適切なセキュリティレコードと読み取り可能な設定ファイルを持つアプリケーションが選択されています。 電子メールMTAは、良いセキュリティー記録を持っていて、セットアップが簡単です。 Postfixは受信メールのために通常ポート25でリッスンします。 それを受信すると、ポート10024のAmavisd-newに転送されます。Amavisd-newは、メールを次のメールサーバーに転送するポート10025のPostfixにメールを戻す前に、異なるフィルタでメールをフィルタリングします。

Amavisd-newは、ウイルスフィルタリングとスパムフィルタリングのためのヘルパーアプリケーションを利用したコンテンツフィルタリングフレームワークです。 この設定では、ウイルスメールをフィルタリングするためのClamAVと、スパムをフィルタリングするためのSpamassassinの2つのヘルパーアプリケーションを使用します。 Spamassassin自体は、コンテンツフィルタリングフレームワークのさらに別の層として機能し、ヘルパアプリケーションVipulのRazor2とDCCを利用することができます。

RBLなどの他の多くのスパム対策テクノロジーとは異なり、Spamassassinは単一のテストに基づいて特定の電子メールを受諾または拒否するだけではありません。 これは、多くの内部テストと外部ヘルパーアプリケーションを使用して、通過するすべてのメールのスパムスコアを計算します。 このスコアは、次のテストに基づいています。

  • ベイジアンフィルタリング
  • 正規表現に基づく静的規則
  • 分散協調ネットワーク:
    • RBLs
    • Razor2
    • Pyzor
    • DCC

ガイドの最初の部分(第1章から第4章)では、メールフィルタリングゲートウェイの基本的なセットアップについて説明します。 次の章は、各章に依存することなく個別に実装できます。 これらの章では、次の方法について説明します。:

  • ベイジアンフィルターの学習と誤検知の配信のために特別なIMAPフォルダーを設定します
  • Postfixでグレイリストを設定する
  • Amavisdのセットアップ-ユーザー設定にMySQLバックエンドを使用するようになりました
  • AWLおよびベイズデータにMySQLバックエンドを使用するようにSpamassassinをセットアップします
メモ
以降 IMAP フォルダには maildir フォーマットを使用します。各メールを個別のファイルに保存すると、取り扱いがよりシンプルになります。mbox を使用している場合は、maildir を試してみることを提案します。まだ maildir を使用いていない場合は、必要なツールを emerge courier-imap で emerge してください。

計画されている第5部には、パフォーマンスに関するさまざまなヒントと、知りたいことが含まれます(chrootの実行、postfixの制限など)。

メモ
Delegating responsibility to third parties is not without risks. You have to know and trust these third parties. In this setup only the decision to quarantine virus mails are based on a single third party. Using Spamassassin's scoring system the decision to stop spam mails are not made by a single authority except perhaps Spamassassins own static rules.
警告
When rejecting spam mails at the MTA level you have to be very careful when selecting the RBL's you want to use, i.e. SpamCop is a bad RBL to use at the MTA level because you will experience false positives because sometimes their listing is just too aggressive. Further info at Realtime Blackhole Lists Are Bad and The Spam Problem: Moving Beyond RBLs

準備

Before you start make sure that you have a working Postfix installation where you can send and receive mails also you need a backend mailserver. If you're not experienced with setting up Postfix it might quickly become too complicated if all should be set up at once. If you need help you can find it in the excellent Complete Virtual Mail Server in the Gentoo Wiki.

必要なプログラムのインストール

まず、最も重要なプログラムであるAmavisd-new、Spamassassin、ClamAVをインストールします。

root #echo "mail-filter/amavisd-new spamassassin clamav" >> /etc/portage/package.use
root #emerge amavisd-new
root #freshclam
メモ
As previously mentioned you should already have a working postfix instance running on the box. Basically this shouldn't be much more than emerge postfix and have a basic understanding of how Postfix is working.

DNS のセットアップ

メモ
If you're not setting up a gateway server but have the mailboxes on the same server you only have to create the MX-Record.

プログラムが出現している間に、別のシェルを起動し、必要なDNSレコードを作成します。

メールゲートウェイのための MX レコードと、次の宛先のための A レコードの作成から始めてください。

コード Setting up DNS
(Create a MX record for the gateway server)
                MX      10      mailgateway.mydomain.tld.
(Create an A record for the gateway server)
mailgateway     A       mgw.ip.add.here
(Create an A record for the next hop mail server)
mail            A       ms.ip.add.here


メモ
Some ADSL providers might block port 25 and force you to relay mail through one of their servers. Typically you have to create a secondary MX-Record like MX 20 backup-mx.some-isp.tld


ファイアウォールを開ける

通常のメールトラフィックを許可することに加えて、ネットワークチェックがサーバーと通信できるようにするには、ファイアウォールを通過するいくつかのサービスを許可する必要があります。

アプリケーション プロトコル ポート
DCC UDP 6277
Razor (外向き ping) TCP 7
Razor TCP 2703

Razorは、pingを使用して、最も近いサーバーを検出します。

Postfix の設定

First we have to tell postfix to listen on port 10025 and we remove most of the restrictions as they have already been applied by the postfix instance listening on port 25. Also we ensure that it will only listen for local connections on port 10025. To accomplish this we have to add the following at the end of /etc/postfix/master.cf

コード Changing the master.cf file
smtp-amavis     unix -        -       n     -       2  smtp
  -o smtp_data_done_timeout=1200
  -o smtp_send_xforward_command=yes
#Equivalently when using lmtp:
#lmtp-amavis    unix -        -       n     -       2  lmtp
#   -o lmtp_data_done_timeout=1200
#   -o lmtp_send_xforward_command=yes
  
127.0.0.1:10025 inet n        -       n     -       -  smtpd
  -o content_filter=
  -o local_recipient_maps=
  -o relay_recipient_maps=
  -o smtpd_restriction_classes=
  -o smtpd_client_restrictions=
  -o smtpd_helo_restrictions=
  -o smtpd_sender_restrictions=
  -o smtpd_recipient_restrictions=permit_mynetworks,reject
  -o mynetworks=127.0.0.0/8
  -o strict_rfc821_envelopes=yes
  -o smtpd_error_sleep_time=0
  -o smtpd_soft_error_limit=1001
  -o smtpd_hard_error_limit=1000
  
#If you want to use proxy filtering instead
#smtp            inet n         -       n      -       8 smtpd
# -o smtpd_proxy_filter=127.0.0.1:10024
# -o smtpd_client_connection_count_limit=4
#If you don't want to scan outgoing mail use this
#10.0.0.2:smtp   inet n         -       n       -      - smtpd
#-o content_filter=
メモ
The smtp-amavis line specifies that a maximum of two of these processes may run at any time. If you need a greater degree of concurrency tune this number to fit your needs. Remember that to match the number with $max_servers in amavisd.conf . Keep in mind that amavisd-new is quite memory-intensive and raising the amount of amavisd-new processes too high can easily lead to memory starvation and heavy swapping, which leads to drastically reduced performance.
メモ
If you want to reject spam early on in the process you can use the Before-Queue (proxy) method instead of the filter method. If you uncomment the three lines you will have to set content_filter= in main.cf . This is not recommended for high traffic servers as the number of concurrent connections are limited to the number of amavisd instances.
警告
The Before-Queue(proxy) method is still not properly tested.
メモ
If you, for any reason whatsoever, want to send mail from this box and don't want it scanned, add another postfix instance by uncommenting the last two lines and substitute with a proper IP.

The file master.cf tells the postfix master program how to run each individual postfix process. More info with man 8 master .

Next we need the main postfix instance listening on port 25 to filter the mail through amavisd-new listening on port 10024.

また、メールのネクストホップ先を設定する必要があります。 Postfixに、外部コンテンツフィルタを介してすべてのメールをフィルタリングし、明示的なルーティングを有効にして、メールの転送先をPostfixに通知するように指示します。

ファイル /etc/postfix/main.cf
biff = no
empty_address_recipient = MAILER-DAEMON
queue_minfree = 120000000
  
content_filter = smtp-amavis:[127.0.0.1]:10024
#Equivalently when using lmtp:
#content_filter = lmtp-amavis:[127.0.0.1]:10024
  
# TRANSPORT MAP
#
# Insert text from sample-transport.cf if you need explicit routing.
transport_maps = hash:/etc/postfix/transport
  
relay_domains = $transport_maps

Postfix has a lot of options set in main.cf . For further explanation of the file please consult man 5 postconf or the same online Postfix Configuration Parameters .

The format of the transport file is the normal Postfix hash file. Mail to the domain on the left hand side is forwarded to the destination on the right hand side.

ファイル /etc/postfix/transport
mydomain.tld                          smtp:mail.mydomain.tld

After we have edited the file we need to run the postmap command. Postfix does not actually read this file so we have to convert it to the proper format with postmap /etc/postfix/transport . This creates the file /etc/postfix/transport.db . There is no need to reload Postfix as it will automatically pick up the changes.

メモ
If the next hop mail server is not listening on the standard SMTP port 25 you can tell postfix to use a given port number, like smtp:mail.mydomain.tld:25000 .

If your first attempts to send mail result in messages bouncing, you've likely made a configuration error somewhere. Try temporarily enabling soft_bounce while you work out your configuration issues. This prevents postfix from bouncing mails on delivery errors by treating them as temporary errors. It keeps mails in the mail queue until soft_bounce is disabled or removed.

root #postconf -e "soft_bounce = yes"
root #/etc/init.d/postfix reload

Once you've finished creating a working configuration, be sure to disable or remove soft_bounce and reload postfix.

amavisd-new の設定

Amavisd-newは、すべてのフィルタリングを処理するために使用され、さまざまなテクノロジーを簡単に接着できます。 メールメッセージを受信すると、メールを抽出し、いくつかのカスタムフィルタでフィルタリングし、白と黒のリストを処理し、さまざまなウイルススキャナーでメールをフィルタリングし、最後にSpamAssassinを使用してメールをフィルタリングします。

Amavisd-new自体には、いくつかの追加機能があります。:

  • it identifies dangerous file attachments and has policies to handle them
  • per-user, per-domain and system-wide policies for:
    • whitelists
    • blacklists
    • spam score thresholds
    • virus and spam policies

Apart from postfix and freshclam we will run all applications as the user amavis .

Edit the following lines in /etc/amavisd.conf

ファイル /etc/amavisd.conf
# (Insert the domains to be scanned)
$mydomain = 'example.com';
# (Bind only to loopback interface)
$inet_socket_bind = '127.0.0.1';
# (Forward to Postfix on port 10025)
$forward_method = 'smtp:127.0.0.1:10025';
$notify_method = $forward_method;
# (Define the account to send virus alert emails)
$virus_admin = "virusalert\@$mydomain";
# (Always add spam headers)
$sa_tag_level_deflt  = -100;
# (Add spam detected header aka X-Spam-Status: Yes)
$sa_tag2_level_deflt = 5;
# (Trigger evasive action at this spam level)
$sa_kill_level_deflt = $sa_tag2_level_deflt;
# (Do not send delivery status notification to sender.  It does not affect
# delivery of spam to recipient. To do that, use the kill_level)
$sa_dsn_cutoff_level = 10;
# Don't bounce messages left and right, quarantine
# instead
$final_virus_destiny      = D_DISCARD;  # (defaults to D_DISCARD)
$final_banned_destiny     = D_DISCARD;  # (defaults to D_BOUNCE)
$final_spam_destiny       = D_DISCARD;  # (defaults to D_BOUNCE)
メモ
With this line $sa_tag2_level_deflt = 5; you set the Spamassassin spam score to 5. This might be a bit low. As you might have noticed the Amavisd-new default is 6.3 . If you don't want to see a single spam mail in your mail folder choose 5 , but if you don't want to deal with false positives choose 6.3 .

ウイルスメールをユーザーに配信したくないため、ウイルスメールの検疫ディレクトリを作成します。

root #mkdir /var/amavis/virusmails
root #chown amavis:amavis /var/amavis/virusmails
root #chmod 750 /var/amavis/virusmails
メモ
Amavisd-new offers finer policy tuning by using policy banks.

ClamAV の設定

As virus scanner we use ClamAV as it has a fine detection rate comparable with commercial offerings, it is very fast and it is Open Source Software. We love log files, so make clamd log using syslog and turn on verbose logging. Also do not run clamd as root . Now edit /etc/clamd.conf

ファイル /etc/clamd.conf
# (Verbose logging with syslog)
LogSyslog
LogVerbose
LogFacility LOG_MAIL
# (Change pid file location)
PidFile /var/run/amavis/clamd.pid
# (Set the clamav socket)
LocalSocket /var/amavis/clamd
# (Close the connection when this limit is exceeded)
StreamMaxLength 10M
# (Don't run clamd as root)
User amavis
# (Newer versions require you to uncomment this)
ScanMail
ScanArchive
メモ
Also remember to remove the Example directive to make ClamAV work

ClamAV comes with the freshclam deamon dedicated to periodical checks of virus signature updates. Instead of updating virus signatures twice a day we will make freshclam update virus signatures every two hours.

ファイル /etc/freshclam.conf
# (Syslog logging)
LogSyslog
# (Verbose logging)
LogVerbose
# (Explicitly drop root privileges)
DatabaseOwner clamav
# (Check for updates every two hours. That is the official recommendation)
Checks 12
# (Use the mirror closest to you. Replace XY with your country code
DatabaseMirror db.XY.clamav.net

Start clamd with freshclam using the init scripts by modifying /etc/conf.d/clamd .

ファイル /etc/conf.d/clamd
START_CLAMD=yes
START_FRESHCLAM=yes
CLAMD_NICELEVEL=3
FRESHCLAM_NICELEVEL=19
IONICE_LEVEL=2

At last modify amavisd.conf with the new location of the socket.

ファイル /etc/amavisd.conf
# (Uncomment the clamav scanner and modify socket location)
['ClamAV-clamd',
\&ask_daemon, ["CONTSCAN {}\n", "/var/amavis/clamd"],
  qr/\bOK$/, qr/\bFOUND$/,
  qr/^.*?: (?!Infected Archive)(.*) FOUND$/ ],
警告
Do NOT modify the $unix_socketname unless you know what you're doing.

Configuring Vipul's Razor

Razor2 is a collaborative and distributed spam checksum network. Install it with emerge razor and create the needed configuration files. Do this as user amavis by running su - amavis followed razor-admin -create .

root #emerge razor
root #su - amavis -s /bin/bash
user $razor-admin -create
user $exit

Configuring Distributed Checksum Clearinghouse (dcc)

Razor2と同様に、dccは協調的で分散型のスパムチェックサムネットワークです。 その哲学は、ファジーチェックサムで各メールを識別する特定のメールの受信者の数を数えることです。

root #emerge dcc

Spamassassin を設定する

Amavis is using the Spamassassin Perl libraries directly so there is no need to start the service. Also this creates some confusion about the configuration as some Spamassassin settings are configured in /etc/mail/spamassassin/local.cf and overridden by options in /etc/amavisd.conf .

ファイル /etc/mail/spamassassin/local.cf
# Enable the Bayes system
use_bayes               1
  
# Enable all network checks
skip_rbl_checks         0
  
# Mail using languages used in these country codes will not be marked
# as being possibly spam in a foreign language.
# - danish english norwegian swedish
ok_languages            da en no sv
  
# Mail using locales used in these country codes will not be marked
# as being possibly spam in a foreign language.
ok_locales              en
  
# Use a sensible bayes path
bayes_path              /var/amavis/.spamassassin/bayes


メモ
With Spamassassin version 3.1 you have to enable DCC, Razor2 by uncommenting the corresponding lines in v310.pre .
メモ
You can find inspiration for your local.cf file by trying the SpamAssassin Configuration Generator .
メモ
You might also want to switch the ok_languages and ok_locales .

すべての適切なルールには適切な例外もあります

Once mail really starts passing through this mail gateway you will probably discover that the above setup is not perfect. Maybe some of your customers like to receive mails that others wouldn't. You can whitelist/blacklist envelope senders quite easily. Uncomment the following line in amavisd.conf .

ファイル amavisd.confdo sitewide scoring
read_hash("/var/amavis/sender_scores_sitewide"),

In the sender_scores_sitewide file you put complete email addresses or just the domian parts and then note a positive/negative score to add to the spam score.

ファイル whitelist_senderexample
# (Whitelist all emails from the specific email address)
postmaster@example.net                -3.0
# (Whitelist all emails from the example.net excluding subdomains)
.example.net                          1.0


メモ
See /etc/amavisd.conf for more examples.
メモ
Placing these addresses outside amavisd.conf is a cleaner and safer solution.
メモ
Alternatively it can be done in Spamassassin's configuration file /etc/mail/spamassassin/local.cf but I think it is cleaner to do it in /etc/amavisd.conf .
メモ
In a later chapter I will show how to implement per-user policies using MySQL.

While waiting for a better method you can add the following to amavisd.conf to bypass spam checks for postmaster and abuse mailboxes.

コード Bypass spam filters for all postmaster and abuse mails
map { $bypass_spam_checks{lc($_)}=1 } (qw(
        postmaster@
        abuse@
));
重要
While we are at it we should never automatically discard mails to the postmaster or the abuse accounts. See RFC 2142 MAILBOX NAMES FOR COMMON SERVICES, ROLES AND FUNCTIONS . Otherwise your domains might end up listed in some of the evil lists over at rfc-ignorant.org .

ルールをさらに追加

If you want to use more rules provided by the SARE Ninjas over at the SpamAssassin Rules Emporium you can easily add and update them using the sa-update mechanism included in Spamassassin.

A brief guide to using SARE rulesets with sa-update can be found here .

テストと仕上げ

構成をテストする

Now before you start freshclam you can manually verify that it works.

root #freshclam
ClamAV update process started at Sun May  2 09:13:41 2004
Reading CVD header (main.cvd): OK
Downloading main.cvd [*]
main.cvd updated (version: 22, sigs: 20229, f-level: 1, builder: tkojm)
Reading CVD header (daily.cvd): OK
Downloading daily.cvd [*]
daily.cvd updated (version: 298, sigs: 1141, f-level: 2, builder: diego)
Database updated (21370 signatures) from database.clamav.net (193.1.219.100).

Now you have updated virus definitions and you know that freshclam.conf is working properly.

Test freshclam and amavisd from the cli and amavisd testmails. Start clamd and amavis with the following commands:

root #/etc/init.d/clamd start
root #/etc/init.d/amavisd start
root #/etc/init.d/postfix reload

If everything went well postfix should now be listening for mails on port 25 and for reinjected mails on port 10024. To verify this check your log file.

root #tail -f /var/log/mail.log
メモ
Depending on your log settings the correct path might be /var/log/messages .

Now if no strange messages appear in the log file it is time for a new test.

Use netcat to manually connect to amavisd on port 10024 and postfix on port 10025.

メモ
Netcat can be used as an advanced replacement for telnet . Install it with emerge netcat .
メモ
For some unknown reason you can not complete a manual mail injection to amavisd with netcat. Use telnet instead.
root #nc localhost 10024
220 [127.0.0.1] ESMTP amavisd-new service ready
root #nc localhost 10025
220 example.com ESMTP Postfix


メモ
If you want to see the complete output from amavisd-new start amavisd debug-sa as the amavis user and send a mail. For this to work you might have to change the default shell in /etc/passwd .

Add amavisd and clamd to the default runlevel.

root #rc-update add clamd default
root #rc-update add amavisd default
メモ
We do not add spamd to the default runlevel as amavisd uses the Spamassassin Perl libraries directly.
メモ
You might notice Net::Server: Couldn't POSIX::setuid to ... [] lines in your log. According to amavis chroot README , if the process UID remains 0 ( root ), the program will terminate, otherwise you can consider the message just as informative. This is because POSIX::setuid() returns a string 0 but true .
重要
If you enabled login for amavis remember to set back the login shell in /etc/passwd to /bin/false .

Autolearning and sidelining emails

Creating the spamtrap user

Create the spamtrap account and directories.

root #useradd -m spamtrap
root #maildirmake /home/spamtrap/.maildir
root #chown -R spamtrap:spamtrap /home/spamtrap/.maildir

Give the spamtrap user a sensible password.

root #passwd spamtrap

If you manually want to check some of the mails to ensure that you have no false positives you can use the following procmail recipe to sideline spam found into different mail folders.

.procmailrc の作成

ファイル /home/spamtrap/.procmailrc
#Set some default variables
MAILDIR=$HOME/.maildir
  
SPAM_FOLDER=$MAILDIR/.spam-found/
  
LIKELY_SPAM_FOLDER=$MAILDIR/.likely-spam-found/
  
#Sort mails with a spamscore of 7+ to the spamfolder
:0:
* ^X-Spam-Status: Yes
* ^X-Spam-Level: \*\*\*\*\*\*\*
$SPAM_FOLDER
  
#Sort mail with a spamscore between 5-7 to the likely spam folder
:0:
* ^X-Spam-Status: Yes
$LIKELY_SPAM_FOLDER
  
#Sort all other mails to the inbox
:0
*
./
警告
If your mail server is going to receive a lot of mail you should NOT use the likely-spam recipe. Instead set $sa_tag2_level_deflt high enough to avoid false positives and filter it directly to $SPAM_FOLDER .
メモ
If you haven't already installed procmail do it with emerge procmail .

Now make sure that Postfix uses procmail to deliver mail.

ファイル /etc/postfix/main.cf
mailbox_command = /usr/bin/procmail -a "DOMAIN"

メールフォルダの作成

Now we will create shared folders for ham and spam.

root #maildirmake /var/amavis/.maildir
root #maildirmake -S /var/amavis/.maildir/Bayes
root #maildirmake -s write -f spam /var/amavis/.maildir/Bayes
root #maildirmake -s write -f ham /var/amavis/.maildir/Bayes
root #maildirmake -s write -f redeliver /var/amavis/.maildir/Bayes

Amavisd-new needs to be able to read these files as well as all mailusers. Therefore we add all the relevant users to the mailuser group along with amavis.

root #groupadd mailusers
root #usermod -G mailusers spamtrap
root #chown -R amavis:mailusers /var/amavis/.maildir/
root #chown amavis:mailusers /var/amavis/
root #chmod -R 1733 /var/amavis/.maildir/Bayes/
root #chmod g+rx /var/amavis/.maildir/
root #chmod g+rx /var/amavis/.maildir/Bayes/
警告
This grants members of the mailusers groups access to amavis mail.

This makes the spam and ham folders writable but not readable. This way users can safely submit their ham without anyone else being able to read it.

Then run the following command as the spamtrap user:

user $maildirmake --add Bayes=/var/amavis/.maildir/Bayes $HOME/.maildir
メモ
We have to give the group read permissions on the Bayes folder in order for the mail client to be able to see the subdirectories used by IMAP.

cron ジョブの追加

Now run crontab -u amavis -e to edit the amavis crontab to enable automatic learning of the Bayes filter every hour.

ファイル crontabfor amavis user
#Auto learn
0 * * * *          /usr/bin/sa-learn --spam /var/amavis/.maildir/Bayes/.spam/{cur,new} \
                    > /dev/null 2>&1
0 * * * *          /usr/bin/sa-learn --ham /var/amavis/.maildir/Bayes/.ham/{cur,new} > \
                   /dev/null 2>&1
メモ
amavis has to be a member of thecron group to run crons.
メモ
It seems like the shared maildir folders will make sa-learn examine all messages twice. This should not be a problem. The output will also show that the maximum of messages learned from is half or less than the messages examined.

amavisd.conf の変更

Now modify amavis to redirect spam emails to the spamtrap account and keep spamheaders.

ファイル /etc/amavisd.conf
# (Define the account to send virus spam emails)
$spam_quarantine_to = "spamtrap\@$myhostname";

後片付け

We don't want to keep mail forever so we use tmpwatch to clean up regularily. Emerge it with emerge tmpwatch . Only root is able to run tmpwatch so we have to edit the root crontab.

ファイル crontabroot user
# Clean up
# Keep virusmails for a week (24*7 hours)
15 0 * * *      /usr/sbin/tmpwatch -c -f -d --quiet 168 /var/amavis/virusmails/
# Delete spam and ham learned after a week
15 0 * * *      /usr/sbin/tmpwatch -c -f -d --quiet 168 /var/amavis/.maildir/Bayes/

グレーリスト

はじめに

Greylisting is one of the newer weapons in the spam fighting arsenal. As the name implies it is much like whitelisting and blacklisting. Each time an unknown mailserver tries to deliver mail the mail is rejected with a try again later message. This means that mail gets delayed but also that stupid spam bots that do not implement the RFC protocol will drop the attempt to deliver the spam and never retry. With time spam bots will probably adjust, however it will give other technologies more time to identify the spam.

メモ
If your ISP blocks incoming traffic on port 25 and relays all mail to you through their own mail server greylisting will not work.

Postfix 2.1 come with a simple Perl greylisting policy server that implements such a scheme. However it suffers from unpredictable results when the partition holding the greylisting database run out of space. There exists an improved version that do not suffer this problem. First I will show how to install the builtin greylisting support that come with Postfix and then I will show how to configure the more robust replacement.

メモ
There are other greylisting policy servers for Postfix around (such as Gld , which is in Portage, and SQLgrey ). Some of them support database backends, auto whitelisting and other neat features.

Simple greylisting

メモ
If you prefer to use the improved greylisting with postgrey you can safely skip this section.

We need the file greylist.pl but unfortunately the ebuild does not install it as default.

root #cp /usr/portage/distfiles/postfix-your-version-here.tar.gz /root/
root #tar xzf postfix-your-version-here.tar.gz
root #cp postfix-2.1.0/examples/smtpd-policy/greylist.pl /usr/bin/

Now we have the file in place we need to create the directory to hold the greylisting database:

root #mkdir /var/mta
root #chown nobody /var/mta
警告
Do not create the greylisting database directory on a partition that might run out of space. While postfix can recover from no-space-left situations for the mail queue and mail box situations, this is not the case with the greylisting database. If the file becomes corrupted you may not be able to receive mail at all until you delete the file by hand.

Configuring greylisting

Now that we have all this ready all that is left is to add it to the postfix configuration. First we add the necessary information to the master.cf :

コード Modifying master.cf to use greylisting
policy-greylist  unix  -       n       n       -       -       spawn
   user=nobody argv=/usr/bin/perl /usr/bin/greylist.pl

The postfix spawn daemon normally kills its child processes after 1000 seconds but this is too short for the greylisting process so we have to increase the timelimit in main.cf :

ファイル main.cfuse greylisting
policy-greylist_time_limit = 3600
# (Under smtpd_recipient_restrictions add:)
check_sender_access hash:/etc/postfix/sender_access
# (Later on add:)
restriction_classes = greylist
greylist = check_policy_service unix:private/policy-greylist
警告
Be sure to specify check_sender_access AFTER reject_unauth_destination or else your system could become an open mail relay.
メモ
The greylist database gets polluted quickly with bogus addresses. It helps if you protect greylist lookups with other restrictions that reject unknown senders and/or recipients.

We don't want to use greylisting for all domains but only for those frequently abused by spammers. After all it will delay mail delivery. A list of frequently forged MAIL FROM domains can be found online . Add the domains you receive a lot of spam from to /etc/postfix/sender_access :

コード sender_access のフォーマット
aol.com     greylist
hotmail.com greylist
bigfoot.com greylist

If you want a more extensive list:

root #cat sender-domain-validate.in | sort | awk {'print $1 "\t\t greylist"'} > /etc/postfix/sender_access

Now we only have to initialize the sender_access database:

root #postmap /etc/postfix/sender_access

Now the setup of simple greylisting is complete.

警告
I tried this on one box handling thousands of mails daily and the results were almost a complete disaster. After four days the box was bogged down with hundreds of old greylist.pl processes.

Configuring improved greylisting with postgrey

You can install the enhanced greylisting policy server with a simple emerge :

root #emerge postgrey

After installing postgrey we have to edit main.cf . Changes are almost exactly like the built in greylisting.

ファイル main.cfuse greylisting
# (Under smtpd_recipient_restrictions add:)
check_sender_access hash:/etc/postfix/sender_access
# (Later on add:)
smtpd_restriction_classes = greylist
greylist = check_policy_service inet:127.0.0.1:10030
メモ
The Postfix SMTPD_POLICY_README only uses restriction_classes but that does not appear to work.
メモ
If you want to greylist everything instead add check_policy_service inet:127.0.0.1:10030 .

Finally, start the server and add it to the proper runlevel.

root #/etc/init.d/postgrey start
root #rc-update add postgrey default
メモ
Some people like to get their mail fast and thus greylisting is worthless. However if you employ a backup mail server you can safely setup greylisting on that server. My limited experiences tell me that it can stop up to a third of the spam received.

SPF (Sender Policy Framework)

はじめに

SPF allows domain owners to state in their DNS records which IP addressess should be allowed to send mails from their domain. This will prevent spammers from spoofing the Return-Path.

メモ
If your ISP blocks incoming traffic on port 25 and relays all mail to you through their own mail server SPF will not work.

First domain owners have to create a special TXT DNS record. Then an SPF-enabled MTA can read this and if the mail originates from a server that is not described in the SPF record the mail can be rejected. An example entry could look like this:

コード Example SPF record
example.com.  IN TXT  "v=spf1 a mx ptr -all"

The -all means to reject all mail by default but allow mail from the A( a ), MX( mx ) and PTR( ptr ) DNS records. For more info consult further resources below.

メモ
If you relay outgoing mail through your ISP you will have to add: include:yourisp.com .

Spamassassin 3.0 has support for SPF, however it is not enabled by default and the new policy daemon in Postfix supports SPF so let's install SPF support for Postfix.

メモ
If you want to use SPF with Spamassassin instead simply emerge dev-perl/Mail-SPF-Query and restart Amavisd-new.

準備

First you have to install Postfix 2.1 as described above. When you have fetched the source grab the spf.pl with:

root #cp postfix-<version>/examples/smtpd-policy/spf.pl /usr/local/bin/
メモ
The spf.pl coming with Postfix is slightly buggy so find and uncomment the following line: push @HANDLERS, "sender_permitted_from"; use Mail::SPF::Query; . Furthermore in about line 199 substitute comemnt with comment . Alternatively you can download a development version .

This Perl script also needs some Perl libraries that are not in portage but it is still quite simple to install them:

root #emerge Mail-SPF-Query Net-CIDR-Lite Sys-Hostname-Long

Now that we have everything in place all we need is to configure Postfix to use this new policy.

ファイル master.cfuse SPF
policy-spf  unix  -       n       n       -       -       spawn
   user=nobody argv=/usr/bin/perl /usr/local/bin/spf.pl

Now add the SPF check in main.cf . Properly configured SPF should do no harm so we could check SPF for all domains:

ファイル main.cfuse SPF
# (Under smtpd_recipient_restrictions add:)
check_policy_service unix:private/policy-spf
メモ
If you're experiencing problems with SPF, e.g. when using fetchmail , you might want to enable SPF for certain domains only.

Configuring amavisd-new to use MySQL

MySOL の設定

メモ
This has not been tested on versions higher than 2.2. Feedback is welcome :)

For large domains the default values you can set in amavisd.conf might not suit all users. If you configure amavisd-new with MySQL support you can have individual settings for users or groups of users.

user $mysql -u root -p mysql
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 78 to server version: 4.0.18-log
  
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>create database maildb;
mysql>GRANT INSERT,UPDATE,DELETE,SELECT ON maildb.* TO 'mail'@'localhost' IDENTIFIED BY 'very_secret_password';
mysql>use maildb;

Now that the database is created we'll need to create the necessary tables. You can cut and paste the following into the mysql prompt:

コード MySQL table layout
CREATE TABLE users (
    id         int unsigned NOT NULL auto_increment,
    priority   int          NOT NULL DEFAULT '7',  -- 0 is low priority
    policy_id  int unsigned NOT NULL DEFAULT '1',
    email      varchar(255) NOT NULL,
    fullname   varchar(255) DEFAULT NULL,    -- not used by amavisd-new
    local      char(1),     -- Y/N  (optional field, see note further down)
    PRIMARY KEY (id),
    KEY email (email)
    );
CREATE UNIQUE INDEX users_idx_email ON users(email);
  
# (any e-mail address, external or local, used as senders in wblist)
CREATE TABLE mailaddr (
   id         int unsigned NOT NULL auto_increment,
   priority   int          NOT NULL DEFAULT '7',  -- 0 is low priority
   email      varchar(255) NOT NULL,
   PRIMARY KEY (id),
   KEY email (email)
   );
CREATE UNIQUE INDEX mailaddr_idx_email ON mailaddr(email);
  
# (-- per-recipient whitelist and/or blacklist,
# -- puts sender and recipient in relation wb)
# (white or blacklisted sender)
CREATE TABLE wblist (
   rid        int unsigned NOT NULL,     -- recipient: users.id
   sid        int unsigned NOT NULL,     -- sender:    mailaddr.id
   wb         char(1) NOT NULL, -- W or Y / B or N / space=neutral
   PRIMARY KEY (rid,sid)
   );
  
CREATE TABLE policy (
   id               int unsigned NOT NULL auto_increment,
   policy_name      varchar(32),     -- not used by amavisd-new
   virus_lover          char(1),     -- Y/N
   spam_lover           char(1),     -- Y/N  (optional field)
   banned_files_lover   char(1),     -- Y/N  (optional field)
   bad_header_lover     char(1),     -- Y/N  (optional field)
   bypass_virus_checks  char(1),     -- Y/N
   bypass_spam_checks   char(1),     -- Y/N
   bypass_banned_checks char(1),     -- Y/N  (optional field)
   bypass_header_checks char(1),     -- Y/N (optional field)
   spam_modifies_subj   char(1),     -- Y/N (optional field)
   spam_quarantine_to   varchar(64) DEFAULT NULL, -- (optional field)
   spam_tag_level  float,  -- higher score inserts spam info headers
   spam_tag2_level float DEFAULT NULL,  -- higher score inserts
               -- 'declared spam' info header fields
   spam_kill_level float,  -- higher score activates evasive actions, e.g.
               -- reject/drop, quarantine, ...
               -- (subject to final_spam_destiny setting)
   PRIMARY KEY (id)
  );
メモ
If you have problems using copy/paste you might have to copy this somewhere else and clean out the unneeded characters.
メモ
Lookups trying to match email are done with raw (rfc2821-unquoted and unbracketed) addresses as a key, i.e.: John "Funny" Smith@example.com
メモ
Lookups are performed in the following order: SQL , LDAP , hash , ACL , regexp , constant . The first that returns a definitive answer (not undef/NULL ) stops the search.

If you wish to use whitelisting and blacklisting you must add the sender and receiver to mailadr after which you create the relation between the two e-mail addresses in wblist and state if it is whitelisting ( W ) or blacklisting ( B ).

Now that we have created the tables let's insert a test user and a test policy:

コード Create test user and test policy
INSERT INTO users
   SET
      id         =1,
      priority   =9,
      policy_id  =1,
      email      ="johndoe@example.com",
      fullname   ="John Doe",
      local      ="Y";
  
INSERT INTO policy
   SET
      id                     =1,
      policy_name            ="Test policy 1",
      virus_lover            ="N",
      spam_lover             ="N",
      banned_files_lover     ="N",
      bad_header_lover       ="N",
      bypass_virus_checks    ="N",
      bypass_spam_checks     ="N",
      bypass_banned_checks   ="N",
      bypass_header_checks   ="N",
      spam_modifies_subj     ="N",
      spam_quarantine_to     =NULL,
      spam_tag_level         =-50.0,
      spam_tag2_level        =7.0,
      spam_kill_level        =10.0;
メモ
Copy this to somewhere else and adjust to suit your own environment.
メモ
local should be set toY otherwise the mail will not be scanned for spam.

This inserts a test user and a Test policy. Adjust these examples to fit your needs. Further explanation of the configuration names can be found in amavisd.conf .

Configuring amavisd to use MySQL

Now that MySQL is ready we need to tell amavis to use it:

ファイル amavisd.confUpdate to use MySQL
@lookup_sql_dsn =
   ( ['DBI:mysql:maildb:host1', 'mail', 'very_secret_password']  );
  
# (For clarity uncomment the default)
$sql_select_policy = 'SELECT *,users.id FROM users,policy'.
   ' WHERE (users.policy_id=policy.id) AND (users.email IN (%k))'.
   ' ORDER BY users.priority DESC';
  
# (If you want sender white/blacklisting)
   $sql_select_white_black_list = 'SELECT wb FROM wblist,mailaddr'.
     ' WHERE (wblist.rid=?) AND (wblist.sid=mailaddr.id)'.
     '   AND (mailaddr.email IN (%k))'.
     ' ORDER BY mailaddr.priority DESC';
</pre>

Configuring Spamassassin to use MySQL

As of Spamassassin 3.0 it is possible to store the Bayes and AWL data in a MySQL database. We will use MySQL as the backend as it can generally outperform other databases. Also, using MySQL for both sets of data makes system management much easier. Here I will show how to easily accomplish this.

First start out by creating the new MySQL user and then create the needed tables.

root #mysql -u root -p mysql
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 78 to server version: 4.0.18-log
  
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>create database dbname;
mysql>GRANT INSERT,UPDATE,DELETE,SELECT ON dbname.* TO 'dbuser'@'localhost' IDENTIFIED BY 'another_very_secret_password';
mysql>use dbname;

Now that the database is created we'll create the necessary tables. You can cut and paste the following into the mysql prompt:

コード MySQL table layout
CREATE TABLE bayes_expire (
          id                    int(11)         NOT NULL default '0',
          runtime               int(11)         NOT NULL default '0',
          KEY bayes_expire_idx1 (id)
          ) TYPE=MyISAM;
  
      CREATE TABLE bayes_global_vars (
          variable              varchar(30)     NOT NULL default '',
          value                 varchar(200)    NOT NULL default '',
          PRIMARY KEY           (variable)
          ) TYPE=MyISAM;
  
      INSERT INTO bayes_global_vars VALUES ('VERSION','3');
  
      CREATE TABLE bayes_seen (
          id                    int(11)         NOT NULL default '0',
          msgid                 varchar(200) binary NOT NULL default '',
          flag                  char(1)         NOT NULL default '',
          PRIMARY KEY           (id,msgid)
          ) TYPE=MyISAM;
  
      CREATE TABLE bayes_token (
          id                    int(11)         NOT NULL default '0',
          token                 char(5)         NOT NULL default '',
          spam_count            int(11)         NOT NULL default '0',
          ham_count             int(11)         NOT NULL default '0',
          atime                 int(11)         NOT NULL default '0',
          PRIMARY KEY           (id, token),
          INDEX (id, atime)
          ) TYPE=MyISAM;
  
      CREATE TABLE bayes_vars (
          id                    int(11)         NOT NULL AUTO_INCREMENT,
          username              varchar(200)    NOT NULL default '',
          spam_count            int(11)         NOT NULL default '0',
          ham_count             int(11)         NOT NULL default '0',
          token_count           int(11)         NOT NULL default '0',
          last_expire           int(11)         NOT NULL default '0',
          last_atime_delta      int(11)         NOT NULL default '0',
          last_expire_reduce    int(11)         NOT NULL default '0',
          oldest_token_age      int(11)         NOT NULL default '2147483647',
          newest_token_age      int(11)         NOT NULL default '0',
          PRIMARY KEY           (id),
          UNIQUE bayes_vars_idx1 (username)
          ) TYPE=MyISAM;
  
      CREATE TABLE awl (
          username              varchar(100)    NOT NULL default '',
          email                 varchar(200)    NOT NULL default '',
          ip                    varchar(10)     NOT NULL default '',
          count                 int(11)         default '0',
          totscore              float           default '0',
          PRIMARY KEY           (username,email,ip)
          ) TYPE=MyISAM;
重要
The INSERT line is needed otherwise Spamassassin will not work.
メモ
This is also available in the source tarball in the files awl_mysql.sql and bayes_mysql.sql .

Configuring Spamassassin to use the MySQL backend

If you have an old Bayes database in the DBM database and want to keep it follow these instructions:

root #su - amavis
user $sa-learn --sync
user $sa-learn --backup > backup.txt
user $sa-learn --restore backup.txt
メモ
Note that the last step should only be performed after the MySQL database and secrets.cf have been updated.

Now give Spamassassin the required info:

ファイル /etc/mail/spamassassin/secrets.cf
# (Tell Spamassassin to use MySQL for bayes data
bayes_store_module              Mail::SpamAssassin::BayesStore::SQL
bayes_sql_dsn                   DBI:mysql:sa_bayes:localhost:3306
bayes_sql_username              db_name
bayes_sql_password              another_very_secret_password
  
# (Tell Spamassassin to use MySQL for AWL data
auto_whitelist_factory          Mail::SpamAssassin::SQLBasedAddrList
user_awl_dsn                    DBI:mysql:sa_bayes:localhost:3306
user_awl_sql_username           db_name
user_awl_sql_password           another_very_secret_password

Next, change its permissions for proper security:

root #chmod 400 /etc/mail/spamassassin/secrets.cf
メモ
To create a very secret password use emerge app-admin/makepasswd and makepasswd -chars=8

Now all you have to do is /etc/init.d/amavisd restart .

トラブルシューティング

Amavisd-new

To troubleshoot Amavisd-new start out by stopping it with /etc/init.d/amavisd stop and then start it manually in the foreground with amavisd debug and watch it for anomalies in the output.

Spamassassin

To troubleshoot Spamassassin you can filter an email through it with spamassassin -D < mail . To ensure that the headers are intact you can move it from another machine with IMAP.

メモ
If you need to troubleshoot you have to enable login for the user amavis by changing the login shell in /etc/passwd to /bin/bash .

If you want you can make get the same information and more with Amavisd-new using amavisd debug-sa .

Repeating tasks after installation

Some of the activities mentioned in this guide will need to be repeated after upgrades. For instance, the chown -R amavis:mailusers will need to be repeated after every update of amavisd-new.

Luckily, Gentoo provides you with the means to perform these steps automatically. In Hooking in the Emerge Process, the Gentoo Handbook explains how to execute tasks after installations of a particular package, like so:

コード Example bashrc snippet for running the mentioned chown
if [ "${PN}" == "amavisd-new" ] &&
   [ "${EBUILD_PHASE}" == "postinst" ]; 
then
  chown -R amavis:mailusers /var/amavis/.maildir
fi

Getting help

If you need help a good place to go is the amavis-user mailing list. Before postting a question try searching the Amavis User mailing list archives . If you find no answer here you can subscribe to the Amavis User mailing list

If your question is specific to SpamAssassin, DCC, Razor, or Postfix, please refer to their respective home pages listed below.

資料

For further information

一般的な資料


This page is based on a document formerly found on our main website gentoo.org.
The following people contributed to the original document: Sune Kloppenborg Jeppesen, Jens Hilligs, Joshua Saddler
They are listed here because wiki history does not allow for any external attribution. If you edit the wiki article, please do not add yourself here; your contributions are recorded on each article's associated history page.