SAGATOR version 0.3

(c) 2003 Jan ONDREJ (SAL) <ondrejj@salstar.sk>

 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
 the Free Software Foundation; either version 2 of the License, or
 (at your option) any later version.

This program is an email antivirus/antispam gateway. It is an interface
to the postfix (or any other smtpd), which runs antivirus
and/or spamchecker. Its modular architecture can use any
combination of antivirus/spamchecker according to configuration.

It currently supports clamav, nod32d, sophie, trophie and spamassassin.
It has some internal checkers (string_scanner and regexp_scanner).
Sagator can parse MIME mails and decompress archives, if it is configured so.

---------------------------------------------------------------------

Features:
  - simple chroot support
  - modular antivirus/spamchecker support
     - currently supports only clamav, nodoldd, sophie, trophie
     - any other antivirus can be implemented
     - spamaassassin support
  - modular architecture
     - you can attach an intrascanner to another intrascanner or realscanner
     - you can combine intrascanners
     - you can combine realscanners
  - you don't need any perl modules or any other modules, only python
  - you can return any quarantined mail to mailq/user mailbox

---------------------------------------------------------------------

Requirements:
  - python1 1.5.2 or higher (python2.2 is recomended,
      python2.2 is needed, if mime parser is configured, and also
      for native chroot support)
  - clamav, spamassassin, ...
  - postfix 1.1.11 (or may be later or newer)

---------------------------------------------------------------------

Tests:
  Hardware & Software:
    Intel Celeron 950 MHz/256 MB RAM Software:
    RedHat Linux 9
    postfix 1.1.11
    spamassassin - used as daemon
    clamav - started as binary
  Pattern:
    # of bigtest patterns (1 Palyh, 1 Lirva, 1 simple spam,
          1 long 1000 lines pattern, 1 long 300 lines pattern,
          1 mime zipped Palyh, 1 mime attached random data)
          size: 556 kB
    files: 7  (viruses: 4, spams: 1, clean: 2)
  Time with spamd: [spamd+MimeParse_stream(CLAMAVD)]
    10*3 /~ 3m20s (viruses+spams=90, clean=120, size=~16MB)
  Time without spamtest: [MimeParse_stream(CLAMAVD)]
    10*3 /~ 1m20s (viruses=60, clean=150, size=~16MB)

---------------------------------------------------------------------

Installation:

1. Getting started

  Unpack the package. Go into main directory and run:
    make install

2. Create chrooted environment.

  Go into main directory, then run:
    ./makechroot.sh /some/where/new/root/
The directory can be for example: "/var/spool/vscan".
This script creates a small copy of your files, where are required
for antivirus, unpacker, ...

  If you want to secure this newroot, do make secure.

3. Main configuration.

  Edit config.py. There are some parameters, described in config.py file.
Scanner configuration:
  Use SCANNERS variable in config.py. You can use more scanners.
If one of them found a virus, scanning is stopped. If one of them
fails, next scanner is used. The last scanner is defined by default
as "finalscanner()", which return no viruses for all mails.

  SCANNERS variable is a python array, defined as:

    SCANNERS=[ scanner1, scanner2, ... ]

  Scanners are defined in separate files, you must include it into
config.py. For best performance do not include scanners, which are not used.
For example for clamav use:
  from clamav import clamav,clamavd

Some examples:
  a) Use only clamscan binary to scan for viruses:
       SCANNERS=[
         clamav(['/usr/bin/clamscan','--stdout','--infected',
                 '--disable-summary','-r','--mbox'])
       ]

  b) Adding spamassassin tester:
       SCANNERS=SCANNERS+[ spamassassind(['localhost',783]) ]

  c) If you want to add some special stuff, like scan for
     names of attachments, and if you don't want any attachments,
     use mime_ext scanner (YOU NEED PYTHON 2.2 FOR THIS):
       EXECUTABLES='(\.exe|\.COM|\.com|\.COM|\.pif|\.PIF|\.lnk|\.LNK|\.scr|\.SCR)'
       SCANNERS=[ mime_ext(EXECUTABLES) ]

There are also subscanners ... they are scanners, which parses emails
and runs another scanner. For example there are mimeparse and
mimeparse_strem. "mimeparse" is used for scanners, which can scan
only files, not streams. "mimeparse_stream" is used for stream scanner.
If you don't know, which you need, try it.
After starting of sagator you see, if it work. :-)
WARNING! YOU NEED PYTHON2 FOR mimeparse MODULE!

Example for mimeparse:
  I wan't to test parsed files by clamd. Clamd can parse mail,
but I don't want to use its scanner now.

  mimeparse_stream(clamavd(['localhost',3310]))

4. Detailed scanner configuration.

There are three types of scanners:
  - streamscanner
  - filescanner
  - combination (filescanner+streamscanner)

Scanners are also divided into:
  - realscanners
     These scanners are real scanners, which scans the stream/file
     and then returns its status.
  - interscanners
     These scanners are an interface between main scanning engine
     and realscanners. You also can attach interscanner to another
     interscanner.

By default streamscanner is used. If you wan't to use filescanner,
there is a interscanner named stream2file().

If you need to know, which scanner is which, look at this table:

  ------------------------------------------------------------------
  | name                  | module       | inter | stream |  file  |
  ------------------------------------------------------------------
  | regexp_scan           | base         |   -   |   *    |   -    |
  | string_scan           | base         |   -   |   *    |   -    |
  | file2stream           | base         |   *   |  -/out |   in/- |
  | stream2file           | base         |   *   | in(m)  |  -/out |
  | l_return              | base         |   -   |   *    |   *    |
  | match_all             | base         |   *   | in/out |   -    |
  | alternatives          | base         |   *   | in/out |   -    |
  | match_any             | base         |   *   | in/out |   -    |
  | mimeparse             | mimeparse    |   *   | in/out |   -    |
  | mime_ext              | mimeparse    |   -   |   *    |   -    |
  | decompress            | decompress   |   *   |   -    | in/out |
  | clamscan              | clamav       |   -   |   *    |   -    |
  | clamd                 | clamav       |   -   |   *    |   -    |
  | spamassassin          | spamassassin |   -   |   *    |   -    |
  | spamassassind         | spamassassin |   -   |   *    |   -    |
  | nodold                | nodold       |   -   |   -    |   *    |
  | sophie                | sophie       |   -   |   -    |   *    |
  ------------------------------------------------------------------

  For detailed scanner description see README.scanners.

When you are using interscanners, you can't attach scanners of
wrong type. This means, that you can't attach a scanner with
input of stream to scanner, which has only file output.

For example you can use:
  mimeparse(string_scan(...))
but can't:
  mimeparse(decompress(...))

5. smtpd configuration.

5.1 Postfix configuration

# /etc/postfix/main.cf
mynetworks = 127.0.0.0/8, ...
content_filter = smtp:localhost:10027

# /etc/postfix/master.cf
# Local smtpd withou scanning
localhost:26	inet	n	-	n	-	5	smtpd
  -o content_filter=
  -o local_recipient_maps=
  -o myhostname=antivir.mydomain.sk

Myhostname can't be equal with your hostname, otherwise postfix
detects a loop.

Now you have 2 choices. You can use sagator as separate daemon,
or you can use it as postfix inetd-like service.
First choice is recomended, because python is too slow at starting.
Sagator is not optimized as inetd-like service now.

1st choice:
  Add sagator to startup scripts. In this package there is
a sagator.init scipt, which can be used in /etc/init.d/ .
You cas start it also manually.

2nd choice:
  Add following lines into master.cf:

localhost:10027	inet	n	n	n	-	5	spawn
  user=root argv=/usr/bin/sagator --inetd

5.2 Another smtpd configuration.

  For other smtpd, you can use sagator as main smtpd on port 25.
Configure your smtpd on another port as 25. Then configure
sagator to run on port 25 on your IP.

For example:
  Configure smtpd on port 26 on localhost. The use followinf configuration
for sagator:

  SMTP_HOST='127.0.0.1'
  SMTP_PORT=26
  BIND_HOST='0.0.0.0'
  BIND_PORT=25

6. ClamAV installation

  Download and install clamav - clam antivirus.
URL: http://clamav.elektrapro.com/. You will also need some
decompressors, like unrar, unace, unarj, zoo, lha, tar, deb.

  If you are using clamd, do not forget to start it in chroot,
and set these parameters:
  Leave commented:
    #ScannerDaemonOutputFormat # required
  Uncoment:
    StreamSaveToDisk # recomended
    ScanMail # required
    ScanArchive # recomended

  If you are using another DataDirectory (directory with viruses),
do not forget to set freshclam to use this directory. Otherwise
your virus database will be not up to date.

7. Testing.

  There some test scripts. Go into test directory and try:

    ./smtptest localhost 25 Eicar
    ./smtptest localhost 25 Bugbear.B
    ...
    ./bigtest yourlogin@yourdomain.sk X Y 25 sender@domain.sk
       Replace X a Y to number of files, which you want to send.
       There are 7*X*Y files sent.

  With these scripts you can also test your sagator daemon,
and you don't need to configure content_filter in postfix.
You can test it without real traffic. I recomend for you to
start sagator as command line program, and see its reports first.
Only if it works really, then use it as daemon.

8. Retunrning badly quarantined mail to mailq/user mailbox.

  If by chance is any mail quarantined, which should not, you can return
to queue or deliver to user mailbox by running command:

  sqback quarantined_file1 [quarantined_file2 [...]]

This can be done also via netcat program:

  nc localhost SMTP_PORT_WITHOUT_FILTER < .../quarantine/filename

9. It's all. :-)

  Congratulations. You have sucessfully installed sagator.
