anomy-list

Re: Re: 100 % CPU utilization Postfix+Anomy+SpamAssassin+ClamAV

From: Derrick Webber (139912@xyz.molar.is)
Date: Wed 09 Feb 2005 - 19:27:11 GMT

  • Next message: Bjarni R. Einarsson: "Daemonizing perl code: fork() is your friend! :-)"

    Paolo said:
    > On Tue, Feb 08, 2005 at 07:57:11PM -0500, Derrick Webber wrote:
    >> Paolo wrote:
    > ...
    > yes, like clamd; but sanitizer may call a number of helpers which may
    > slow things down as well.

    Absolutely... executing another process does. And if Sanitizer has to call
    a non-daemonized helper you'll lose performance same as before. Ideally
    you'd want to use daemonized versions of the helpers that are called for
    every message, like a virus scanner. For less used helpers (like our
    'tnef2multipart' and 'zip_policy' add-ons), only certain messages will be
    affected.

    On average though, message processing will be much faster with a
    daemonized sanitizer.

    >
    > so, what's your numbers with .pl, perlcc'd, perperl cases?
    >

    Here's what I get using the 'time' command with plain old sanitizer.pl and
    a small test message:

     real 0m0.753s
     user 0m0.440s
     sys 0m0.070s

    Here's the same message processed with a 'perlcc' compiled sanitizer:

     real 0m0.179s
     user 0m0.070s
     sys 0m0.080s

    (The above are from a Sun e250 running Solaris 8. Not a fast machine.)

    With Persistent Perl (on an Athlon XP 2500 running Debian Testing and a
    manually compiled perl 5.8.5):

    First run (perperl backend not running)
     real 0m0.089s
     user 0m0.001s
     sys 0m0.001s

    Subsequent runs (backend running so Sanitizer is 'daemonized')
     real 0m0.009s
     user 0m0.001s
     sys 0m0.002s

    I don't use perperl with sanitizer on any production system. Last time I
    tried it didn't work well but that was several versions ago.

    There are problems daemonizing code that wasn't written to stay in memory.
    Global variables are assumed to be empty, unused memory is not released,
    etc. (folks who port stuff to mod_perl are very familiar with the issues).
    Perperl uses a few tricks to limit the effects of this, but it's not as
    good as code designed from the start to run as a daemon.

    Sanitizer may not have these issues... only real way to know is to try it.
    With a compiled sanitizer, design of the code is much less of an issue
    (assuming you can get perlcc to work at all ;-) There are commercial perl
    compilers available that work better than perlcc)

    --
     Derrick Webber
     Advosys Consulting Inc. Ottawa
     http://advosys.ca/
    -- 
    



    hosted by molar.is