anomy-list

Re: The QP encoding issue - again!

From: Dustin Puryear (101065@xyz.molar.is)
Date: Thu 13 Nov 2003 - 17:27:46 GMT

  • Next message: Dustin Puryear: "Re: The QP encoding issue - again!"

    Here is my initial idea. Let me know what you think.

    I configure Sanitizer.pm to look for the new configuration parameter
    "qpe_special_files", which works list a normal file_list. So a configuration
    may be:

    bin/sanitizer.pl 'qpe_special_files = (?)\.pdf' < test.txt

    Next, in SanitizeFile I check for the filename:

    ---><----
    sub SanitizeFile
            ...

            # XX -
            # qpencoding_special_files
            print "OrigFilename = $ofn\n";
            if ($ofn =~ $conf->{"qpe_special_files"}) {
                    $part->{"is_qpe_special_file"} = 1;
                    print "Yes, it is a PDF\n";
                    print "Setting is_qpe_special_file = 1\n";
            }
            else {
                    $part->{"is_qpe_special_file"} = 0;
                    print "No, it is not a PDF\n";
                    print "Setting is_qpe_special_file = 0\n";
            }

        # Check policies...
            my @rules = (0..$conf->{"file_list_rules"}, "default");
            my $rec = 0;
            my $ip = 1;
            my $mysize = 0;
    ---><----

    This just sets a flag for later on in $reader that will tell
    MIMEStream.pm::RawRead() that we are reading a quoted-printable encoded file
    that we know is actually binary. (Thanks Outlook.)

    In MIMEStream.pm I modified RawRead(). The big change is that I will only do
    the newline code if $reader->{"is_qpe_special_file"} is false:

    ---><----
    sub RawRead
            ...

            my $line;
            if (!$reader->{"INforce"})
            {
                    # Slurp at most 8k of input at a time.
                    $line = $reader->{"parent"}->Read(8192);
            }
            else
            {
                    # Slurp 8k of input at a time.
                    $reader->{"IN"}->read($line, 8192);

                    # XX
                    if (defined $reader->{"newline_in"} &&
    !$reader->{"is_qpe_special_file"})
                    {
                        # Figure out what our newline convention is...
                        my $nl = $reader->{"newline_in"};
                            if ((!$nl) && ($line =~ /(\015?\012)/s))
                            {
                                my $writer = $reader->{"writer"} || $reader;
                            $reader->{"newline_in"} = $nl = $1;
                                    $writer->{"newline_out"} = $nl
                                      unless ($writer->{"newline_out"});
                            }

                            $line =~ s/(\015?\012)/\012/gs; # if ($nl);
                    }
            }

    ---><----

    This just ensures that MIMEStream.pm doesn't do any newline conversions when
    it is reading a known-binary file that is quoted-printable encoded.

    What are your thoughts on this kind of solution?

    ----- Original Message -----
    From: "Noel Clarkson" <101102@xyz.molar.is>
    To: "Dustin Puryear" <101147@xyz.molar.is>
    Cc: <101189@xyz.molar.is>
    Sent: Tuesday, November 11, 2003 6:42 PM
    Subject: Re: [anomy-list]: The QP encoding issue - again!

    > Hi there,
    >
    > a while back the suggested way to go with this seemed to be to save the
    > pdf, scan it for viruses, if the scan comes up clean then leave the
    > orriginal pdf in place, if not then do something else with the pdf. I
    > was looking into doing this and writing a bit of a description on how to
    > do it but it kind of fell by the wayside and I've got too many things to
    > do to pick it up now. If you think this might be the way to solve your
    > problem, have a look back through the archives, below is a quote out of
    > one of the mail messages that sums up what the idea is.
    >
    > From an email around May 2003 on the Anomy List:
    > > > >
    > > > > Currently the only way to avoid this problem is to wrap the
    > > > > sanitizer in a procmail ruleset or helper script which will
    > > > > quarantine the original unmodified message and use it verbatim if
    > > > > the sanitizer doesn't find any security risks in it.
    > > >
    > > > I could do that, as I always write incomming message to disk before
    > > > sending to sanitizer, then amavis adn another script for loging
    > > > attachment in DB.
    > > >
    > > > But how could I know if sanitizer hit something or no, I desn't see a
    > > > particular exitcode using $?
    > > >
    > > I didn't see the score_bad option and the "!" possible on the policy to
    > > increment the score. I now use this to get an exitcode != 0 and got a
    > > new working script which send the original message if the exit code is
    > > 0.
    > >
    > > It just needs a little bit more disk io ;-// but it works.
    > >
    >
    >
    >
    > If you do get something to work, then I'm sure others would appreciate
    > if you can post a description so that we all don't have to work it out
    > step by step.
    >
    > cheers,
    >
    > noel
    >
    >
    >
    > Dustin Puryear wrote:
    > > Precisely. Because binary files, i.e., a PDF, is being encoded using QP
    > > encoding Anomy is garbling them. (Not that Anomy is buggy. It's the mail
    > > clients that are buggy.) If you hit the Anomy archives you will
    understand
    > > the problem.
    > >
    > > The actual plan right now is to either undo the changes made by Anomy,
    or to
    > > simply stop Anomy from making the changes in the first place. I'm simply
    > > hoping someone has already modified and tested these changes in Anomy.
    > >
    > > ----- Original Message -----
    > > From: "Marvin Herbold" <101240@xyz.molar.is>
    > > To: "Dustin Puryear" <101065@xyz.molar.is>
    > > Cc: <101189@xyz.molar.is>
    > > Sent: Monday, November 10, 2003 7:52 PM
    > > Subject: Re: [anomy-list]: The QP encoding issue - again!
    > >
    > >
    > >
    > >>Umm - aren't PDF files binary? Simply converting LF to CRLF would
    > >>royally screw up these files... like in places where there is supposed
    > >>to be a 0x0A byte without a preceeding 0x0D byte.
    > >>
    > >>Dustin Puryear wrote:
    > >>
    > >>
    > >>>I'm glad you mentioned that. That is actually the plan of attack that
    we
    > >>>decided on, but for some reason I didn't list it. If someone has
    already
    > >>>done this then let us know. I would hate to reproduce the effort.
    Thanks!
    > >>>
    > >>>Reports suggest that this happens mostly with some graphics files and
    > >>
    > > PDFs.
    > >
    > >>>Anyone have experience with this on other files?
    > >>>
    > >>>
    > >>
    > >>--
    > >>Marvin Herbold
    > >>101240@xyz.molar.is
    > >>http://www.herbold-family.com
    > >>
    > >>
    > >>
    > >>
    > >>
    > >
    > >
    > >
    >
    >
    >
    >
    >



    hosted by molar.is