anomy-list

Re: The QP encoding issue - again!

From: Dustin Puryear (101286@xyz.molar.is)
Date: Fri 14 Nov 2003 - 01:15:39 GMT

  • Next message: Dustin Puryear: "Re: The QP encoding issue - again!"

    Hmm, on second though this won't fix the real problem. I was
    misunderstanding the true cause of this issue. Has someone created a HOWTO
    for solutions on this? It looks like we need to go with the wrapper script
    solution, unless there is a fix coming through for Anomy?

    ----- Original Message -----
    From: "Dustin Puryear" <101286@xyz.molar.is>
    To: "Noel Clarkson" <101374@xyz.molar.is>;
    <101323@xyz.molar.is>
    Sent: Thursday, November 13, 2003 11:27 AM
    Subject: Re: [anomy-list]: The QP encoding issue - again!

    > Here is my initial idea. Let me know what you think.
    >
    > I configure Sanitizer.pm to look for the new configuration parameter
    > "qpe_special_files", which works list a normal file_list. So a
    configuration
    > may be:
    >
    > bin/sanitizer.pl 'qpe_special_files = (?)\.pdf' < test.txt
    >
    > Next, in SanitizeFile I check for the filename:
    >
    > ---><----
    > sub SanitizeFile
    > ...
    >
    > # XX -
    > # qpencoding_special_files
    > print "OrigFilename = $ofn\n";
    > if ($ofn =~ $conf->{"qpe_special_files"}) {
    > $part->{"is_qpe_special_file"} = 1;
    > print "Yes, it is a PDF\n";
    > print "Setting is_qpe_special_file = 1\n";
    > }
    > else {
    > $part->{"is_qpe_special_file"} = 0;
    > print "No, it is not a PDF\n";
    > print "Setting is_qpe_special_file = 0\n";
    > }
    >
    > # Check policies...
    > my @rules = (0..$conf->{"file_list_rules"}, "default");
    > my $rec = 0;
    > my $ip = 1;
    > my $mysize = 0;
    > ---><----
    >
    > This just sets a flag for later on in $reader that will tell
    > MIMEStream.pm::RawRead() that we are reading a quoted-printable encoded
    file
    > that we know is actually binary. (Thanks Outlook.)
    >
    > In MIMEStream.pm I modified RawRead(). The big change is that I will only
    do
    > the newline code if $reader->{"is_qpe_special_file"} is false:
    >
    > ---><----
    > sub RawRead
    > ...
    >
    > my $line;
    > if (!$reader->{"INforce"})
    > {
    > # Slurp at most 8k of input at a time.
    > $line = $reader->{"parent"}->Read(8192);
    > }
    > else
    > {
    > # Slurp 8k of input at a time.
    > $reader->{"IN"}->read($line, 8192);
    >
    > # XX
    > if (defined $reader->{"newline_in"} &&
    > !$reader->{"is_qpe_special_file"})
    > {
    > # Figure out what our newline convention is...
    > my $nl = $reader->{"newline_in"};
    > if ((!$nl) && ($line =~ /(\015?\012)/s))
    > {
    > my $writer = $reader->{"writer"} || $reader;
    > $reader->{"newline_in"} = $nl = $1;
    > $writer->{"newline_out"} = $nl
    > unless ($writer->{"newline_out"});
    > }
    >
    > $line =~ s/(\015?\012)/\012/gs; # if ($nl);
    > }
    > }
    >
    > ---><----
    >
    > This just ensures that MIMEStream.pm doesn't do any newline conversions
    when
    > it is reading a known-binary file that is quoted-printable encoded.
    >
    > What are your thoughts on this kind of solution?
    >
    >
    > ----- Original Message -----
    > From: "Noel Clarkson" <101374@xyz.molar.is>
    > To: "Dustin Puryear" <101419@xyz.molar.is>
    > Cc: <101323@xyz.molar.is>
    > Sent: Tuesday, November 11, 2003 6:42 PM
    > Subject: Re: [anomy-list]: The QP encoding issue - again!
    >
    >
    > > Hi there,
    > >
    > > a while back the suggested way to go with this seemed to be to save the
    > > pdf, scan it for viruses, if the scan comes up clean then leave the
    > > orriginal pdf in place, if not then do something else with the pdf. I
    > > was looking into doing this and writing a bit of a description on how to
    > > do it but it kind of fell by the wayside and I've got too many things to
    > > do to pick it up now. If you think this might be the way to solve your
    > > problem, have a look back through the archives, below is a quote out of
    > > one of the mail messages that sums up what the idea is.
    > >
    > > From an email around May 2003 on the Anomy List:
    > > > > >
    > > > > > Currently the only way to avoid this problem is to wrap the
    > > > > > sanitizer in a procmail ruleset or helper script which will
    > > > > > quarantine the original unmodified message and use it verbatim if
    > > > > > the sanitizer doesn't find any security risks in it.
    > > > >
    > > > > I could do that, as I always write incomming message to disk before
    > > > > sending to sanitizer, then amavis adn another script for loging
    > > > > attachment in DB.
    > > > >
    > > > > But how could I know if sanitizer hit something or no, I desn't see
    a
    > > > > particular exitcode using $?
    > > > >
    > > > I didn't see the score_bad option and the "!" possible on the policy
    to
    > > > increment the score. I now use this to get an exitcode != 0 and got a
    > > > new working script which send the original message if the exit code
    is
    > > > 0.
    > > >
    > > > It just needs a little bit more disk io ;-// but it works.
    > > >
    > >
    > >
    > >
    > > If you do get something to work, then I'm sure others would appreciate
    > > if you can post a description so that we all don't have to work it out
    > > step by step.
    > >
    > > cheers,
    > >
    > > noel
    > >
    > >
    > >
    > > Dustin Puryear wrote:
    > > > Precisely. Because binary files, i.e., a PDF, is being encoded using
    QP
    > > > encoding Anomy is garbling them. (Not that Anomy is buggy. It's the
    mail
    > > > clients that are buggy.) If you hit the Anomy archives you will
    > understand
    > > > the problem.
    > > >
    > > > The actual plan right now is to either undo the changes made by Anomy,
    > or to
    > > > simply stop Anomy from making the changes in the first place. I'm
    simply
    > > > hoping someone has already modified and tested these changes in Anomy.
    > > >
    > > > ----- Original Message -----
    > > > From: "Marvin Herbold" <101461@xyz.molar.is>
    > > > To: "Dustin Puryear" <101286@xyz.molar.is>
    > > > Cc: <101323@xyz.molar.is>
    > > > Sent: Monday, November 10, 2003 7:52 PM
    > > > Subject: Re: [anomy-list]: The QP encoding issue - again!
    > > >
    > > >
    > > >
    > > >>Umm - aren't PDF files binary? Simply converting LF to CRLF would
    > > >>royally screw up these files... like in places where there is supposed
    > > >>to be a 0x0A byte without a preceeding 0x0D byte.
    > > >>
    > > >>Dustin Puryear wrote:
    > > >>
    > > >>
    > > >>>I'm glad you mentioned that. That is actually the plan of attack that
    > we
    > > >>>decided on, but for some reason I didn't list it. If someone has
    > already
    > > >>>done this then let us know. I would hate to reproduce the effort.
    > Thanks!
    > > >>>
    > > >>>Reports suggest that this happens mostly with some graphics files and
    > > >>
    > > > PDFs.
    > > >
    > > >>>Anyone have experience with this on other files?
    > > >>>
    > > >>>
    > > >>
    > > >>--
    > > >>Marvin Herbold
    > > >>101461@xyz.molar.is
    > > >>http://www.herbold-family.com
    > > >>
    > > >>
    > > >>
    > > >>
    > > >>
    > > >
    > > >
    > > >
    > >
    > >
    > >
    > >
    > >
    >
    >
    >
    >



    hosted by molar.is