List ecs-isp@2rosenthals.com Arkiverade meddelande #984

Från: "Massimo S." <ecs-isp@2rosenthals.com> Meddelandehuvud
Oavkodat meddelande
Ämne: Re: [eCS-ISP] clamscan issue - directories with a lot of files
Datum: Mon, 4 Nov 2024 09:43:20 +0100
Till: eCS ISP Mailing List <ecs-isp@2rosenthals.com>



Il 02/11/2024 16:37, Steven Levine ha scritto:
In <list-11202333@2rosenthals.com>, on 11/01/24
    at 12:05 PM, "Massimo S." <ecs-isp@2rosenthals.com> said:

HI Massimo,

damn, i just deleted a debug.txt output for mistake.. :-(

Oops.

anyway it was a 4GB file, i don't have it in the bkups,
maybe ZIP is not able to add a so large file

Depends on the version of zip.

and i guess that the issue of "Can't read file ERROR"
is not related to a directory with a lot of files inside
since it appears allmost the times in "big dirs", but
it may happens that clamscan keep on working again in the same dir
e.g. in this case:

X:/weasel/MailRoot/mydomain.com/myemail/OLAYI6.MSG: Can't read file ERROR
X:/weasel/MailRoot/mydomain.com/myemail/OLAZFS.MSG: Can't read file ERROR
X:/weasel/MailRoot/mydomain.com/myemail/OLB25S.MSG: Can't read file ERROR
X:/weasel/MailRoot/mydomain.com/myemail/OLB3TA.MSG: Can't read file ERROR
X:/weasel/MailRoot/mydomain.com/myemail/OLB5K9.MSG: Can't read file ERROR

i'm able to reproduce the debug output
uncompressed it's 5,6GB..  doh  :-)
compressed is 2,6GB

For a file this large, I recommed you use 7z.

i can put it on an ftp
let me know

Not yet.  I busy with some other projects.

when it show the errors i see a lot of these in the output on the screen:
LibClamAV Error: fmap_readpage: pread error: Not enough memory

That message pretty much explains itself.

the VM has 4GB ram assigned (i know /2 see only about 3,3GB) and
vaddresslimit is 3072

That's as good as it's going to get.

Clamscan is a memory hog.  There are two issues - overall memory usage and
fragmentation.

hi,

could that be improved?

I don't seem to have a copy of your clamscan command line.

clamscan --database=X:\usr\local\clamav\share\clamav --tempdir=Y:\temp --scan-archive=no --exclude=MSGLIST.DAT --exclude=DOMAIN.*   --move=X:\quarantine -lZ:\clamav_day.log X:\weasel\MailRoot\mydomain.com

I suspect you are going have to break the clamscan run into smaller chunks
to avoid the memory issues.

there are about 25 mail domains and more than 350 subdirectories..
with the small ones with just one or two emails i'm already doing this

Steven

massimo

Prenumerera: Sändning, Uppsamling, Index.
Stoppa prenumeration
Meddelande till ListMaster