Il 19/04/2021 16:58, Steven Levine ha scritto:
In <list-1821020@2rosenthals.com>, on 04/19/21
at 11:18 AM, "Massimo S." <ecs-isp@2rosenthals.com> said:
Hi Massimo,
thanks, but it seems is not working
Why would you expect it to?
Ian, FWIW, Massimo is trying to do URL matching, whic does allow
wildcards.
Massimo, that said, you are trying to match a URL to a User-Agent name
which is almost never going to work.
What you need to do is match on the bot name in the User-Agent header.
I recommend you start with a logging rule so you can see what the packet
actually contains. Baring typos, you want:
Log-mj12_2-URLs
Comment = "Log HTTP requests for MJ bot",
Destination-Port = "80 443",
Log-Control = Enabled,
Log-Mask = "date time severity message rule source resolved_source prot
packet_data",
Log-Severity = Major,
Hex-String = "<nocase>12bot",
Depth = 200
Rule-Action = Log,
Direction = Incoming
Once you understand the packet you want to match on, you can convert this
into a Deny Rule-Action, tuning the Depth and Log-Mask to optimized the
filter performance.
FWIW, if this were my problem, I would use an observe rule to blacklist
the source.
Massimo, is the really the logged URL:
/referenze.html/.../referenze.html/
It does not look valid to me. If you edited it, don't do that in the
future. I am only willing to try so hard to help you.
Steven
MJ12bot is over with the string and depth you suggested
thanks a lot
i'me trying to use log to understand that stuff
i now have this (similar) issue:
[2021/04/19][00:07:07][05:Major][MSG:][URL-Filter-bot_petalsearch23][SRC:114.119.147.152][SRC:petalbot-114-119-147-152.petalsearch.com][tcp]
0000: 474554202F746573 - 692E7068703F7465 GET /tesi.php?te
0010: 73694F726465723D - 536F727465725F4C siOrder=Sorter_L
0020: 6976656C6C6F4C61 - 7572656126746573 ivelloLaurea&tes
0030: 694469723D415343 - 2674657369506167 iDir=ASC&tesiPag
0040: 653D382048545450 - 2F312E310D0A4163 e=8 HTTP/1.1..Ac
0050: 636570743A207465 - 78742F68746D6C2C cept: text/html,
0060: 6170706C69636174 - 696F6E2F7868746D application/xhtm
0070: 6C2B786D6C2C6170 - 706C69636174696F l+xml,applicatio
0080: 6E2F786D6C3B713D - 302E392C2A2F2A3B n/xml;q=0.9,*/*;
0090: 713D302E380D0A55 - 7365722D4167656E q=0.8..User-Agen
00A0: 743A204D6F7A696C - 6C612F352E302028 t: Mozilla/5.0 (
00B0: 4C696E75783B2041 - 6E64726F69642037 Linux; Android 7
00C0: 2E303B2920417070 - 6C655765624B6974 .0;) AppleWebKit
00D0: 2F3533372E333620 - 284B48544D4C2C20 /537.36 (KHTML,
00E0: 6C696B6520476563 - 6B6F29204D6F6269 like Gecko) Mobi
00F0: 6C65205361666172 - 692F3533372E3336 le Safari/537.36
0100: 2028636F6D706174 - 69626C653B205065 (compatible; Pe
0110: 74616C426F743B2B - 68747470733A2F2F talBot;+https://
0120: 7765626D61737465 - 722E706574616C73 webmaster.petals
0130: 65617263682E636F - 6D2F736974652F70 earch.com/site/p
0140: 6574616C626F7429 - 0D0A416363657074 etalbot)..Accept
0150: 2D4C616E67756167 - 653A20656E2C7A68 -Language: en,zh
0160: 3B713D302E310D0A - 4163636570742D45 ;q=0.1..Accept-E
0170: 6E636F64696E673A - 20677A69702C6465 ncoding: gzip,de
0180: 666C6174650D0A48 - 6F73743A20777777 flate..Host: www
0190: 2E65636F6469636F - 6D2E6E65740D0A43 .tecnicom.com..C
01A0: 616368652D436F6E - 74726F6C3A206D61 ache-Control: ma
01B0: 782D6167653D300D - 0A436F6E6E656374 x-age=0..Connect
01C0: 696F6E3A206B6565 - 702D616C6976650D ion: keep-alive.
01D0: 0A0D0A ...
this rule filter something:
URL-Filter-bot-petal-string1
Comment = "Deny HTTP requests for petalsearch-bot 19-4-2021",
Destination-Port = "80 443",
Protocol = TCP,
Log-Control = Enabled,
Log-Mask = "date time severity message rule source resolved_source prot",
Log-Size = 9990,
Log-Severity = Major,
Depth = 200,
Hex-String = "<nocase>petalbot",
Rule-Action = Deny,
Direction = Incoming
these 2 nothing at all:
URL-Filter-bot-petal-string2
Comment = "Deny HTTP requests for petalsearch-bot2 19-4-2021",
Destination-Port = "80 443",
Protocol = TCP,
Log-Control = Enabled,
Log-Mask = "date time severity message rule source resolved_source prot",
Log-Size = 9990,
Log-Severity = Major,
Depth = 200,
Hex-String = "<nocase>petalsearch",
Rule-Action = Deny,
Direction = Incoming
URL-Filter-bot-petal-string3
Comment = "Deny HTTP requests for petalsearch-bot2 19-4-2021",
Destination-Port = "80 443",
Protocol = TCP,
Log-Control = Enabled,
Log-Mask = "date time severity message rule source resolved_source prot",
Log-Size = 9990,
Log-Severity = Major,
Depth = 200,
Offset = URL,
Hex-String = "*PetalBot*",
Rule-Action = Deny,
Direction = Incoming
thanks
massimo
|