Hello everybody.

Inside each message area config at Synchronet there's a parameter called "Duplicate Checking" where I shoud add a maximum number of "CRCs"

How does that work? What would be the optimum value for this?

Regards,


Flavio

... "WARLOCK! OLHA PRA CÍ! Seguinte... esqueci" - RÂ, 3· Enco
--- MacFidoIP 1.0 (OSX)
* Origin: Hyperion's Orbit - Resisting since 1995! (4:801/189.1)
þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net

Re: Duplicate Checking
By: Flavio Bessa to All on Fri Oct 28 2016 03:53 pm

Hello everybody.

Inside each message area config at Synchronet there's a parameter called "Duplicate Checking" where I shoud add a maximum number of "CRCs"

A CRC is method for computing a checksum or hash of data. So when a new message is received/imported, it is first compared against the data (history of previously posted messages) to see if the new message is actually a duplicate.

The help text for this option in SCFG:

This value is the maximum number of CRCs that will be kept in the
sub-board for duplicate message checking. Once this maximum number of
CRCs is reached, the oldest CRCs will be automatically purged.

How does that work? What would be the optimum value for this?

10000 is a good value.

digital man

Synchronet/BBS Terminology Definition #28:
LF = Line Feed (ASCII 10, Ctrl-J)
Norco, CA WX: 70.3øF, 60.0% humidity, 6 mph SE wind, 0.00 inches rain/24hrs
---
þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net

28 Oct 16 11:53, Digital Man wrote to you:

Inside each message area config at Synchronet there's a parameter
called "Duplicate Checking" where I shoud add a maximum number of
"CRCs"

A CRC is method for computing a checksum or hash of data. So when a
new message is received/imported, it is first compared against the
data (history of previously posted messages) to see if the new message
is actually a duplicate.

one should be aware that they may end up missing monthly rules postings if the postings are exactly the same except for the MSGID and possibly the time stamp... nick boel knows about this as he was missing many automated postings in echos that i moderate... he thought the echos were totally dead because he was not seeing the monthly messages which also serve to show that the echo is still connected... i'm not sure how close the time stamps were in each month's post but they do include the seconds in the time portion... of course the date of the posting and the MSGID serial numbers were most assuredly different in each one...

The help text for this option in SCFG:

This value is the maximum number of CRCs that will be kept in the sub-board for duplicate message checking. Once this maximum number of
CRCs is reached, the oldest CRCs will be automatically purged.

How does that work? What would be the optimum value for this?

10000 is a good value.

for FTNs it would be good to have a "number of days" setting for retaining dupe
CRC values... this because the specification says the serial numbers should not
repeat for three (3) years... 1100 days is just over three years... then you don't have to worry about the actual number of CRCs unless there's a filesize limitation problem to deal with...

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... Indifference error, press any key. Or don't. See if I care.

Digital Man wrote to Flavio Bessa <=-

Inside each message area config at Synchronet there's a parameter called "Duplicate Checking" where I shoud add a maximum number of "CRCs"

A CRC is method for computing a checksum or hash of data. So when a new message is received/imported, it is first compared against the data (history of previously posted messages) to see if the new message is actually a duplicate.

The help text for this option in SCFG:

This value is the maximum number of CRCs that will be kept in the sub-board for duplicate message checking. Once this maximum number of
CRCs is reached, the oldest CRCs will be automatically purged.

How does that work? What would be the optimum value for this?

10000 is a good value.

Clear as water now. Thanks a million!

---
.-----________________--_ ________.--'-`--.____ Hugs from Flavio Bessa \____==================_) \_'===================' Syzo of Saturn's Orbit
- -|__|-.______|=====/ `---' Netmail 4:801/189.1
Live long ` ù._ _ _ _~~~~~| fcbessa@gmail.com
and prosper... `-.__________,' Always UnionNET Addicted!


... "Mas cadˆ o 13??" - Pappires, 1 Encontro UnionNet: TNG
--- MultiMail/Darwin v0.49
þ Synchronet þ Ninho do Abutre 2 BBS - Rio de Janeiro, Brazil

Hey DM,

I'm playing with message packets (trying to debug something), and I know I'm importing a message that would normally be a "duplicate" (same MSGID and potentially same content.)

However, in the message area for this message, I have specifically set "Duplicate Checking" to "Disabled" (SCFG -> Message Areas -> "Group" -> Sub-boards -> "Area" -> Duplicate Checking)

When I import a mail packet, it still not passing duplicate checking:

== LOG ==
Scanning for Inbound Packets...
Scanning secure inbound: /opt/sbbs/fido/inbound/
Importing /opt/sbbs/fido/inbound/086b6b15.pkt (Type 2e, 1.6KB) from 10:3/1 to 10:3/998
PVT_TEST: pvt_test ERROR 1 (smb_addmsg duplicate X-FTN-MSGID: 10:3/1.0 08657817 found in message #4) line 3575 adding message
to pvt_test
PVT_TEST Duplicate message from Clearing Houz (10:3/1) to Deon, subject: Test Reply

Deleting /opt/sbbs/fido/inbound/086b6b15.pkt (from line 6015)
Scanning non-secure inbound: /opt/sbbs/fido/nonsecure/
Duplicate: 1 detected in PVT_TEST
...
Writing 0 areas to ../data/badareas.lst
Deleting /opt/sbbs/ctrl/sbbsecho.bsy (from line 2965)
SBBSecho (PID 25448) exiting with error level 0, Packets(1 imported, 0 sent)
== END LOG ==

Is there somewhere else that duplicate checking is set?


...ëîåï

---
þ Synchronet þ Alterant | an SBBS in Docker on Pi!

Re: Duplicate Checking
By: deon to Digital Man on Thu Feb 03 2022 10:42 pm

Hey DM,

I'm playing with message packets (trying to debug something), and I know I'm importing a message that would normally be a "duplicate" (same MSGID and potentially same content.)

However, in the message area for this message, I have specifically set "Duplicate Checking" to "Disabled" (SCFG -> Message Areas -> "Group" -> Sub-boards -> "Area" -> Duplicate Checking)

When I import a mail packet, it still not passing duplicate checking:

== LOG ==
Scanning for Inbound Packets...
Scanning secure inbound: /opt/sbbs/fido/inbound/
Importing /opt/sbbs/fido/inbound/086b6b15.pkt (Type 2e, 1.6KB) from 10:3/1 to 10:3/998
PVT_TEST: pvt_test ERROR 1 (smb_addmsg duplicate X-FTN-MSGID: 10:3/1.0 08657817 found in message #4) line 3575 adding message to pvt_test PVT_TEST Duplicate message from Clearing Houz (10:3/1) to Deon, subject: Test Reply

Is there somewhere else that duplicate checking is set?

There is no way to disable duplicate message-ID checking. The help text in SCFG is wrong in this regard and I'll fix that.
--
digital man (rob)

Breaking Bad quote #25:
Now if I could only learn how to lick myself. - Hank Schrader
Norco, CA WX: 60.7øF, 20.0% humidity, 4 mph W wind, 0.00 inches rain/24hrs
---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net