• too many open files

    From Ragnarok@docksud.com.ar to DOVE-Net.Synchronet_Discussion on Thu Feb 9 14:03:50 2017
    Hi.
    I notice that sbbs get this error sometimes. I now increase the file-max
    from sysctl i will check during this week.but i think that no will solve finnaly the issue.

    Maybe sbbs forgot close file desciptors in some place?
    I detect this when reading via nntp, and something sbbs cannot open the
    areas databases
    =|

    open files by sbbs:
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs |wc -l
    11314


    log related to nntp:


    Feb 9 13:45:26 localhost sbbs[1139]: 2/9 13:45:26 term Node 0 !ERROR 24
    (Too many open files) in logon.cpp line 556 (logonstats) opening "/sbbs/ctrl/dsts.dab" access=2
    Feb 9 13:45:28 localhost sbbs[1139]: 2/9 13:45:28 srvc 0068 NNTP [Dock_Sud_OLD.Informatica] cmd: GROUP Dock_Sud_OLD.Resentimientos
    Feb 9 13:45:28 localhost sbbs[1139]: 2/9 13:45:28 srvc 0068 NNTP !no
    such group
    Feb 9 13:45:28 localhost sbbs[1139]: 2/9 13:45:28 srvc 0068 NNTP [Dock_Sud_OLD.Informatica] cmd: GROUP Dock_Sud_OLD.SEXARG
    Feb 9 13:45:28 localhost sbbs[1139]: 2/9 13:45:28 srvc 0068 NNTP !no
    such group
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0068 NNTP [Dock_Sud_OLD.Informatica] cmd: GROUP Dock_Sud_OLD.Servinet
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0064 NNTP [Dock_Sud_OLD.Hello] cmd: GROUP Dock_Sud_OLD.red_catamarca
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP DOVE-Net.HAM_Radio
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0066 NNTP [Dock_Sud_OLD.Fulbo] cmd: GROUP DOVE-Net.Internet
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0064 NNTP !no
    such group
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0068 NNTP !no
    such group
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0066 NNTP !no
    such group
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP DOVE-Net.Programming
    Feb 9 13:45:29 localhost sbbs[1139]: 2/9 13:45:29 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:31 localhost sbbs[1139]: 2/9 13:45:31 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP DOVE-Net.Sports
    Feb 9 13:45:31 localhost sbbs[1139]: 2/9 13:45:31 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:31 localhost sbbs[1139]: 2/9 13:45:31 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP DOVE-Net.Synchronet_Announcements
    Feb 9 13:45:31 localhost sbbs[1139]: 2/9 13:45:31 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:32 localhost sbbs[1139]: 2/9 13:45:32 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP DOVE-Net.Synchronet_Discussion
    Feb 9 13:45:32 localhost sbbs[1139]: 2/9 13:45:32 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:33 localhost sbbs[1139]: 2/9 13:45:33 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP DOVE-Net.Pro-Audio
    Feb 9 13:45:33 localhost sbbs[1139]: 2/9 13:45:33 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:34 localhost sbbs[1139]: 2/9 13:45:34 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP DOVE-Net.Sync_JavaScript
    Feb 9 13:45:34 localhost sbbs[1139]: 2/9 13:45:34 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:34 localhost sbbs[1139]: 2/9 13:45:34 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP Docksud.General
    Feb 9 13:45:34 localhost sbbs[1139]: 2/9 13:45:34 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:34 localhost sbbs[1139]: 2/9 13:45:34 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP Docksud.Notices
    Feb 9 13:45:34 localhost sbbs[1139]: 2/9 13:45:34 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:34 localhost sbbs[1139]: 2/9 13:45:34 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP Fidonet.BINKD
    Feb 9 13:45:34 localhost sbbs[1139]: 2/9 13:45:34 srvc 0062 NNTP !no
    such group
    Feb 9 13:45:35 localhost sbbs[1139]: 2/9 13:45:35 srvc 0062 NNTP [Dock_Sud_OLD.Hacking] cmd: GROUP comp.os.msdos.misc
    Feb 9 13:45:35 localhost sbbs[1139]: 2/9 13:45:35 srvc 0062 NNTP !no
    such group
    :


    log relate to node file:

    root@scarlet:/etc/sysctl.d# grep "open files" /var/log/syslog
    Feb 9 13:44:26 localhost sbbs[1139]: 2/9 13:44:26 term Node 0 !ERROR 24
    (Too many open files) in logon.cpp line 556 (logonstats) opening "/sbbs/ctrl/dsts.dab" access=2
    Feb 9 13:45:26 localhost sbbs[1139]: 2/9 13:45:26 term Node 0 !ERROR 24
    (Too many open files) in logon.cpp line 556 (logonstats) opening "/sbbs/ctrl/dsts.dab" access=2
    Feb 9 13:46:26 localhost sbbs[1139]: 2/9 13:46:26 term Node 0 !ERROR 24
    (Too many open files) in logon.cpp line 556 (logonstats) opening "/sbbs/ctrl/dsts.dab" access=2
    Feb 9 13:47:26 localhost sbbs[1139]: 2/9 13:47:26 term Node 0 !ERROR 24
    (Too many open files) in logon.cpp line 556 (logonstats) opening "/sbbs/ctrl/dsts.dab" access=2
    Feb 9 13:48:26 localhost sbbs[1139]: 2/9 13:48:26 term Node 0 !ERROR 24
    (Too many open files) in logon.cpp line 556 (logonstats) opening "/sbbs/ctrl/dsts.dab" access=2
    Feb 9 13:49:10 localhost sbbs[1139]: 2/9 13:49:10 term Node 0 !ERROR 24
    (Too many open files) in pack_rep.cpp line 86 (pack_rep) creating "/sbbs/temp/event/DIGDIST.MSG" access=577
    Feb 9 13:49:10 localhost sbbs[1139]: 2/9 13:49:10 term Node 0 !ERROR 24
    (Too many open files) in main.cpp line 2761 (event_thread) opening "/sbbs/ctrl/qnet.dab" access=1
    Feb 9 13:49:26 localhost sbbs[1139]: 2/9 13:49:26 term Node 0 !ERROR 24
    (Too many open files) in logon.cpp line 556 (logonstats) opening "/sbbs/ctrl/dsts.dab" access=2
    Feb 9 13:50:26 localhost sbbs[1139]: 2/9 13:50:26 term Node 0 !ERROR 24
    (Too many open files) in logon.cpp line 556 (logonstats) opening "/sbbs/ctrl/dsts.dab" access=2
  • From Ragnarok@docksud.com.ar to Ragnarok on Thu Feb 9 14:12:26 2017
    when start sbbs this opens a 1400 files
    when open nntp client its up to 16000

    systemctl start sbbsroot@scarlet:/etc/sysctl.d# systemctl start sbbs root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    1425
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    1409
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    1425

    <start read via nntp>

    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    12760
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    17881
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    18508
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    18508
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    18508
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    16039
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    16039
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    16039
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l

    its normal?
  • From Ragnarok@docksud.com.ar to Ragnarok on Thu Feb 9 14:15:54 2017
    wow!

    when close the thunderbird y get low open files again

    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    1548

    i have ~ 300 areas that opening 16000 files...
    =|
  • From Digital Man@VERT to Ragnarok on Thu Feb 9 15:54:55 2017
    Re: too many open files
    By: Ragnarok to DOVE-Net.Synchronet_Discussion on Thu Feb 09 2017 02:03 pm

    Hi.
    I notice that sbbs get this error sometimes. I now increase the file-max from sysctl i will check during this week.but i think that no will solve finnaly the issue.

    Maybe sbbs forgot close file desciptors in some place?
    I detect this when reading via nntp, and something sbbs cannot open the areas databases
    =|

    open files by sbbs:
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs |wc -l
    11314

    By default, lsof outputs sockets as well as "files". To just get a list of files, use "lsof /".

    digital man

    Synchronet "Real Fact" #64:
    Synchronet PCMS (introduced w/v2.0) is Programmable Command and Menu Structure. Norco, CA WX: 73.6øF, 62.0% humidity, 8 mph ESE wind, 0.01 inches rain/24hrs ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net
  • From Digital Man@VERT to Ragnarok on Thu Feb 9 15:55:38 2017
    Re: Re: too many open files
    By: Ragnarok to Ragnarok on Thu Feb 09 2017 02:12 pm

    when start sbbs this opens a 1400 files
    when open nntp client its up to 16000

    systemctl start sbbsroot@scarlet:/etc/sysctl.d# systemctl start sbbs root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    1425
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    1409
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    1425

    <start read via nntp>

    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    12760
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    17881
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    18508
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    18508
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    18508
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    16039
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    16039
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    16039
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l

    its normal?

    I think you're conflating sockets with files. It might be normal for hundreds of TCP sockets to be opened by your NNTP client.

    digital man

    Synchronet "Real Fact" #55:
    Synchronet Terminal Server introduced RLogin support w/v3.00c (2000).
    Norco, CA WX: 73.6øF, 62.0% humidity, 8 mph ESE wind, 0.01 inches rain/24hrs ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net
  • From Digital Man@VERT to Ragnarok on Thu Feb 9 15:55:59 2017
    Re: Re: too many open files
    By: Ragnarok to Ragnarok on Thu Feb 09 2017 02:15 pm

    wow!

    when close the thunderbird y get low open files again

    root@scarlet:/etc/sysctl.d# lsof|grep sbbs|wc -l
    1548

    i have ~ 300 areas that opening 16000 files...
    =|

    Are sure it's "files" and not "sockets"?

    digital man

    Synchronet "Real Fact" #25:
    The Digital Dynamics company ceased day-to-day opperations in late 1995.
    Norco, CA WX: 73.6øF, 62.0% humidity, 8 mph ESE wind, 0.01 inches rain/24hrs ---
    þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net
  • From mark lewis@VERT to Digital Man on Fri Feb 10 07:33:50 2017
    On 2017 Feb 09 15:54:54, you wrote to Ragnarok:

    open files by sbbs:
    root@scarlet:/etc/sysctl.d# lsof|grep sbbs |wc -l
    11314

    By default, lsof outputs sockets as well as "files". To just get a
    list of files, use "lsof /".

    when i went researching and digging after reading the initial post, i found a page that basically said to find out what the PID of the program in question is
    and then use that on the lsof command line... on max's system i ran the following...


    ps -C sbbs -o pid=


    which gave me the PID i needed so that now gets fed to lsof...


    sudo lsof -p $(ps -C sbbs -o pid=)


    which gave me only what the sbbs task was using... from there, one might filter
    for lines containing a '/' but that may not be what one wants... in my research, i found a page on lsof for AIX systems that uses "-t type" to list only items of that type... on my ubuntu system, the lines with file names are of type "REG" (regular file) but i don't find a command line option to output only those lines... so grep to the rescue in the end...


    sudo lsof -p $(ps -C sbbs -o pid=) | grep -e " REG "


    and i have only the open files... i've added the header for clarity of the fields...

    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sbbs 902 sbbs txt REG 253,1 180592 1068949 /sbbs/src/sbbs3/gcc.linux.x64.exe.release/sbbs
    sbbs 902 sbbs mem REG 253,1 43616 670420 /lib/x86_64-linux-gnu/libnss_files-2.19.so
    sbbs 902 sbbs mem REG 253,1 47760 670412 /lib/x86_64-linux-gnu/libnss_nis-2.19.so
    sbbs 902 sbbs mem REG 253,1 97296 670408 /lib/x86_64-linux-gnu/libnsl-2.19.so
    sbbs 902 sbbs mem REG 253,1 39824 670407 /lib/x86_64-linux-gnu/libnss_compat-2.19.so
    sbbs 902 sbbs mem REG 253,1 31792 670416 /lib/x86_64-linux-gnu/librt-2.19.so
    sbbs 902 sbbs mem REG 253,1 14664 670404 /lib/x86_64-linux-gnu/libdl-2.19.so
    sbbs 902 sbbs mem REG 253,1 90160 655389 /lib/x86_64-linux-gnu/libgcc_s.so.1
    sbbs 902 sbbs mem REG 253,1 1071552 665000 /lib/x86_64-linux-gnu/libm-2.19.so
    sbbs 902 sbbs mem REG 253,1 979056 1793 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
    sbbs 902 sbbs mem REG 253,1 239440 1342 /usr/lib/x86_64-linux-gnu/libnspr4.so
    sbbs 902 sbbs mem REG 253,1 10680 670419 /lib/x86_64-linux-gnu/libutil-2.19.so
    sbbs 902 sbbs mem REG 253,1 1840928 670417 /lib/x86_64-linux-gnu/libc-2.19.so
    sbbs 902 sbbs mem REG 253,1 141574 670409 /lib/x86_64-linux-gnu/libpthread-2.19.so
    sbbs 902 sbbs mem REG 253,1 99085 1068947 /sbbs/src/sbbs3/gcc.linux.x64.lib.release/libservices.so
    sbbs 902 sbbs mem REG 253,1 158767 1068946 /sbbs/src/sbbs3/gcc.linux.x64.lib.release/libmailsrvr.so
    sbbs 902 sbbs mem REG 253,1 152444 1068948 /sbbs/src/sbbs3/gcc.linux.x64.lib.release/libwebsrvr.so
    sbbs 902 sbbs mem REG 253,1 126447 1068771 /sbbs/src/sbbs3/gcc.linux.x64.lib.release/libftpsrvr.so
    sbbs 902 sbbs mem REG 253,1 7373462 1061592 /sbbs/src/sbbs3/gcc.linux.x64.lib.release/libsbbs.so
    sbbs 902 sbbs mem REG 253,1 18952 655531 /lib/x86_64-linux-gnu/libcap.so.2.24
    sbbs 902 sbbs mem REG 253,1 149120 670410 /lib/x86_64-linux-gnu/ld-2.19.so
    sbbs 902 sbbs 10uR REG 253,1 871 1059009 /sbbs/ctrl/cryptlib.key
    sbbs 902 sbbs 15u REG 253,1 150 1058387 /sbbs/ctrl/node.dab
    sbbs 902 sbbs 17u REG 253,1 150 1058387 /sbbs/ctrl/node.dab


    sorry for the wrap... there's nine columns ending with the full path and file name...

    i notice that node.dab is opened twice... maybe there's something to that with the problem we see on Max's system about not being able to lock it or whatever that error is that we've been seeing?

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... WYMI - the all-philosophy radio station.
    ---
    * Origin: (1:3634/12.73)
    þ Synchronet þ Vertrauen þ Home of Synchronet þ telnet://vert.synchro.net