OS Bug/Feature preventing ASTsend from starting

All installation and configuration problems and questions

Moderators: gerski, enjay, williamconley, Op3r, Staydog, gardo, mflorell, MJCoate, mcargile, Kumba, Michael_N

OS Bug/Feature preventing ASTsend from starting

Postby Cyberwolf2 » Mon Sep 16, 2024 3:34 pm

Hey 8)

I've been debugging this for weeks on a scratch install on Ubuntu 24.04 running on a MacPro 3,1 and can't get ASTsend screen to activate

Hear me out, I think I found the problem and it has to do on how the script looks for the listen screen

Here is the perl script section that checks for the listen screen from AST_manager_send.pl :

Code: Select all
                        #my @psoutput = `/bin/ps -f --no-headers -A`;
                        my @psoutput = `/bin/ps -o "%p %a" --no-headers -A`;
                        foreach my $line (@psoutput)
                                {
                                chomp($line);
                                print "|$line|     \n" if ($DBX);
                                my @psline = split(/\/usr\/bin\/perl /,$line);
                                if ($psline[1] =~ /AST_manager_li/)
                                        {
                                        $running_listen++;
                                        print "SEND RUNNING: |$psline[1]|\n" if ($DB);
                                        }
                                }


From what I understand is that it gets the output of

Code: Select all
/bin/ps -o "%p %a" --no-headers -A


    1. it works on each line in the "foreach" function
    2. splits "/usr/bin/perl" from the string if any, the rest of the string is stored on psline[1], a string array
    3. tries to match psline[1] with "~ /AST_manager_li/"

here is the correct output on a running system: /bin/ps -o "%p %a" --no-headers -A | grep perl (credit to capernox)

Image

My /bin/ps -o "%p %a" --no-headers -A | grep perl:

Code: Select all
root@macpro:~# /bin/ps -o "%p %a" --no-headers -A | grep perl
   1746 /usr/bin/perl /usr/share/as
   1747 /usr/bin/perl /usr/share/as
   1796 /usr/bin/perl /usr/share/as
   1797 /usr/bin/perl /usr/share/as
   1798 /usr/bin/perl /usr/share/as
   1799 /usr/bin/perl /usr/share/as
   1800 /usr/bin/perl /usr/share/as
   2030 /usr/bin/perl /usr/share/as
  60296 /usr/bin/perl /usr/share/as
  75952 /usr/bin/perl /usr/share/as
  75954 /usr/bin/perl /usr/share/as
  75990 /usr/bin/perl /usr/share/as
  75995 /usr/bin/perl /usr/share/as
  76000 /usr/bin/perl /usr/share/as
  76051 grep --color=auto perl


The problem: My ps binary is truncating the output and the script can not verify if the listen screen is running.

Potential cause - ps or it's linked libraries have a bug/feature not existent in previous versions. Here are my ps dependancies and versions:

... maybe it's one of these suckers

dd --verbose /usr/bin/ps

Code: Select all
root@macpro:/usr/share/astguiclient# ldd --verbose /usr/bin/ps
   linux-vdso.so.1 (0x00007ffe8c7bf000)
   libproc2.so.0 => /lib/x86_64-linux-gnu/libproc2.so.0 (0x00007188cb855000)
   libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007188cb600000)
   libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007188cb520000)
   /lib64/ld-linux-x86-64.so.2 (0x00007188cb8db000)
   libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007188cb848000)
   libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007188cb3d8000)
   liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007188cb824000)
   liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007188cb3a6000)
   libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007188cb2ec000)
   libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007188cb2c7000)

   Version information:
   /usr/bin/ps:
      libproc2.so.0 (LIBPROC_2) => /lib/x86_64-linux-gnu/libproc2.so.0
      libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.33) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.38) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.32) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.34) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
   /lib/x86_64-linux-gnu/libproc2.so.0:
      ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
      libsystemd.so.0 (LIBSYSTEMD_209) => /lib/x86_64-linux-gnu/libsystemd.so.0
      libc.so.6 (GLIBC_2.8) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.33) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.34) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.38) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
   /lib/x86_64-linux-gnu/libc.so.6:
      ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
      ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
      ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
   /lib/x86_64-linux-gnu/libsystemd.so.0:
      liblzma.so.5 (XZ_5.0) => /lib/x86_64-linux-gnu/liblzma.so.5
      ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
      libgcrypt.so.20 (GCRYPT_1.6) => /lib/x86_64-linux-gnu/libgcrypt.so.20
      libc.so.6 (GLIBC_2.35) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.30) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.28) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.25) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.27) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.8) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.33) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.38) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.5) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.26) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.17) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.32) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.16) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.7) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.34) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.9) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
   /lib/x86_64-linux-gnu/libcap.so.2:
      libc.so.6 (GLIBC_2.9) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.7) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.8) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.33) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.38) => /lib/x86_64-linux-gnu/libc.so.6
   /lib/x86_64-linux-gnu/libgcrypt.so.20:
      libgpg-error.so.0 (GPG_ERROR_1.0) => /lib/x86_64-linux-gnu/libgpg-error.so.0
      libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.33) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.25) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.38) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
   /lib/x86_64-linux-gnu/liblz4.so.1:
      libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
   /lib/x86_64-linux-gnu/liblzma.so.5:
      libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.32) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.34) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.6) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.17) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
   /lib/x86_64-linux-gnu/libzstd.so.1:
      libc.so.6 (GLIBC_2.34) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
   /lib/x86_64-linux-gnu/libgpg-error.so.0:
      libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.32) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.34) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.38) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
      libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6




My question: How do I get my system's ps to quit truncating the output of /bin/ps -o "%p %a" --no-headers -A on Ubuntu 24.04 ?

vici SVN: 3874

Thanks a million :D


in before:
uncommenting #my @psoutput = `/bin/ps -f --no-headers -A`; from AST_manager_send.pl won't work with out further script modification, but this might be a hot fix, tho a bandaid. working on it ;)
Last edited by Cyberwolf2 on Tue Sep 17, 2024 1:29 am, edited 5 times in total.
Scratch install Ubuntu 24.04, Asterisk 18.18.1, AstGUIclient: (VERSION: 2.14-928a, BUILD: 240826-0918, SVN: 3874), MacPro 3,1 21GB, VOIP Provider: Skyetel, NGINX, Self Hosted + WireGuard tunnel
Cyberwolf2
 
Posts: 29
Joined: Mon May 10, 2021 7:59 pm

Re: OS Bug/Feature preventing ASTsend from starting

Postby Cyberwolf2 » Mon Sep 16, 2024 6:43 pm

Offending hardcoded limit: https://gitlab.com/procps-ng/procps/-/blame/v4.0.4/src/ps/output.c#L1655 some dev added a limit to 27 characters to the command option aka "%a". It was 8 years ago, has anyone run into this?

here is my version:

root@macpro:~# ps -V
ps from procps-ng 4.0.4
Scratch install Ubuntu 24.04, Asterisk 18.18.1, AstGUIclient: (VERSION: 2.14-928a, BUILD: 240826-0918, SVN: 3874), MacPro 3,1 21GB, VOIP Provider: Skyetel, NGINX, Self Hosted + WireGuard tunnel
Cyberwolf2
 
Posts: 29
Joined: Mon May 10, 2021 7:59 pm

Re: OS Bug/Feature preventing ASTsend from starting

Postby Cyberwolf2 » Tue Sep 17, 2024 12:15 am

FIXED! Here is what I did:

Each flavor of linux has it's own patches, from preferences and beliefs to bugs. REHL and Debian based flavors behave a little differently and this is just how it is. So here is my thoughts on a bandaid to get it to work:

This conclusion is according to the Reporting Bugs section at https://gitlab.com/procps-ng/procps/-/blob/master/README.md

Method

    1. Keep Ubuntu's 24.04 /bin/ps as is
    2. Download, patch and Compile the latest version of /bin/ps aka procps
    3. install compiled ps to a different destination
    4. backup and update all scripts at once in /usr/share/astguiclient with the new ps binary path
    5. Clean /usr/share/astguiclient by moving backed up pre patch files to /usr/share/astguiclient_bak

1. Keep Ubuntu's /bin/ps as is

Let's not mess with the distro's install /bin/ps binary, leave it be

2. Download, patch and Compile the latest version of /bin/ps aka procps

Download
Code: Select all
mkdir -p /usr/src/procps
cd /usr/src/procps
git clone https://gitlab.com/procps-ng/procps.git
nano src/ps/output.c


Patch
    location might be different, version I have is 4.0.4 . If you are having issues here find output.c in the repository and adjust this command.
    press SHIFT+CONTROL+"-" keys to pull up goto line
    for this version it's in line 1655, type it in.
    for others press CTL+W to go text search and type in "COMMAND" with the ""
    keep pressing CTL+W to go to the next line until you see a table
    on the left column it needs to be "args", this is the settings table, note the column name .wigth. Let's raise the limit from 27 to what ever you like 450
    CTL+O to save
    CTL+X to close

Example
In this screen shot I set it to 100, but I went back and updated to 450. I believe this is the limit in REHL.
Image

Compile and Install to a different place
Code: Select all
./autogen.sh
mkdir /usr/src/procps/altinstall
./configure --prefix=/usr/src/procps/altinstall
make
make install


backup and update all scripts at once in /usr/share/astguiclient


This code back ups all scripts, finds every instance in the directory with the string "/bin/ps" and replaces it with "/usr/src/procps/altinstall/bin/ps"
Code: Select all
find /usr/share/astguiclient -type f -exec sed -i.bak 's/\/bin\/ps/\/usr\/src\/procps\/altinstall\/bin\/ps/g' {} \;


Clean up a little bit
Code: Select all
mkdir /usr/share/astguiclient_bak
mv -r /usr/share/astguiclient/*.bak /usr/share/astguiclient_bak


Give yourself a pat on your back and reboot for good measure

Special thanks to capernox :D

UPDATE: NOT ENTIRELY WORKING!

I still have a time synchronization problem. While the above fixed /bin/ps, I still am hunting down why time is still out of sync.

Question: which script calls agent's meetme conference?

This Mac doesn't play well with REHL. CPU is old and doesn't support modern REHL kernel use of x86-64-v2 instruction set. I'm almost sure recompiling the kernel and re imaging a REHL distro is harder than this mission...
Scratch install Ubuntu 24.04, Asterisk 18.18.1, AstGUIclient: (VERSION: 2.14-928a, BUILD: 240826-0918, SVN: 3874), MacPro 3,1 21GB, VOIP Provider: Skyetel, NGINX, Self Hosted + WireGuard tunnel
Cyberwolf2
 
Posts: 29
Joined: Mon May 10, 2021 7:59 pm

Re: OS Bug/Feature preventing ASTsend from starting

Postby williamconley » Fri Sep 20, 2024 8:42 pm

Special thanks to carpenox

Carpenox is cool. True statement.

We're going to be playing in 24.04 LTS shortly and may be able (or ... required? LOL) to shed some light on this. If you're curious ping me in a couple months and see if we "got there yet".

What time sync system are you using? What Servers are you trying to sync to? And what country is your "time sync request" emanating from?
Vicidial Installation and Repair, plus Hosting and Colocation
Newest Product: Vicidial Agent Only Beep - Beta
http://www.PoundTeam.com # 352-269-0000 # +44(203) 769-2294
williamconley
 
Posts: 20258
Joined: Wed Oct 31, 2007 4:17 pm
Location: Davenport, FL (By Disney!)

Re: OS Bug/Feature preventing ASTsend from starting

Postby Cyberwolf2 » Fri Sep 27, 2024 1:14 am

It's the database, this is the only constant.

Fixed it, I simply reloaded the base install database, and added everything via the web interface and not phpMyAdmin

working now...

I wasted about a month on this :oops:

Steeeeep learning curve ...
Scratch install Ubuntu 24.04, Asterisk 18.18.1, AstGUIclient: (VERSION: 2.14-928a, BUILD: 240826-0918, SVN: 3874), MacPro 3,1 21GB, VOIP Provider: Skyetel, NGINX, Self Hosted + WireGuard tunnel
Cyberwolf2
 
Posts: 29
Joined: Mon May 10, 2021 7:59 pm

Re: OS Bug/Feature preventing ASTsend from starting

Postby williamconley » Fri Sep 27, 2024 9:25 pm

LOL:

Good postback, though.

And never use phpMyAdmin for bulk load. Always CLI.
Vicidial Installation and Repair, plus Hosting and Colocation
Newest Product: Vicidial Agent Only Beep - Beta
http://www.PoundTeam.com # 352-269-0000 # +44(203) 769-2294
williamconley
 
Posts: 20258
Joined: Wed Oct 31, 2007 4:17 pm
Location: Davenport, FL (By Disney!)


Return to Support

Who is online

Users browsing this forum: Google [Bot] and 69 guests