[ASTERISK-83] res_pjsip_device_features: Add forwarding and DND synchronization
Noticed some excessive refcount issues, and upon decreasing the threshold to under 1,000 and running some tests, it is caused by res_pjsip_device_features
, not res_pjsip_sca
or any other modules.
This should be resolved before [ASTERISK-83]
/var/log/asterisk/refs1.txt
and /var/log/asterisk/refs1_proc.txt
on polypbx
contains a ref debug which can be used as a starting point for debugging when time permits.
[Oct 11 15:02:01] NOTICE[121357][C-000000cf] app_dial.c: Unable to create channel of type 'PJSIP' (cause 3 - No route to destination)
[Oct 12 13:16:42] ERROR[114771] res_pjsip.c: Excessive refcount 100000 reached on ao2 object 0x5572e1bf40b8
[Oct 12 13:16:42] ERROR[114771] res_pjsip.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5572e1bf40b8 (0)
[Oct 12 13:16:42] ERROR[114771] : Got 18 backtrace records
# 0: asterisk __ast_assert_failed()
# 1: asterisk __ao2_ref()
# 2: [0x7f5a5c2c2d51] res_pjsip.so res_pjsip.c:1545 send_request_data_alloc()
# 3: [0x7f5a5c2c421a] res_pjsip.so res_pjsip.c:1946 ast_sip_send_out_of_dialog_request()
# 4: [0x7f5a5c2f2b1e] res_pjsip.so pjsip_options.c:910 sip_options_qualify_contact()
# 5: asterisk <unknown>()
# 6: asterisk __ao2_callback()
# 7: [0x7f5a5c2f2c8b] res_pjsip.so pjsip_options.c:932 sip_options_qualify_aor()
# 8: [0x7f5a5c2fc02f] res_pjsip.so pjsip_scheduler.c:100 run_task()
# 9: asterisk ast_taskprocessor_execute()
#10: asterisk <unknown>()
#11: asterisk ast_taskprocessor_execute()
#12: asterisk <unknown>()
#13: asterisk <unknown>()
#14: asterisk <unknown>()
#15: asterisk <unknown>()
#16: [0x7f5a5f86cea7] libpthread.so.0 pthread_create.c:478 start_thread()
#17: [0x7f5a5f07aacf] libc.so.6 clone.S:97 __GI___clone()
From another system:
[2024-11-06 00:43:48.984] ERROR[1552501][C-0000262c]: res_pjsip_device_features.c:649 feature_state_sub_by_endpoint: FRACK!, Failed assertion bad magic number 0x413b59d for object 0x7fd7f80aade0 (0)
[2024-11-06 00:43:49.267] ERROR[1552501][C-0000262c]: Got 19 backtrace records
# 0: [0x555b686b3bdf] asterisk utils.c:2831 __ast_assert_failed()
# 1: [0x555b6851b4bc] asterisk astobj2.c:220 log_bad_ao2()
# 2: [0x555b6851be30] asterisk astobj2.c:501 __ao2_ref()
# 3: [0x7fd7dd67dd80] res_pjsip_device_features.so res_pjsip_device_features.c:651 feature_state_sub_by_endpoint()
# 4: [0x7fd7dd67e29c] res_pjsip_device_features.so res_pjsip_device_features.c:737 func_features_write()
# 5: [0x555b6861bea9] asterisk pbx_functions.c:712 ast_func_write()
Unrelated segfault:
Thread 1 (Thread 0x7fdd47a406c0 (LWP 3870419)):
#0 0x00007fdd52f529c3 in ast_sip_subscription_is_terminated (sub=0x400) at res_pjsip_pubsub.c:2667
#1 0x00007fdd52ee52f6 in func_features_write (chan=0x0, function=0x7fdd47a3c440 "PJSIP_DEVICE_FEATURES", data=0x7fdd47a3c456 "Polycom2122", value=0x7fdd47a3c520 "1,0") at res_pjsip_device_features.c:743
feature_state_sub = 0x7fdd74322650
res = 0
tmp = 0x7fdd47a3c360 "1"
args = {argc = 2, {argv = {0x7fdd47a3c456 "Polycom2122", 0x7fdd47a3c462 "donotdisturb"}, {endpoint = 0x7fdd47a3c456 "Polycom2122", feature = 0x7fdd47a3c462 "donotdisturb"}}}
args2 = {argc = 2, {argv = {0x7fdd47a3c360 "1", 0x7fdd47a3c362 "0"}, {status = 0x7fdd47a3c360 "1", rings = 0x7fdd47a3c362 "0"}}}
__FUNCTION__ = "func_features_write"
__PRETTY_FUNCTION__ = "func_features_write"
#2 0x00005638033daea9 in ast_func_write (chan=0x0, function=0x7fdd47a3c5a0 "PJSIP_DEVICE_FEATURES(Polycom2122,donotdisturb)", value=0x7fdd47a3c520 "1,0") at pbx_functions.c:712
Though this issue is very obvious with the device feature module and will happen quickly, after weeks and weeks of running, the SCA module also exhibits a similar fault:
/var/log/asterisk/refs2.txt
(and proc):
[Nov 25 19:17:01] ERROR[256373] threadpool.c: Excessive (warning) refcount 60 reached on ao2 object 0x55af51b371b0
[Nov 25 19:17:01] ERROR[221099] rtp_engine.c: Excessive refcount 200 reached on ao2 object 0x55af5174f5f8
[Nov 25 19:17:01] ERROR[221099] rtp_engine.c: FRACK!, Failed assertion Excessive refcount 200 reached on ao2 object 0x55af5174f5f8 (0)
This is suspicious, and an interesting (possible) clue:
root@polypbx:/usr/src/asterisk-21.4.3# tail -2000000 /var/log/asterisk/messages.log | grep "Excessive refcount"
[Nov 25 19:17:01] ERROR[221099] rtp_engine.c: Excessive refcount 200 reached on ao2 object 0x55af5174f5f8
[Nov 25 19:17:01] ERROR[221099] rtp_engine.c: FRACK!, Failed assertion Excessive refcount 200 reached on ao2 object 0x55af5174f5f8 (0)
root@polypbx:/usr/src/asterisk-21.4.3# tail -2000000 /var/log/asterisk/messages.log.1 | grep "Excessive refcount"
root@polypbx:/usr/src/asterisk-21.4.3#
-rw-r----- 1 asterisk asterisk 27863465 Nov 25 19:17 messages.log
-rw-r----- 1 asterisk asterisk 94712385 Nov 24 00:00 messages.log.1
You must be