[LBBS-86] Deadlock on internal lock mutex

LBBS-13
pty.c: Attempt to lock previously destroyed mutex
PHREAKSCRIPT-20
dsp.c: coin DSP patch breaks faxing
LBBS-16
io_compress: Occasional memory corruption on shutdown
ASTERISK-119
app_dial: Progress timeout doesn't cause Dial to exit
ASTERISK-82
app_queue: Allow queue strategy to be manipulated externally
PHREAKNET-46
Polycom IP phones not provisioning
LBBS-86
Deadlock on internal lock mutex
PHREAKNET-54
Fix trunking to Telstar
ASTERISK-122
res_pjsip_device_features: Excessive refcount on endpoint object
DAHLIN-9
No rule to make target 'vpmadt032_loader/vpmadt032_x86_64.o'
DAHTOOL-5
Disabling span auto-assignment implodes kernel
PHREAKSCRIPT-28
app_predial: Failure to retrieve user agent property
PHREAKSCRIPT-63
chan_dahdi: Unused variable x
PHREAKNET-6
Verification flags international calls
LBBS-70
mod_smtp_delivery_external: Use SRS for externally forwarded messages
PHREAKSCRIPT-56
DAHDI Tools: Apply unmerged patches
ASTERISK-124
app_mixmonitor: Allow cancelling in-progress recording/recipients
DAHTOOL-4
dahdi_genconf: Hangs when run initially after span assignment
PHREAKSCRIPT-36
app_audichron: Tone not working after announcements
SLACK-5
Failed to receive reply... when message sent
LBBS-31
mod_sieve: Major Sieve filtering overhaul and improvements
PHREAKNET-34
Pat Fleet prompts
CAMI-1
Action Originate Returns -1
PHREAKSCRIPT-53
app_george: Cancel in-progress recording if nobody said anything
WSSMAIL-3
Message listing missing recipient for some messages
ASTERISK-137
sig_analog: Add alternate dial pulse scheme support
LBBS-65
net_smtp: Allow internal trusted hosts to be exempt from protocol violation checks
ASTERISK-102
app_voicemail: Play RDNIS if present when listening to envelope
DAHLIN-4
Improve manual span assignment process
ASTERISK-68
res_pjsip_sca: Segfaults on startup with SCA
ASTERISK-88
chan_dahdi: Add ADSI on-hook download support
ASTERISK-83
res_pjsip_device_features: Add forwarding and DND synchronization
ASTERISK-111
chan_dahdi: MWI while off-hook when hung up on after recall ring
ASTERISK-19
chan_dahdi: Remove sig_analog paths
ASTERISK-78
sig_analog: Add Last Number Redial
ASTERISK-77
sig_analog: Add Call Waiting Deluxe support
ASTERISK-130
Verify that res_xmpp can compile and be used
LBBS-64
mod_operator: Crash with more than 64 options?
ASTERISK-71
app_if: ElseIf executed even if If is true
PHREAKNET-42
Provisioning fails for lines with certain features
PHREAKNET-40
Recent Change provisioning bug increments device name by 1
PHREAKNET-52
Cisco end of CSR signing support
DAHLIN-7
Mega-issue to upstream out-of-tree DAHDI patches
PHREAKSCRIPT-30
MD5/RSA authenticated calls frequently fail
ASTERISK-97
func_groupcount: Add group vars
PHREAKNET-35
TSPS improvements
PHREAKNET-23
Wakeup calls not dialling at scheduled time
ASTERISK-101
func_tonedetect: 2600 Hz detection not working or suboptimal
ASTERISK-115
chan_iax2: Some pattern matches ignored if included in incoming context
ASTERISK-112
chan_dahdi: Raw power ring audible on recall ring from IAX2 to DAHDI
LBBS-32
mod_spamassassin: Improve default spam handling
LBBS-6
Add menu system scripting engine
ASTERISK-66
app_sf: ReceiveSF not detecting incoming SF
EVERGREEN-1
Use default identity per folder
ASTERISK-110
chan_iax2: Implement MWI
ASTERISK-80
app_record: Add RECORD_TIME function
INTERLINKED-2
PhreakNet Email for BoD Members
LBBS-88
net_imap: IDLE data not being processed in realtime
INTERLINKED-20
Closing multiple issues simultaneously doesn't work
PHREAKSCRIPT-39
Add optional flag to not update system first
LBBS-58
net_irc: Make ping interval adjustable
DAHLIN-1
Makefile: remove update target
PHREAKSCRIPT-43
Add spanconfig command
ASTERISK-106
chan_pjsip: Modify From header appropriately for unavailable presentation
PHREAKNET-47
Add automation for bill mailings
LBBS-71
Limit registrations from same IP within a certain amount of time
ASTERISK-64
chan_dahdi: autoreoriginate doesn't work if caller hung up on
LBBS-66
logger: Add rate limiting of log messages
INTERLINKED-16
Add option to not be notified about user's own issues
LBBS-46
Block malicious mail servers after repeated failures
ASTERISK-52
dsp.c: Add ast_freq_reset
LBBS-11
door_evergreen: Use isonetexec instead of exec
LBBS-7
Fix backspace detection/handling
ASTERISK-37
chan_pjsip: CALLERID(ani2) not working
ASTERISK-36
res_pjsip_logger: Add filtering by username
LBBS-22
mod_mail_trash: Eliminate bbs_pthread_cancel_kill
ASTERISK-108
config.c: Investigate and fix various config file rewrite bugs
ASTERISK-90
chan_iax2: Add log message for rejected calls
ASTERISK-27
chan_dahdi: add IS_FXO_SIG macro
SLACK-2
README: Add missing pre-reqs
ASTERISK-55
chan_dahdi: add NUM_ANALOG_SUBS
ASTERISK-34
app_queue: Allow setting context at runtime
ASTERISK-31
Add AMI event for assertion
ASTERISK-30
Add ast_channel_tech_name_match function
ASTERISK-29
chan_iax2: Add ${CHANNEL(authmethod)}
ASTERISK-15
chan_dahdi: Allow sending SDMF Caller ID
ASTERISK-98
Add hangup callbacks
ASTERISK-47
res_pjsip_session: Warn if no shared codecs
PHREAKSCRIPT-49
wanpipe doesn't compile above 6.1 kernel
ASTERISK-86
app_confbridge: Channel can join wrong bridge due to race condition
INTERLINKED-6
PhreakNet Forums
LBBS-74
net_ssh: Sessions persist even when they no longer exist
DAHLIN-10
Allow enabling EMPULSE and EMFLASH simultaneously
LBBS-82
localtime_r deadlock
ASTERISK-135
Mutex errors on FreeBSD
DAHLIN-6
Frame relay broken
LBBS-41
socket.c: SEGV in bbs_node_readline
LBBS-69
net_ssh: SFTP uploads fail with WinSCP
PHREAKSCRIPT-19
app_verify: Double free
PHREAKNET-14
1+ calls do not work for single numbers
LBBS-90
mod_smtp_delivery_external: Improve queuing process
PHREAKNET-39
Splice Fred Covington prompts for speaking clock
LBBS-47
Email enhancements
ASTERISK-131
chan_dahdi: Automatically update time when DST starts/ends
WSSMAIL-5
Add full attachment support
ASTERISK-129
core: Add ref/unref for ast_moh callbacks
DAHLIN-8
Readd native SF support
PHREAKNET-49
ML model for 3-slot coin detection
PHREAKSCRIPT-45
TDMoIP driver module
PHREAKSCRIPT-44
Finish ADSI applications
PHREAKSCRIPT-41
Create virtual modem application using softmodem
PHREAKNET-41
Allow for more granular billing based on switch ZIP code
PHREAKNET-33
Finish TSPS web console
ASTERISK-121
Upstream PhreakScript patches
LBBS-72
mod_webdav: Add WebDAV support
ASTERISK-118
chan_iax2: Improve encryption support
DAHLIN-5
FXO rings off hook when rotary dialing on the line
ASTERISK-105
chan_dahdi: Add CLI command to display line voltages
PHREAKSCRIPT-37
app_softmodem: Support speeds higher than 2400bps
ASTERISK-93
chan_dahdi: DSP optimizations
WSSMAIL-13
Improve multi-INBOX navigation
LBBS-56
net_snpp: Add Simple Network Paging Protocol support
LBBS-48
net_irc: Support multi-server networks
EVERGREEN-3
Add NNTP support for newsgroups
LBBS-38
mod_smtp_mailing_lists: Allow dynamic membership
LBBS-36
net_telnet: Improve protocol support
LBBS-35
net_acap: Add ACAP protocol support
WSSMAIL-4
Add PHPMailer support for format=flowed
ASTERISK-7
Improve sound control capabilities
LBBS-30
net_nntp: Complete overhaul/rewrite to use maildirs
ASTERISK-99
pjsip: Add Cisco CUCM patch functionality
LBBS-49
Add functionality expected of a traditional BBS package
LBBS-34
net_xmpp: Add XMPP protocol support
LBBS-33
net_imap: Add/standardize commands to modify or purge keywords
ASTERISK-75
chan_dahdi: Add Call Hold feature
ASTERISK-57
bridging: Make DAHDI, always softmix bridging modules
LBBS-91
readline.c: Crash in memmove
EVERGREEN-2
Navigation breaks when requirefromhelomatch=no in LBBS
DOCS-1
Kickstart the wiki
ASTERISK-79
config.c: Make ast_variable_retrieve return last match
ASTERISK-87
asterisk.c: Prevent creation of duplicate Asterisk processes
PHREAKNET-36
Trunking integration with Phil McCarter
ASTERISK-16
res_pbx_validate: Add commands to warn of dialplan problems
PHREAKSCRIPT-62
res_smdr_whozz: Initialization usually fails in practice
LBBS-80
mod_mimeparse: Occasional segfault when running tests
ASTERISK-134
chan_dahdi: Improve configuration validation
DAHTOOL-3
Compilation warnings on 32-bit (i686)
PHREAKSCRIPT-33
verify.conf: Flag invalid NXXs as illegitimate for NANPA calls
LBBS-61
net_imap: Soft assertion in parse_flags_string
LBBS-63
net_imap, io_compress: Spurious warnings if quota exceeded
ASTERISK-100
chan_dahdi: PRI lock held at module unload
LBBS-39
net_smtp: Occasional memory corruption on email delivery
LBBS-37
Restrict outgoing port 25 from containers while allowing other traffic
LBBS-23
net_smtp: DMARC reports not sent due to rejection of localhost
LBBS-20
node.c: Occasional race conditions during node shutdown cause deadlock
LBBS-17
Temporary files not always cleaned up
LBBS-79
Makefile: cp -n is deprecated
ASTERISK-109
voicemail.conf.sample: Clarify misnamed option behavior
PHREAKSCRIPT-35
app_mail: SendMail returns SUCCESS
LBBS-51
General TDD improvements
PHREAKSCRIPT-23
res_phreaknet: Increase cURL timeout
ASTERISK-94
app_chanspy: Fix bad audio with single direction operation
ASTERISK-76
app_dial: Allow MOH until progress received
LBBS-28
mod_discord: Warn users if IRC users are not online
PHREAKSCRIPT-16
app_selective: Missing pause after add/delete/instructions prompt
DAHTOOL-2
dahdi_cfg: Compiler warnings
ASTERISK-43
pbx_builtins: allow Answer to specify the channel
ASTERISK-35
sig_analog: Fix buffer overflow
DAHLIN-11
Use dev_addr_set instead of directly modifying const buffer
PHREAKSCRIPT-58
sig_analog: Allow hearpulsing gain to be adjusted
ASTERISK-128
Rename ast_sip_presence_xml_ functions
ASTERISK-127
func_codecs: view/update codecs
ASTERISK-123
app_voicemail: Add ability to disable "thank-you" when ending voicemail
WSSMAIL-15
Allow navigating messages using arrow keys
ASTERISK-104
Improve dahdi show channels output
PHREAKSCRIPT-29
res_phreaknet: Don't try making API requests if not a PhreakNet node
LBBS-59
Add HTTP endpoint for updating IRC presence automatically
WSSMAIL-14
Checkbox range selection doesn't always work reliably
PHREAKSCRIPT-34
app_verify: Check additional headers for STIR/SHAKEN attestation
SLACK-3
examples: Add example program to log everything to a CSV
PHREAKSCRIPT-22
Add framehook to display dialed digits
PHREAKSCRIPT-21
app_looparound: Add LoopAround application
LBBS-45
Use isoexec whenever possible
PHREAKSCRIPT-17
res_phreaknet: iax.conf, et. al. are truncated when there is no disk space left
LBBS-43
mod_discord: Improve handling of in-thread replies
LBBS-40
net_smtp: Add additional loop avoidance mechanisms
ASTERISK-73
chan_dahdi: Hunt to next trunk in group if busy
LBBS-29
net_irc: Add reminder bot
ASTERISK-69
res_cli_originate: Allow specifying Caller ID on CLI
DAHLIN-3
build: Compiler warnings
ASTERISK-65
sig_analog: Allow simple switch to time out to dialplan
LBBS-10
mod_sysop: Allow modifying user info from console
LBBS-8
net_imap: Accounts occasionally fail to load or are reordered
LBBS-5
tests: Add TLS tests
LBBS-4
tests: Add tests that establish remote console connections
ASTERISK-63
Upstream certain PhreakScript modules
ASTERISK-53
formats: add ulaw/alaw WAV support
ASTERISK-51
app_originate: Allow spooling future calls
INTERLINKED-4
InterLinked Issues API
PHREAKNET-7
Subscriber Service Disconnect Request Option
LBBS-89
mod_sysop: Improve history navigation and tab completion
ASTERISK-126
chan_iax2: Add fixed jitterbuffer
INTERLINKED-17
Blog: Issues with tag editing and viewing
INTERLINKED-15
Improve paging API
PHREAKNET-45
Operator system PSPs are outdated
LBBS-52
net_smtp: Add limited support for VRFY and EXPN
PHREAKSCRIPT-25
chan_sccp no longer compiles
WSSMAIL-8
Add NNTP support and IRC integration
PHREAKSCRIPT-18
Add better test coverage for 32-bit systems
ASTERISK-84
Add 'core show variables' command
ASTERISK-74
chan_dahdi: Add AMI event for callwaiting
WSSMAIL-7
Add ability for zero-trust type authentication
LBBS-27
mod_irc_relay: Allow relaying of away/here status
LBBS-24
build: Pre-reqs don't compile on Fedora
LBBS-9
mod_webmail: Improve MIME parsing to extract HTML/PT components
ASTERISK-58
xml: Add config documentation
ASTERISK-56
res_musiconhold: Add recursive files support
PHREAKNET-32
Set up Millennium Manager
PHREAKNET-24
[REQ] TTY Telegrams
ASTERISK-11
chan_dahdi: Add wink guard
ASTERISK-8
Optimize boilerplate RSA to MD5 fallback
PHREAKNET-20
Billing based on individual CLLI zip codes
PHREAKNET-1
Add third-party billing integration
PHREAKNET-10
Enhanced utilization of Feature Group D
ASTERISK-10
Proctor application for DAHDI
PHREAKNET-9
General ARTNVAAXCG0 enhancements
ASTERISK-136
app_voicemail: Voicemails stored but no email sent
PHREAKSCRIPT-50
app_acts: Finish fine tuning and bug fixes
PHREAKSCRIPT-47
make webvmail
INTERLINKED-7
TTY BBS lines (45.5+50) issues
ASTERISK-85
chan_pjsip: Add channel encryption integration
PHREAKSCRIPT-51
Add 32-bit/ARM builds
ASTERISK-89
app_stack: Print proper exit location for PBXless channels
ASTERISK-125
app_voicemail: Warning when hanging up during playback
ASTERISK-120
app_senddtmf: SendDTMF failure if not answering channel
LBBS-15
smtp: Locking errors, deadlock when queue runs during delivery
PHREAKSCRIPT-24
DAHDI builds in GitHub CI occasionally fail due to IPv6
ASTERISK-67
voicemail.conf.sample: maxsilence/minsecs trigger warning
ASTERISK-50
app_dial: Fix or document r tonezone option
DAHTOOL-1
system.conf: Fix typos
ASTERISK-40
pjsip: document MessageSend AOR limitation
ASTERISK-32
chan_dahdi: Allow Call Forwarding to be set/get programatically
ASTERISK-25
app_originate: allow early media, labels
ASTERISK-24
app_chanspy: Add no answer option
ASTERISK-22
general: Fix various typos
DAHLIN-2
Fix typos and dead links
PHREAKSCRIPT-54
Replace LOAD_STR_INT with ast_yesno
ASTERISK-48
func_devstate: Delete empty from AstDB
ASTERISK-107
indications.conf.sample: Add confirmation tone
WSSMAIL-12
Add one-click "Not junk" button

← All Issues || New Issue →

Sort Normal || Priority

Details


Strange deadlock observed tonight:

Last few successfully logged (and flushed to log file) log messages:

[2024-11-29 21:09:42.351]   DEBUG[1493365]: imap_client.c:147 imap_poll: Polling 12 fds for IMAP session 0x7f55b1f845d0 (for 25s)
[2024-11-29 21:10:06.828]   DEBUG[1493808]: readline.c:165 bbs_readline: bbs_poll_read returned 0
[2024-11-29 21:10:06.829]   DEBUG[1493808]: net_imap.c:4892 handle_client: 0x7f559fff45d0 <= * BYE IMAP4rev1 server terminating connection
[2024-11-29 21:10:06.829]   DEBUG[1493808]: mod_mail.c:232 mailbox_dispatch_event: Dispatching mailbox event 'Logout' (maildir: )
[2024-11-29 21:10:06.829]   DEBUG[1493808]: node.c:2259 bbs_node_exit: Node 2 has ended its IMAP session
[2024-11-29 21:10:06.829]   DEBUG[1493808]: io.c:414 bbs_io_teardown_all_transformers: Removing I/O transformer at index 0

Issue appears similar to this one: https://stackoverflow.com/questions/6697058/pthread-mutex-lock-locks-but-no-owner-is-set In the backtrace, all the threads are blocked on line 533 of logger.c, which is RWLIST_RDLOCK(&remote_log_fds); Since they are all trying to do a read lock, they all succeed, but what they are all blocking on is line 345 in lock.c, since this is a mutex (the internal BBS lock mutex). As can be seen from the gdb output (bottom), this mutex has no apparent owner (normally, the locking LWP would be listed here). So, there are no clues as to why we are failing to acquire the mutex. In the code, there is no way that this could still be held legitimately (according to the backtrace), and the lock doesn't seem to be held at all, so it's unclear why all attempts to acquire the mutex are blocked.

Of course, it's always possible a thread was cancelled while it was holding the mutex... seems unlikely, but I can't think of another explanation either.

(gdb) print t
$1 = (bbs_rwlock_t *) 0x55af29d6f910 <remote_log_fds+16>

$2 = {lock = {__data = {__readers = 104, __writers = 0, __wrphase_futex = 0, __writers_futex = 0, __pad3 = 0, __pad4 = 0, __cur_writer = 0, __shared = 0, __rwelision = 0 '\000',
      __pad1 = "\000\000\000\000\000\000", __pad2 = 0, __flags = 0}, __size = "h", '\000' <repeats 54 times>, __align = 104}, intlock = {__data = {__lock = 2, __count = 0, __owner = 0,
      __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = "\002", '\000' <repeats 38 times>, __align = 2}, info = {initialized = 1,
    staticinit = 1, destroyed = 0, owners = 0, lastlocked = 1732932606, lwp = 1493808, lineno = 533, filename = "logger.c", '\000' <repeats 15 times>}}

Thread 22 (Thread 0x7f559fff76c0 (LWP 1493808) "lbbs"):
#0  futex_wait (private=0, expected=2, futex_word=0x55af29d6f948 <remote_log_fds+72>) at ../sysdeps/nptl/futex-internal.h:146
#1  __GI___lll_lock_wait (futex=futex@entry=0x55af29d6f948 <remote_log_fds+72>, private=0) at ./nptl/lowlevellock.c:49
#2  0x00007f55ebd2d3c2 in lll_mutex_lock_optimized (mutex=0x55af29d6f948 <remote_log_fds+72>) at ./nptl/pthread_mutex_lock.c:48
#3  ___pthread_mutex_lock (mutex=mutex@entry=0x55af29d6f948 <remote_log_fds+72>) at ./nptl/pthread_mutex_lock.c:93
#4  0x000055af29d1ef14 in __bbs_rwlock_rdlock (t=t@entry=0x55af29d6f910 <remote_log_fds+16>, filename=filename@entry=0x55af29d57ec6 "logger.c", lineno=lineno@entry=533, func=func@entry=0x55af29d58f9>
#5  0x000055af29d20916 in __bbs_log (loglevel=loglevel@entry=LOG_DEBUG, level=level@entry=6, file=file@entry=0x55af29d61d38 "thread.c", lineno=lineno@entry=327, func=func@entry=0x55af29d621e0  at logger.c:533

(gdb) print *mutex
$7 = {__data = {__lock = 2, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = "\002", '\000' <repeats 38 times>,
  __align = 2}

Comments

You must be logged in to leave a comment.