[LBBS-20] node.c: Occasional race conditions during node shutdown cause deadlock
This particular issue was noted as having occurred "from kicking an IMAP session". A cursory analysis suggests that because the function was deadlocked by the same function trying to acquire the lock, there was an attempt to shut down the node active in two different places at once. Unfortunately, I don't have a backtrace from this instance:
[2024-04-26 09:25:22.076] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.126] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.162] WARNING[3537839]: node.c:636 bbs_node_unlink: Spent 30 seconds so far waiting to wrlock &nodes, possible deadlock? (rwlock acquired at node.c:656 30 s ago by LWP 3550107)
[2024-04-26 09:25:22.162] ERROR[3537839]: backtrace.c:263 bbs_log_backtrace: Got 7 backtrace records
[2024-04-26 09:25:22.177] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.227] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.277] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.328] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.381] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.431] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.484] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.535] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.586] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.623] ERROR[3537839]: backtrace.c:287 bbs_log_backtrace: 0: lbbs bbs_log_backtrace()
[2024-04-26 09:25:22.623] ERROR[3537839]: backtrace.c:287 bbs_log_backtrace: 1: lbbs __bbs_rwlock_wrlock()
[2024-04-26 09:25:22.623] ERROR[3537839]: backtrace.c:287 bbs_log_backtrace: 2: lbbs bbs_node_unlink()
[2024-04-26 09:25:22.623] ERROR[3537839]: backtrace.c:287 bbs_log_backtrace: 3: [0x7f4190df61bc] net_imap.so net_imap.c:4743 __imap_handler()
[2024-04-26 09:25:22.623] ERROR[3537839]: backtrace.c:287 bbs_log_backtrace: 4: lbbs <unknown>()
[2024-04-26 09:25:22.624] ERROR[3537839]: backtrace.c:287 bbs_log_backtrace: 5: [0x7f4195df8044] libc.so.6 pthread_create.c:442 start_thread()
[2024-04-26 09:25:22.624] ERROR[3537839]: backtrace.c:287 bbs_log_backtrace: 6: [0x7f4195e7861c] libc.so.6 clone3.S:83 clone3()
[2024-04-26 09:25:22.636] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.687] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1s
[2024-04-26 09:25:22.737] DEBUG[3550107]: thread.c:354 __bbs_pthread_join: Thread 139918391064256 not yet joined after 1sComments
You must be