[LBBS-90] mod_smtp_delivery_external: Improve queuing process
Improve the queue file format (as described in the comments in this module)- this has been completed for the most part- Use flock to lock individual queue files, instead of locking the whole queue. This can cause contention for mail delivery, as even though the locks are eventually released, it results in deadlock-like behavior and delays delivery of unaffected and unrelated messages, e.g:
[2025-08-08 17:04:17.573] WARNING[751157]: socket.c:574 __bbs_tcp_connect: Aborting connection to REDACTED:25 (timed out after 30 seconds)
[2025-08-08 17:04:17.573] DEBUG[751157]: socket.c:561 __bbs_tcp_connect: Attempting connection to REDACTED:25
/mailq
ControlFile Try Orig Date Last Retry Est. Next Retry Size Sender -> Recipient
[2025-08-08 17:04:48.293] WARNING[751157]: socket.c:574 __bbs_tcp_connect: Aborting connection to REDACTED:25 (timed out after 30 seconds)
[2025-08-08 17:04:48.293] DEBUG[751157]: socket.c:561 __bbs_tcp_connect: Attempting connection to REDACTED:25
[2025-08-08 17:04:57.351] WARNING[722437]: mod_smtp_delivery_external.c:1601 run_queue: Spent 71 seconds so far waiting to wrlock &queue_lock, possible deadlock? (rwlock acquired at mod_smtp_delivery_exter:1963 30 s ago by LWP 751159)
[2025-08-08 17:04:57.363] ERROR[722437]: backtrace.c:255 bbs_log_backtrace: Got 7 backtrace records
[2025-08-08 17:04:58.039] ERROR[722437]: backtrace.c:279 bbs_log_backtrace: 0: lbbs bbs_log_backtrace()
[2025-08-08 17:04:58.040] ERROR[722437]: backtrace.c:279 bbs_log_backtrace: 1: lbbs __bbs_rwlock_wrlock()
[2025-08-08 17:04:58.040] ERROR[722437]: backtrace.c:279 bbs_log_backtrace: 2: [0x7faee1d24b3d] mod_smtp_delivery_external.so mod_smtp_delivery_external.c:1602 run_queue()
[2025-08-08 17:04:58.040] ERROR[722437]: backtrace.c:275 bbs_log_backtrace: 3: [0x7faee1d255e0] mod_smtp_delivery_external.so mod_smtp_delivery_external.c:1644 queue_handler()
[2025-08-08 17:04:58.041] ERROR[722437]: backtrace.c:275 bbs_log_backtrace: 3: [ inlined] mod_smtp_delivery_external.so mod_smtp_delivery_external.c:1630 queue_handler()
[2025-08-08 17:04:58.041] ERROR[722437]: backtrace.c:279 bbs_log_backtrace: 4: lbbs <unknown>()
[2025-08-08 17:04:58.041] ERROR[722437]: backtrace.c:279 bbs_log_backtrace: 5: [0x7faee2ca81f5] libc.so.6 pthread_create.c:442 start_thread()
[2025-08-08 17:04:58.041] ERROR[722437]: backtrace.c:279 bbs_log_backtrace: 6: [0x7faee2d2889c] libc.so.6 clone3.S:83 clone3()- If an external sender is delivering a message for a single recipient, and it needs to be relay further "into the network", have the option of attempting delivery during the transaction, rather than queuing it. This would allow us to reject the message during the transaction if rejected downstream, rather than having to generate a bounce
- Be able to do one SMTP transaction for all recipients of a message at a given domain, rather than doing one transaction per recipient as we do now
In the case of delivering a message to several recipients at the same server, NOT batching recipients into one transaction can result in being rate limited, e.g.:
[2025-01-22 22:33:40.261] DEBUG[2061347]: tcp.c:111 bbs_tcp_client_expect: <= 421 #4.4.5 Too many connections from your host.
[2025-01-22 22:33:40.262] WARNING[2061347]: tcp.c:116 bbs_tcp_client_expect: Missing expected response (220), got: 421 #4.4.5 Too many connections from your host.
[2025-01-22 22:33:40.262] DEBUG[2061347]: mod_smtp_client.c:116 bbs_smtp_client_expect_final: Found '220': 421 #4.4.5 Too many connections from your host.
[2025-01-22 22:33:40.262] WARNING[2061347]: mod_smtp_client.c:119 bbs_smtp_client_expect_final: Expected '220', got: 421 #4.4.5 Too many connections from your host.
[2025-01-22 22:33:40.262] DEBUG[2061347]: mod_smtp_delivery_external.c:619 try_send: => QUITComments
You must be