[LBBS-159] Implement/standardize various NNTP extensions

LIST ACTIVE.ALL

Advertising the new LIST command

[C] CAPABILITIES
[S] 101 Capability list:
[S] ...
[S] LIST ACTIVE ACTIVE.ALL
[S] ...

Usage

Syntax

LIST ACTIVE.ALL [wildmat]

The LIST ACTIVE.ALL command returns a combination of the information returned by the LIST (ACTIVE), LIST ACTIVE.TIMES, LIST NEWSGROUPS, and LIST COUNT commands in a single response to allow clients to more efficiently request all this data than using multiple LIST commands in succession.

The information is returned as a multi-line data block following the 215 response code and contains one line per newsgroup. Each line of this list MUST consist of eight fields separated from each other by a TAB, in the following order:

* The name of the newsgroup
* The reported high water mark for the group
* The reported low water mark for the group
* The estimated number of articles in the group
* The current status of the group on this server
* The time when this group was created on this news server, measured in seconds since the start of January 1, 1970
* The entity that created the newsgroup
* A short description of the group

If no description is available for a group, the TAB separating the creator entity and description MUST NOT be omitted.

The set of groups returned by LIST ACTIVE.ALL should match the set of groups returned by LIST ACTIVE, given an identical wildmat argument. That is to say, newsgroups that would be included in the response to LIST ACTIVE (with the same wildmat argument, if provided) MUST be included. Likewise, any groups present in the LIST ACTIVE.ALL response MUST also be returned in the LIST ACTIVE response.

The meanings of these parameters are as provided for the corresponding LIST commands (LIST ACTIVE, LIST ACTIVE.TIMES, and LIST NEWSGROUPS from RFC 3977 and LIST COUNT from RFC 6048).

Examples (TAB has been represented as a | for visual clarity):

[C] LIST ACTIVE.ALL misc.test
[S] 215 List of newsgroups follows
[S] misc.test|322|310|11|y|1780168665|John Smith <newsmaster@example.com>|A miscellaneous test group
[S] .

SEARCH

Existing proposal here: https://datatracker.ietf.org/doc/html/draft-ietf-nntpext-srch-00

THREAD/SORT

As in IMAP.

NEWNEWS limit support (by article count)

NEWNEWS currently only supports a date cutoff, which usually works well except in the case of a server that has just been fed another server's spool (or sucked a bunch of articles to initialize its spool). In this case, NEWNEWS may nominally return ALL articles (or a very large number of them). Clients furthermore have no way of knowing about this.

Since this is probably undesirable by both client and server, supporting a limit to cap the # of results would help.

INN does not seem order its NEWNEWS results, but LBBS does, so ideally the limit would be interpreted as "the most recent N articles", but we may not want to require this, for compatibility / ease of implementing this extension.

A further extension to NEWNEWS would also allow returning overview data along with the NEWNEWS response, so that we don't need to store the message-IDs and then separately run one OVER command per message afterwards (in nntp-web). There is reasonable precedent for this as well, for example, the SEARCH extension above proposes returning OVER response data.

e.g.

NEWNEWS * .... GMT LIMIT 100 (only 100 most recent articles - though order not guaranteed from NEWNEWS, so...)
NEWNEWS * .... GMT LIMIT 100 OVER

IDLE/NOTIFY for NNTP

way to get notified in real time about the presence of new articles in certain groups - ideally, this would build on top of the NEWNEWS/OVER combination extension so that the server would advertise overview data for new articles in all groups of interest without the client needing to stop idling and do anything.

Extension to synchronize NNTP clients (very rudimentary and primitive)

useful for online only clients or multiple online and/or offline clients
Why build this into NNTP? Not \Seen per message like IMAP, and using ACAP would involve a separate connection
not anticipated large news servers will offer this extension due to the amount of additional data they would need to store. Smaller or personal news servers offer the largest benefit.

X-GET-FIRSTUNSEEN
X-SET-FIRSTUNSEEN 23
X-SET-FIRSTUNSEEN 23 IFLASTUNSEEN 20

The last case prevent lost updates - that way can mark older if need to revert, without offline out of sync reverting an update. In many ways, this is reminiscent of CONDSTORE and that RFC should probably be reviewed in formalizing this idea.

Caveats/potential issues:

one complication is that, as with UIDs, ARTICLE id may not be always increasing re: date (sent). Clients will show messages in order of date, typically, not UID or article number, so even if a user reads them "in order" there, there may be gaps in the article numbers that cannot be preserved using just a single number.

DKIM/DMARC

Use DKIM - a new DNS record, for NNTP - to indicate whether to apply DMARC policy for the domain
Would help prevent (well, categorize as spam) other users' spoofing another user's email address without authorization
Since NNTP uses peer to peer flooding, SPF is not useful for NNTP since the end server may not receive the article directly from the source. In contrast, DKIM survives peering transit and even UUCP
Should work well since, unlike emails, net news articles are not known for being modified by intermediate news servers, so DKIM should rarely fail on legitimate articles
Would be an optional enhancement - nothing existing will break, but new servers can add support to check for a special "NNTP DMARC" record, and if it exists, would require DKIM to pass for articles for that domain
Although DKIM could probably be simplified / combined with DMARC for NNTP, I feel it's best to leverage the existing DKIM/DMARC logic for email as much as possible so the same libraries can simply be reused for NNTP without modification. For example, the need for separator "DKIM selectors" probably has little benefit for net news, given the publicity of Usenet articles, but this would be done to keep with the existing use for email
could also be useful for preventing spoofing of cancels/control messages (though there are already some ways of dealing with this)

Other

Comments

You must be logged in to leave a comment.