Moderation relays

From Usenet Big-8 Management Board

The way the system is designed to work

1. A user posts to a moderated newsgroup.

2. Is the newsgroup configured as moderated by the user's news service provider?

YES. The news server has two options:

  • Using the ISC relay: the news server
    • replaces dots with dashes in the group's name.
    • appends "@moderators.isc.org" to the revised name
    • stuffs the post in an envelope with an envelope sender address
    • e-mails the post to <reformated-newsgroup-name@moderators.isc.org>
      • For example: for news.groups.proposals, the server would mail to
        <news-groups-proposals@moderators.isc.org>.
  • If the news server keeps its own list of aliases (possibly for regional hierarchies), it may bypass the ISC relays and e-mail the post directly to the submission address for the moderated newsgroup.

NO.

  • The post is injected into the news server's spool. It may circulate to other servers on which the group is flagged as unmoderated if there is a pathway to those other servers. The author of the post sees it on the misconfigured news server immediately, but the moderators of the group and most users on other servers never see the post.

3. Is the envelope sender address resolvable?

YES.

  • One of two moderation relays is chosen at random (moderators.individual.net or moderators.schnuerpel.eu).
  • The relay looks up the submission address for the newsgroup on its alias list.
  • The relay e-mails the post to the group's submission address.

NO.

  • The e-mail will probably be discarded by the e-mail system. See the next section for an explanation of how envelope sender addresses work.
  • The moderators of the group will probably never see the post.

4. Do the moderators approve the post?

YES.

  • The moderators mark the post as approved.
  • The moderators inject the post into a news server.

NO.

  • The post is discarded.

How envelope sender addresses work

  • Mail servers generally reject mail with invalid (unresolvable) envelope sender addresses. This isn't specific to the moderation relays; it's a standard anti-spam practice for many mail servers. Mail administrators started rejecting mail with non-resolving envelope From: addresses because it blocked spam and almost no legitimate mail. For example, sendmail 8.9, released in 2000, blocks mail with unresolvable envelope-from addresses *by default*.
    • The envelope of an e-mail has two essential pieces of information written on it, just like a physical envelope for a letter: a From: address and a To: address.
    • The news server program would generally send the mail with an envelope sender address of <news@domain> because "news" is the standard user account under which the server software is run. When a news server program generates an email, the envelope's From: address is automatically set to the program's user and domain. If the news server program is running under the user "news", which is typical, <news@domain> would be used for the From: address on the envelope. If the news server program is being run under another username, the From: address on the envelope would just be that username at the server's host--e.g., if the news server program is running under the clark_kent account on dailyplanet.com, the From: address on the envelope would be <clark_kent@dailyplanet.com>.

Things that break the system

  • If the server is running on a host with an invalid name, and the program has not been configured to set the mail envelope to something else, the e-mail sent by the server will probably be discarded; this isn't common, because that's just a sysadmin no-no.
    • Note: the envelope's From: address doesn't have to be valid in the sense that you can send mail to it; it really just has to have a valid hostname and be in the right format. Having a server running with a totally invalid hostname is just broken, but it has happened.
    • The word "valid" in this context is being used to indicate that the moderation relays can find the domain in the public Domain Name Service (DNS). The problem with the news server that was generating unresolvable addresses was that the host name used on the envelope addresses was valid on the news service's internal network, but the name was not identifiable from the public Internet through DNS.
  • If the server has been configured to use the poster's From: address as the envelope's From: address, and if the poster is using an address with an unrecognizable domain, then the From: address on the envelope is invalid. This is also something that is the result of a misconfiguration because it is not the proper method to set the envelope address on a mail server. In fact, almost all mail server software would require you to take extra steps to make it so you could do that without a warning being generated. The entire point of the envelope sender is so that users can set their From: address to whatever they want and still have a correct return-path generated from the envelope address.
  • If the server gets blacklisted as a spam server, the posts it relays may be discarded.
  • As of 2011-12-21, there are three relays supporting the moderation system: Individual.net, schnuerpel.eu and mit.edu.

The Bottom Line

If a news server is misconfigured in one of the two ways described above, the e-mail from the news server to the central moderation relays gets discarded and never reaches the moderator's submission address. It's not something that moderators can do anything about because the mail never reaches their server.

In practice, almost no one has their servers set up in either of the above ways. Andrew, one of the administrators at the Supernews relay, notes that 99.9% of submissions aren't broken like this -- almost all news servers have the envelope sender addresses set correctly because all of the server software programs configure envelope From: addresses correctly by default. It's easy to get right and hard to get wrong.

Simplified Flow Chart of the Moderation Relay System

Mod relay.jpg

Flow chart created with demo version of RFFlow.

Changing moderation relay addresses

Moderators should send requests to change a newsgroup's submission address to moderators-request@isc.org.

Links

How DNS Works
SMTP and the Message Envelope
PMDF Programmer's Reference: 1.2 Enqueuing Messages
To: and From: Email Headers can be Spoofed/Forged
Using the SMTP Message Envelope Fields