applyfilter(8) [debian man page]
applyfilter(8) System Manager's Manual applyfilter(8) NAME
applyfilter - apply filter settings to news spool SYNOPSIS
applyfilter [-v] ... newsgroup DESCRIPTION
Leafnode is a USENET package intended for small sites, where there are few users and little disk space, but where a large number of groups is desired. Applyfilter is the program to apply filter settings to your newsspool, particularly after changes that were not in effect at the time the articles were downloaded with fetchnews. Applyfilter is used if you want to remove postings from your spool matching a certain filter pat- tern (see FILES below). If the headers of an article match any of the filtering rules, the article is being deleted from the newsgroup. Applyfilter sets its real and effective uid to "news". OPTIONS
-v Be verbose. newsgroup is the newsgroup you want to apply the filter to. FILES
A file specified in the filterfile option of /etc/news/leafnode/config contains the set of filtering rules that are applied to the newsspool. Each rule must be on a separate line. Empty lines and comments (marked by a # which MUST be at the beginning of the line) are allowed and will be ignored. Each line contains a regular expression that will be compiled with the PCRE_MULTILINE option set (unless it is a comment line or an empty line). Every regular expression applied to the whole message header (folded message headers are unfolded before the match is attempted). If any regular expression matches the header, the posting gets removed. For example, the following set of rules will delete all crosspost- ings to alt.flame. ^Newsgroups:.*[, ]alt.flame$ ^Newsgroups:.*[, ]alt.flame, ^Newsgroups: indicates that the pattern should only be applied to the Newsgroups: header. After the header, an arbitrary number of charac- ters may follow, ending in either a comma or a space. After that, alt.flame must be in the line, either at the end of the line (first entry) or in the middle of the line, followed by other newsgroups (second entry). Unfortunately, regular expressions are a complex matter and outside the scope of this document. Please see the manual pages provided with the PCRE library for details. Note that some leafnode programs (such as fetchnews) feed the header line by line to the regexp (after unfolding), others (such as apply- filter) feed the whole header in one large blob. This does not usually matter because patterns are compiled in multiline mode, but will matter if you're trying to match multiple headers in the same regexp -- this would work with applyfilter, but not with fetchnews. ENVIRONMENT
LN_LOCK_TIMEOUT This variable is parsed as an unsigned integer value and determines how many seconds applyfilter will wait when trying to obtain the lock file from another leafnode program. 0 means to wait indefinitely. This variable takes precedence over the configuration file. AUTHOR
Copyright 1999 by Cornelius Krasel <krasel@wpxx02.toxi.uni-wuerzburg.de>. Copyright 2002 - 2006 by Matthias Andree <matthias.andree@gmx.de>. Leafnode was originally written by Arnt Gulbrandsen <agulbra@troll.no> and is copyright 1995-96 Troll Tech AS, Postboks 6133 Etterstad, 0602 Oslo, Norway, fax +47 22646949. SEE ALSO
pcrepattern(3), leafnode(8), fetchnews(8), texpire(8). leafnode 1.11.8 applyfilter(8)
Check Out this Related Man Page
EXPIRE.CTL(5) InterNetNews Documentation EXPIRE.CTL(5) NAME
expire.ctl - Configuration file for article expiration DESCRIPTION
The file pathetc/expire.ctl is the default configuration file for expire and expireover, which read it at start-up. It serves two purposes: it defines how long history entries for expired or rejected articles are remembered, and it determines how long articles stored on the server are retained. Normally, if all of the storage methods used by the server are self-expiring (such as CNFS), all lines except the "/remember/" setting (described below) are ignored. This can be changed with the -N option to expire or expireover. Blank lines and lines beginning with a number sign ("#") are ignored. All other lines should be in one of two formats. The order of the file is significant, and the last matching entry will be used. The first format specifies how long to keep history entries for articles that aren't present in the news spool. These are articles that have either already expired, or articles which the server rejected (when remembertrash is set to true in inn.conf). There should be one and only one line in this format, which looks like: /remember/:<days> where <days> is a decimal number that specifies the minimum number of days a history record for a given message-ID is retained (from its original posting time), regardless of whether the article is present in the spool. (History entries for articles still present in the spool are always retained.) The primary reason to retain a record of old articles is in case a peer offers old articles that were previously accepted but have already expired. Without a history record for such articles, the server would accept the article again and readers would see duplicate articles. Articles older than a certain number of days won't be accepted by the server at all (see artcutoff in inn.conf(5) and the -c flag in innd(8)), and this setting should probably match that time period to ensure that the server never accepts duplicates. As the default value for artcutoff is 10 days, it means that "/remember/" should be set to 11 days in order to take into account articles whose posting date is one day into the future. Most of the lines in this file will be in the second format, which consists of either four or five colon-separated fields: <pattern>:<flag>:<min>:<default>:<max> if groupbaseexpiry is true in inn.conf (the default), and otherwise: <classnum>:<min>:<default>:<max> All lines must be in the correct format given the current setting of groupbaseexpiry, and therefore the two formats cannot co-exist in the same file. Normally, a rule matches a newsgroup through the combination of the <pattern> and <flag> fields. <pattern> is a uwildmat(3)-style pattern, specifying the newsgroups to which the line is applied. Note that the last matching entry will be used, so general patterns (such as defaults for all groups where <pattern> is "*") should appear at the beginning of the file before more specific settings. The <flag> field can be used to further limit newsgroups to which the line applies, and should be chosen from the following set: M Only moderated groups U Only unmoderated groups A All groups X Remove the article from all groups it appears in One of "M", "U", or "A" must be specified. "X" should be used in combination with one of the other letters, not by itself. An expiration policy is applied to every article in a newsgroup it matches. There is no way to set an expiration policy for articles crossposted to groups you don't carry that's different than other articles in the same group. Normally, articles are not completely deleted until they expire out of every group to which they were posted, but if an article is expired following a rule where <flag> contains "X", it is deleted out of all newsgroups to which it was posted immediately. If groupbaseexpiry is instead set to false, there is no <pattern> and <flag> field and the above does not apply. Instead, there is a single <classnum> field, which is either a number matching the storage class number specified in storage.conf or "*" to specify a default for all storage classes. All articles stored in a storage class will be expired following the instructions in the line with a matching <classnum>, and when articles are expired, they're always removed from all groups to which they were posted. The remaining three fields are the same in either format, and are used to determine how long an article should be kept from its original arrival time (unless the -p flag is passed to expire(8) or expireover(8), in which case its original posting time is used). Each field should be either a decimal number of days (fractions like 8.5 are allowed, but remember that articles are only removed when expire or expireover is run, normally once a day by news.daily) or the word "never". The middle field, <default>, will be used as the expiration period for most articles. The other two fields, <min> and <max>, only come into play if the article requests a particular expiration date with an Expires: header. Articles with an Expires: header will be expired at the date given in that header, subject to the constraints that they will be retained at least <min> days and no longer than <max> days. If <min> is set to "never", no article matching that line will ever be expired. If <default> is set to "never", no article matching that line without an explicit Expires: header will ever be expired. If <max> is set to "never", Expires: headers will be honored no matter how far into the future they are. One should think of the fields as a lower bound, the default, and an upper bound. Since most articles do not have an Expires: header, the second field is the most important and most commonly applied. Articles that do not match any expiration rule will not be expired, but this is considered an error and will result in a warning. There should always be a default line (a line with a <pattern> of "*" and <flag> of "A", or a line with a <classnum> of "*"), which can explicitly state that articles should never expire by default if that's the desired configuration. The default line should generally be the first line of the file (except for "/remember/") so that other expiration rules can override it. It is often useful to honor the Expires: header in articles, especially those in moderated groups. To do this, set <min> to zero, <default> to whatever normal expiration you wish, and <max> to "never" or some large number, like 365 days for a maximum article life of a year. To ignore any Expires: header, set all three fields to the same value. EXAMPLES
When groupbaseexpiry is true (the default): # Keep expired article history for 11 days, matching artcutoff plus one. /remember/:11 # Most articles stay for two weeks, ignoring Expires: headers. *:A:14:14:14 # Accept Expires: headers in moderated groups for up to a year and # retain moderated groups for a bit longer. *:M:1:30:365 # Keep local groups for a long time and local project groups forever. example.*:A:1:90:90 example.project.*:A:never:never:never When groupbaseexpiry is false, for class-based expiration: # Keep expired article history for 11 days, matching artcutoff plus one. /remember/:11 # Set a default expiration of seven days and honour Expires: headers # within reasonable limits. *:1:7:35 # Class 0 is retained for two weeks and honor Expires: headers # within reasonable limits. 0:1:14:65 HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews. Converted to POD by Russ Allbery <rra@stanford.edu>. $Id: expire.ctl.pod 8970 2010-02-08 21:07:13Z iulius $ SEE ALSO
expire(8), expireover(8), inn.conf(5), innd(8), news.daily(8), storage.conf(5), uwildmat(3). INN 2.5.2 2010-02-08 EXPIRE.CTL(5)