LR_ANONYMIZE.IN(1)					  LogReport's Lire Documentation					LR_ANONYMIZE.IN(1)

NAME

       lr_anonymize - anonymize a logfile

SYNOPSIS

       lr_anonymize dumpfilestem

DESCRIPTION

       lr_anonymize is typically used when sending logs to a responder.  See the section on "Sending Anonymized Log Files To A Responder" in the
       chapter on "Using A Responder" in the Lire User Manual for usage examples.

       lr_anonymize reads a file containing emailaddresses, ipnumbers, and hostnames (typically a logfile from an internet service) from stdin,
       and prints an "anonymized" version of this file to stdout. It dumps the information to "deanonymize" the file, using lr_deanonymize(1), to
       5 Berkeley DB databases; the names of the files holding these databases is contructed by concatenating the strings 'inaddr', 'maildomain',
       'email', 'ip' and 'domain' to dumpfilestem. dumpfilestem can be e.g. /tmp/dump, ../../var/dump or dump.	The db files created silently
       overwrite stale ones, if found.

       The script builds temporary files. By default, these are created in /tmp.  One can override this by setting the TMPDIR environment
       variable.

CONNECTION WITH lr_deanonymize
       When running

	$ lr_anonymize dump < log > log.anon
	$ lr_deanonymize dump < log.anon > log.new

       then log and log.new have the same content (except for case, check it with diff -i).

NOTES

       We tried to optimize this script for memoryusage. This has the drawback the script will run for quite some time when anonymizing a big
       logfile.

       We've run the script on a 25 M sendmail logfile.

       Typical values in such a case are 2500 K for total amount of physical memory used, and 15m real, 8m user and 22s systime spent on a 64 MB
       system with a 300 MHz Pentium II processor. de_anonymizing this file took 11m real, 8m user and 17s system time.

       We store maildomains in the dumpfile. These are used by lr_deanonymize(1), in case email addresses in our input file reoccur in the file
       read by lr_deanonymize(1) in split form, i.e. joe.user@example.com occurs as both joe.user@example.com and example.com in the to be
       deanonymized file.

       All dumped objects are casted to lowercase.

BACKGROUND

       For your convenience, we quote a bit of rfc822:

	SPACE	    =  <ASCII SP, space>	    ; (     40,      32.)
	CTL	    =  <any ASCII control	    ; (  0- 37,  0.- 31.)
			character and DEL>	    ; (    177,     127.)
	specials    =  "(" / ")" / "<" / ">" / "@"  ; Must be in quoted-
		    /  "," / ";" / ":" / "" / <">  ;  string, to use
		    /  "." / "[" / "]"		    ;  within a word.
	atom	    =  1*<any CHAR except specials, SPACE and CTLs>
	quoted-string = <"> *(qtext/quoted-pair) <">; Regular qtext or
						    ;	quoted chars.
	word	    =  atom / quoted-string
	domain-ref  =  atom			    ; symbolic reference
	domain-literal =  "[" *(dtext / quoted-pair) "]"
	sub-domain  =  domain-ref / domain-literal
	domain	    =  sub-domain *("." sub-domain)
	local-part  =  word *("." word) 	    ; uninterpreted
						    ; case-preserved
	addr-spec   =  local-part "@" domain	    ; global address

       and of rfc 2181

	The DNS itself places only one restriction on the particular labels
	that can be used to identify resource records.	That one restriction
	relates to the length of the label and the full name.  The length of
	any one label is limited to between 1 and 63 octets.  A full domain
	name is limited to 255 octets (including the separators).

       rfc1123

	However, a valid host name can never
	have the dotted-decimal form #.#.#.#, since at least the
	highest-level component label will be alphabetic.

       rfc819

	<domain> ::= <naming-domain> | <naming-domain> "." <domain>
	<naming-domain> ::=  <simple-name> | <address>
	<simple-name> ::= <a> <ldh-str> <let-dig>
	<ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> <ldh-str>
	<let-dig> ::= <a> | <d>
	<let-dig-hyp> ::= <a> | <d> | "-"
	<a> ::= any one of the 52 alphabetic characters A through Z in upper
	case and a through z in lower case
	<d> ::= any one of the ten digits 0 through 9

EXAMPLE

       A 'logfile' like e.g.

	blaat fkrf 1.2.3.4.in-addr.arpa] pietje@bigcompany.com bla 1 2 3 lj;agas;gag
	blaat 1.2.3.4 fkrf 3.2.3.4.in-addr.arpa] bla 1 www.hotsex.com 2 3 lj;agas;gag
	jan@blaat.frut.com agagag
	blaat fkrf 4.2.3.4.in-addr.arpa] bla pietje@bigcompany.com www.hotsex.com
	234.34.2.0 jan@blaat.frut.com 4.2.3.4.in-addr.arpa1 2 3 lj;agas;gag
	blaat fkrf tweede 3.2.3.4.in-addr.arpa] bla 1.2.3.4 1 blablabla.com
	2 mdcc.cx
	3 lj;agas;gag

       wil get anonymized to

	blaat fkrf 1.0.0.10.in-addr.arpa] john.doe.1@example.com bla 1 2 3 lj;agas;gag
	blaat 10.0.0.1 fkrf 2.0.0.10.in-addr.arpa] bla 1 1.example.com 2 3 lj;agas;gag
	john.doe.2@example.com agagag
	blaat fkrf 3.0.0.10.in-addr.arpa] bla john.doe.1@example.com 1.example.com
	10.0.0.2 john.doe.2@example.com 3.0.0.10.in-addr.arpa1 2 3 lj;agas;gag
	blaat fkrf tweede 2.0.0.10.in-addr.arpa] bla 10.0.0.1 1 2.example.com
	2 3.example.com
	3 lj;agas;gag

       The dumps will represent something like

	ip 234.34.2.0 10.0.0.2
	ip 1.2.3.4 10.0.0.1
	inaddr 3.2.3.4.in-addr.arpa 2.0.0.10.in-addr.arpa
	inaddr 1.2.3.4.in-addr.arpa 1.0.0.10.in-addr.arpa
	inaddr 4.2.3.4.in-addr.arpa 3.0.0.10.in-addr.arpa
	domain mdcc.cx 3.example.com
	domain blablabla.com 2.example.com
	domain www.hotsex.com 1.example.com
	email jan@blaat.frut.com john.doe.2@example.com
	email pietje@bigcompany.com john.doe.1@example.com

BUGS

       We can't handle files containing hostnames or email addresses in the example.com domain, usernames of the form john.doe.<someletters> or
       ipnumbers in the rfc 1918 private network 10.0.0.0 - 10.255.255.255 (10/8 prefix).

       We don't handle quoted-string's in email addresses. We don't handle domain-literals in email addresses' domain.

       We regard 999.999.999.999 as an IP address: we don't mind the 255 limit.

       We don't treat network ipaddresses like 100.10.3 as ipaddresses. These will not get anonymized.

SEE ALSO

       lr_deanonymize(1), the README.lire-client file in the Lire distribution.

VERSION

       $Id: lr_anonymize.in,v 1.5 2006/07/23 13:16:32 vanbaal Exp $

COPYRIGHT

       Copyright (C) 2000-2001 Stichting LogReport Foundation LogReport@LogReport.org

       This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
       the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

       This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with this program (see COPYING); if not, check with
       http://www.gnu.org/copyleft/gpl.html.

AUTHOR

       Joost van Baal <joostvb@logreport.org>

Lire 2.1.1							    2006-07-23							LR_ANONYMIZE.IN(1)