Matching 10 Million file records with 10 Million in other file Post: 302654743

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract data from large file 80+ million records

Hello, I have got one file with more than 120+ million records(35 GB in size). I have to extract some relevant data from file based on some parameter and generate other output file. What will be the besat and fastest way to extract the ne file. sample file format :--...

2. Shell Programming and Scripting

sort a file which has 3.7 million records

hi, I'm trying to sort a file which has 3.7 million records an gettign the following error...any help is appreciated... sort: Write error while merging. Thanks

3. What is on Your Mind?

Pick a Number Between 0 and 20 for 1 Million Bits

Here is an easy game! I wrote a number between 0 and 20 (that can include 0 and 20) on a piece of paper. I am staring at it now, imagining the number so you can read my mind ;) Reply once, and only once, with a number from 0 to 20 and the first person to guess it wins 1,000,000 Bits. ...

4. Shell Programming and Scripting

Tail 86000 lines from 1.2 million line file?

I have a log file that is about 1.2 million lines long and about 300MB. we need a way to clean up this file and only keep the last few thousand lines. if i use tail command we run our of memory as the file is too big. I do have a key word to match on. example, we want to keep every line...

5. UNIX for Dummies Questions & Answers

Pls. help with script to remove million files

Hi, one of the server, log directory was never cleaned up. We have so many files. I want to remove all the files that starts with dfr* but I get error message when I use the *. rm qfr* bash: /usr/bin/rm: Arg list too long I am trying to write this script but not working. ...

6. UNIX for Dummies Questions & Answers

Deleting a million of files ..

Hi, Which way is faster rm -rf /path/ or find / -name -exec rm {} \; and why?

7. UNIX for Dummies Questions & Answers

Add 1 million columns

Hi, here is my problem: I've got a file with 6 columns (file1): a b c d e f a b c d e f a b c d e f a b c d e f I need to add 1 million columns to this file, each column needs to be a zero. Here is how the result file (file2) should look like (for the sake of the example, I've only...

LEARN ABOUT BSD

expireover

EXPIREOVER(8)						      System Manager's Manual						     EXPIREOVER(8)

NAME

       expireover - Expire entries from the news overview database

SYNOPSIS

       expireover [ -a ] [ -D overviewdir ] [ -f file ] [ -n ] [ -O overview.fmt ] [ -s ] [ -v ] [ -z ] [ file...  ]

DESCRIPTION

       Expireover  expires  entries  from  the	news  overview	database.   It	reads  a  list	of  pathnames  (relative  to  the spool directory,
       /var/spool/news), from the specified files or standard input if none are specified.  (A file name of ``-'' may be used to specify the stan-
       dard input.)  It then removes any mention of those articles from the appropriate overview database.

OPTIONS

       -z     If  the ``-z'' flag is used, then the input is assumed to be sorted such that all entries for a newsgroup appear together so that it
	      can be purged at once.  This flag can be useful when used with the sorted output of expire(8)'s ``-z'' flag.

       -s     If the ``-s'' flag is used, then expireover will read the spool directory for all groups mentioned in the active(5) file, and remove
	      the overview entries of any articles that do not appear in the directory.

       -f     To specify an alternate file, use the ``-f'' flag; a name of ``-'' is taken to mean the standard input.

       -a     The  ``-a''  flag  reads the spool directory and adds any missing overview entries.  It will create files if necessary.  This can be
	      used to initialize a database, or to sync up a overview database that may be lacking articles due to a crash.   Overchan	should	be
	      running, to ensure that any incoming articles get included.  Using this flag implies the ``-s'' flag; the ``-f'' flag may be used to
	      add only a subset of the newsgroups.

       -v     To see a list of the entries that would be added or deleted, use the ``-v'' flag.

       -n     To perform no real updates, use the ``-n'' flag.

       -D     The ``-D'' flag can be used to specify where the databases are stored.  The default directory is /var/spool/news/over.view.

       -O     The ``-O'' flag may be used to specify an alternate location for the overview.fmt(5) file; this is normally only useful  for  debug-
	      ging.

HISTORY

       Written	by  Rob  Robertson <rob@violet.berkeley.edu> and Rich $alz <rsalz@uunet.uu.net> (with help from Dave Lawrence <tale@uunet.uu.net>)
       for InterNetNews.  This is revision 1.8, dated 1996/10/29.

SEE ALSO

       expire(8), overview.fmt(5).

																     EXPIREOVER(8)