I've got file A with (say) 1M lines in it ... ascii text, space delimited ...
I've got file B with (say) 10M lines in it ... same structure.
I want to remove any lines from A that appear (identically) in B and print the remaining (say) 900K lines. (And I want to do it in zero time of course!)
Best I've come up with so far is somehow marking the lines in A, then doing a sort and applying an awk script to the result so that the marked lines are only printed if the following (or previous) line isn't "identical" except for the mark.
But after 1000 years of shell programming I've GOT to believe I'm missing an easier/faster solution ... I'm using bash and cygwin tools - and compiling is not an option.
ADVthanksANCE for your help!
=Gneen
Code:
cat fileA | while read line
do
grep -q "$line" fileB
if [ $? -eq 1 ]; then
echo "$line" > fileC
fi
done
Not sure how fast that would be, but fileC will end up with all the lines that were in fileA that were in not in fileB.
I need help to parse a file where there are many records, all of which are consistently separated by lines containing “^=============” and "^ End of Report".
Example:
=============
1
2
3
4
End of record
=============
1
3
4
End of record
Etc....
I only need specific lines... (5 Replies)
Hi,
I have two files. 1st file has 1 column (huge file containing ~19200000 lines) and 2nd file has 2 columns (small file containing ~6000 lines).
#################################
huge_file.txt
a
a
ab
b
##################################
small_file.txt
a 1.5
b 2.5
ab ... (4 Replies)
hi,
i have two files.
file1.sh
echo "unix"
echo "linux"
file2.sh
echo "unix linux forums"
now the output i need is
$./file2.sh
unix linux forums (3 Replies)
I have a configuration file that contains hundreds of these chunks. Each "chunk" is the section that begins with "define service {" and ends with "}".
define service {
check_command check_proc!java
hostgroup_name
service_description ... (5 Replies)
I have a file that looks like this:
cat includes
CORP-CRASHTEST-BU
e:\crashplan\
CORP-TEST
/usr/openv/java
/usr/openv/logs
/usr/openv/man
CORP-LABS_TEST
/usr/openv/java
/usr/openv/logs
/usr/openv/man
What I want to do is make three new files with just those selections. So the three... (4 Replies)
I really can't figure this one out.
I have 2 files, one file is a list of hostnames and the other is a list of their corresponding IPs:
fileA:
example.com
another.org
thirdie.net
fileB:
1.1.1.1
2.2.2.2
3.3.3.3
I want to create a fileC that looks like:
example.com 1.1.1.1... (2 Replies)
Hello All,
this is my first post so I don't know if I am doing this right.
I would like to append entries from a series of strings (contained in a text file) consecutively at the end of specifically labeled lines in another file.
As an example:
- the file that contains the values to be... (3 Replies)
I have a bunch of file numbers in the file 'test':
I'm trying the above command to change all the instances of "H" to "Na+" in the file testsds.pdb at the line numbers indicated in the file 'test'. I've tried the following and various similar alternatives but nothing is working:
cat test |... (3 Replies)
HI,
I have 2 text files. file1 and file2.
file1.txt (There are no duplicates in this file)
1234
3232
4343
3435
6564
6767
1213
file2.txt
1234,wq,wewe,qwqw
1234,as,dfdf,dfdf
4343,asas,sdds,dsds
6767,asas,fdfd,fdffd
I need to search each number in file1.txt in file2.txt's 1st... (6 Replies)
I have a file where every line includes four expressions with a caret in the middle (plus some other "words" or fields, always separated by spaces). I would like to extract from this file, all those lines such that each of the four expressions containing a caret appears in at least four different... (9 Replies)
Discussion started by: uncleMonty
9 Replies
LEARN ABOUT REDHAT
prunehistory
PRUNEHISTORY(8) System Manager's Manual PRUNEHISTORY(8)NAME
prunehistory - remove file names from Usenet history file
SYNOPSIS
prunehistory [ -f filename ] [ -p ]
DESCRIPTION
Prunehistory modifies the history(5) text file to ``remove'' a set of filenames from it. The filenames are removed by overwriting them
with spaces, so that the size and position of any following entries does not change.
Prunehistory reads the standard input. The input is taken as a set of lines. Blank lines and lines starting with a number sign (``#'')
are ignored. All other lines are should consist of a Message-ID followed by zero or more filenames.
The Message-ID is used as the dbz(3) key to get an offset into the text file. If no filenames are mentioned on the input line, then all
filenames in the text are ``removed.'' If any filenames are mentioned, they are converted into the history file notation. If they appear
in the line for the specified Message-ID then they are removed.
Since innd(8) only appends to the text file, prunehistory does not need to have any interaction with it.
OPTIONS -p Prunehistory will normally complain about lines that do not follow the correct format. If the ``-p'' flag is used, then the program
will silently print any invalid lines on its standard output. (Blank lines and comment lines are also passed through.) This can be
useful when prunehistory is used as a filter for other programs such as reap.
-f The default name of the history file is <pathdb in inn.conf>/history; to specify a different name, use the ``-f'' flag.
HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews. This is revision 1.7, dated 1998/04/09.
SEE ALSO dbz(3), history(5), inn.conf(5), innd(8).
PRUNEHISTORY(8)