10-31-2006
Hi npolayan,
I can able to create the temp file with appropriate values to look at but when i executed the awk statement
awk '{ for ($0 in `cat temp`)
key[$0]++
if(key[$0] < 2 ) print $0
}' inputfile > outputfile
I am getting error messages
illegal statement at source line 1
illegal statement at source line 4
Let me know if you have the answer.
Thanks in advance.
cheers,
gops
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have an input text that looks like this (comes already sorted):
on Caturday 22 at 10:15, some event
on Caturday 22 at 10:15, some other event
on Caturday 22 at 21:30, even more events
on Funday 23 at 11:00, yet another event
I need to delete all the matching words between the lines, from... (2 Replies)
Discussion started by: GrinningArmor
2 Replies
2. UNIX for Dummies Questions & Answers
Hello,
Merry Christmas to all! I wish you the best for these holidays and the best for the next year 2011.
I'd like your help please, I need to delete all the rows in the third column of my file, but without touching nor changing the first and last value position, this is an example of my... (2 Replies)
Discussion started by: Gery
2 Replies
3. UNIX for Dummies Questions & Answers
Hi,
I have a text file and I want to delete all rows that contain a particular string of characters. How do I go about doing that? Thanks! (9 Replies)
Discussion started by: evelibertine
9 Replies
4. UNIX for Advanced & Expert Users
Hi All,
I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space.
I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies
5. Shell Programming and Scripting
Hi All,
I have a small query. I have a file containing the following lines
File 1:
29-Jul-2011 GMT Static data requires update <Extraction should start here>
-----------
-----------
--------------------
-----------------------
----------- <should stop here>
Pattern1 will be time... (2 Replies)
Discussion started by: gangii87
2 Replies
6. Shell Programming and Scripting
Hi
I have requirement to find nth occurrence in a file and capture data from with in lines (between lines)
Data in File.
<QUOTE>
<SESSION>
<ATTRIBUTE NAME='Parameter Filename' VALUE='file1.parm'/>
<ATTRIBUTE NAME='Service Name' VALUE='None'/>
</SESSION>
<SESSION>
<ATTRIBUTE... (6 Replies)
Discussion started by: tmalik79
6 Replies
7. Shell Programming and Scripting
Hi,
I want to add a text to the end of the specific line in a file. Now my file looks like this:
999
111
222
333
111
444
I want to add the string " 555" to the end of the first line contaning 111. Moreover, I want to insert a newline after this line containg the "000" string. The... (8 Replies)
Discussion started by: wenclu
8 Replies
8. Shell Programming and Scripting
if there's a file containing:
money king money queen money cat money also money king
all those strings are on one line in the file.
how can i find out how many times "money king" shows up in the line?
egrep -c "money king" wont work. (7 Replies)
Discussion started by: SkySmart
7 Replies
9. Shell Programming and Scripting
Hi,
I would like to know how, using sed, be able to insert contents of file2 in file1 after say the second occurrence of a given string? e.g.
> cat file1
banana
apple
orange
apple
banana
pear
tangerine
apple
> cat file2
I don't like apples
What would be the sed command to insert... (5 Replies)
Discussion started by: dimocn
5 Replies
10. UNIX for Beginners Questions & Answers
Hi,
I have a file as follows.
file1
Hello
Hi
His
Hi
Hi
Hungry
hi
so I want to delete identical lines while leaving one of them undeleted.
So desired output will be
Hello
Hi (2 Replies)
Discussion started by: beginner_99
2 Replies
LEARN ABOUT DEBIAN
fdupes
FDUPES(1) General Commands Manual FDUPES(1)
NAME
fdupes - finds duplicate files in a given set of directories
SYNOPSIS
fdupes [ options ] DIRECTORY ...
DESCRIPTION
Searches the given path for duplicate files. Such files are found by comparing file sizes and MD5 signatures, followed by a byte-by-byte
comparison.
OPTIONS
-r --recurse
for every directory given follow subdirectories encountered within
-R --recurse:
for each directory given after this option follow subdirectories encountered within (note the ':' at the end of option; see the
Examples section below for further explanation)
-s --symlinks
follow symlinked directories
-H --hardlinks
normally, when two or more files point to the same disk area they are treated as non-duplicates; this option will change this behav-
ior
-n --noempty
exclude zero-length files from consideration
-f --omitfirst
omit the first file in each set of matches
-A --nohidden
exclude hidden files from consideration
-1 --sameline
list each set of matches on a single line
-S --size
show size of duplicate files
-m --summarize
summarize duplicate files information
-q --quiet
hide progress indicator
-d --delete
prompt user for files to preserve, deleting all others (see CAVEATS below)
-N --noprompt
when used together with --delete, preserve the first file in each set of duplicates and delete the others without prompting the user
-v --version
display fdupes version
-h --help
displays help
SEE ALSO
md5sum(1)
NOTES
Unless -1 or --sameline is specified, duplicate files are listed together in groups, each file displayed on a separate line. The groups are
then separated from each other by blank lines.
When -1 or --sameline is specified, spaces and backslash characters () appearing in a filename are preceded by a backslash character.
EXAMPLES
fdupes a --recurse: b
will follow subdirectories under b, but not those under a.
fdupes a --recurse b
will follow subdirectories under both a and b.
CAVEATS
If fdupes returns with an error message such as fdupes: error invoking md5sum it means the program has been compiled to use an external
program to calculate MD5 signatures (otherwise, fdupes uses internal routines for this purpose), and an error has occurred while attempting
to execute it. If this is the case, the specified program should be properly installed prior to running fdupes.
When using -d or --delete, care should be taken to insure against accidental data loss.
When used together with options -s or --symlink, a user could accidentally preserve a symlink while deleting the file it points to.
Furthermore, when specifying a particular directory more than once, all files within that directory will be listed as their own duplicates,
leading to data loss should a user preserve a file without its "duplicate" (the file itself!).
AUTHOR
Adrian Lopez <adrian2@caribe.net>
FDUPES(1)