Sponsored Content
Top Forums UNIX for Dummies Questions & Answers delete string in a text file leaving the first occurrence Post 302094727 by gopskrish on Tuesday 31st of October 2006 09:32:23 AM
Old 10-31-2006
Hi npolayan,

I can able to create the temp file with appropriate values to look at but when i executed the awk statement
awk '{ for ($0 in `cat temp`)
key[$0]++
if(key[$0] < 2 ) print $0
}' inputfile > outputfile

I am getting error messages

illegal statement at source line 1
illegal statement at source line 4

Let me know if you have the answer.

Thanks in advance.

cheers,
gops
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed/awk: Delete matching words leaving only the first instance

I have an input text that looks like this (comes already sorted): on Caturday 22 at 10:15, some event on Caturday 22 at 10:15, some other event on Caturday 22 at 21:30, even more events on Funday 23 at 11:00, yet another event I need to delete all the matching words between the lines, from... (2 Replies)
Discussion started by: GrinningArmor
2 Replies

2. UNIX for Dummies Questions & Answers

Delete all rows but leaving first and last ones

Hello, Merry Christmas to all! I wish you the best for these holidays and the best for the next year 2011. I'd like your help please, I need to delete all the rows in the third column of my file, but without touching nor changing the first and last value position, this is an example of my... (2 Replies)
Discussion started by: Gery
2 Replies

3. UNIX for Dummies Questions & Answers

Delete all rows that contain a specific string (text)

Hi, I have a text file and I want to delete all rows that contain a particular string of characters. How do I go about doing that? Thanks! (9 Replies)
Discussion started by: evelibertine
9 Replies

4. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

5. Shell Programming and Scripting

Extracting text between two patterns 1 and 2 and pattern2 should be second occurrence of the file

Hi All, I have a small query. I have a file containing the following lines File 1: 29-Jul-2011 GMT Static data requires update <Extraction should start here> ----------- ----------- -------------------- ----------------------- ----------- <should stop here> Pattern1 will be time... (2 Replies)
Discussion started by: gangii87
2 Replies

6. Shell Programming and Scripting

find string nth occurrence in file and print line number

Hi I have requirement to find nth occurrence in a file and capture data from with in lines (between lines) Data in File. <QUOTE> <SESSION> <ATTRIBUTE NAME='Parameter Filename' VALUE='file1.parm'/> <ATTRIBUTE NAME='Service Name' VALUE='None'/> </SESSION> <SESSION> <ATTRIBUTE... (6 Replies)
Discussion started by: tmalik79
6 Replies

7. Shell Programming and Scripting

Adding text to the end of the specific line in a file(only to the first occurrence of it)

Hi, I want to add a text to the end of the specific line in a file. Now my file looks like this: 999 111 222 333 111 444 I want to add the string " 555" to the end of the first line contaning 111. Moreover, I want to insert a newline after this line containg the "000" string. The... (8 Replies)
Discussion started by: wenclu
8 Replies

8. Shell Programming and Scripting

Count number of occurrence of a string in file

if there's a file containing: money king money queen money cat money also money king all those strings are on one line in the file. how can i find out how many times "money king" shows up in the line? egrep -c "money king" wont work. (7 Replies)
Discussion started by: SkySmart
7 Replies

9. Shell Programming and Scripting

How to insert file contents after nth occurrence of a string using sed?

Hi, I would like to know how, using sed, be able to insert contents of file2 in file1 after say the second occurrence of a given string? e.g. > cat file1 banana apple orange apple banana pear tangerine apple > cat file2 I don't like apples What would be the sed command to insert... (5 Replies)
Discussion started by: dimocn
5 Replies

10. UNIX for Beginners Questions & Answers

How to delete identical lines while leaving one undeleted?

Hi, I have a file as follows. file1 Hello Hi His Hi Hi Hungry hi so I want to delete identical lines while leaving one of them undeleted. So desired output will be Hello Hi (2 Replies)
Discussion started by: beginner_99
2 Replies
FDUPES(1)						      General Commands Manual							 FDUPES(1)

NAME
fdupes - finds duplicate files in a given set of directories SYNOPSIS
fdupes [ options ] DIRECTORY ... DESCRIPTION
Searches the given path for duplicate files. Such files are found by comparing file sizes and MD5 signatures, followed by a byte-by-byte comparison. OPTIONS
-r --recurse for every directory given follow subdirectories encountered within -R --recurse: for each directory given after this option follow subdirectories encountered within (note the ':' at the end of option; see the Examples section below for further explanation) -s --symlinks follow symlinked directories -H --hardlinks normally, when two or more files point to the same disk area they are treated as non-duplicates; this option will change this behav- ior -n --noempty exclude zero-length files from consideration -f --omitfirst omit the first file in each set of matches -A --nohidden exclude hidden files from consideration -1 --sameline list each set of matches on a single line -S --size show size of duplicate files -m --summarize summarize duplicate files information -q --quiet hide progress indicator -d --delete prompt user for files to preserve, deleting all others (see CAVEATS below) -N --noprompt when used together with --delete, preserve the first file in each set of duplicates and delete the others without prompting the user -v --version display fdupes version -h --help displays help SEE ALSO
md5sum(1) NOTES
Unless -1 or --sameline is specified, duplicate files are listed together in groups, each file displayed on a separate line. The groups are then separated from each other by blank lines. When -1 or --sameline is specified, spaces and backslash characters () appearing in a filename are preceded by a backslash character. EXAMPLES
fdupes a --recurse: b will follow subdirectories under b, but not those under a. fdupes a --recurse b will follow subdirectories under both a and b. CAVEATS
If fdupes returns with an error message such as fdupes: error invoking md5sum it means the program has been compiled to use an external program to calculate MD5 signatures (otherwise, fdupes uses internal routines for this purpose), and an error has occurred while attempting to execute it. If this is the case, the specified program should be properly installed prior to running fdupes. When using -d or --delete, care should be taken to insure against accidental data loss. When used together with options -s or --symlink, a user could accidentally preserve a symlink while deleting the file it points to. Furthermore, when specifying a particular directory more than once, all files within that directory will be listed as their own duplicates, leading to data loss should a user preserve a file without its "duplicate" (the file itself!). AUTHOR
Adrian Lopez <adrian2@caribe.net> FDUPES(1)
All times are GMT -4. The time now is 05:40 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy