Sponsored Content
Top Forums UNIX for Beginners Questions & Answers How to delete identical lines while leaving one undeleted? Post 302979722 by RavinderSingh13 on Thursday 18th of August 2016 08:28:46 AM
Old 08-18-2016
Hello beginner_99,

Could you please try following and let us know how it goes then.
Code:
awk '!A[$0]++'  Input_file

OR
Code:
uniq -u  Input_file

Output will be as follows in both of above commands.
Code:
Hello
Hi
His
Hungry
hi

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

delete string in a text file leaving the first occurrence

Hi, How can i delete the second and subsequent occurrence of a particular string from a file ? eg) test.txt cattle bat battle mat matter cattle cattle my output file should be cattle bat battle mat matter (12 Replies)
Discussion started by: gopskrish
12 Replies

2. Shell Programming and Scripting

replace 2 identical strings on different lines

I am looking to replace two or more strings on different lines using sed, but not with the same variable. IE # cat xxx.file <abc> abc def ghi abc def ghi abc def ghi currently I can only change each line with the same pattern: # sed -e '/<abc>/!s/abc\(.*\)/jkl mno/' xxx.file abc jkl mno... (3 Replies)
Discussion started by: prkfriryce
3 Replies

3. Shell Programming and Scripting

Ignore identical lines

Hello Experts, I have two files called "old" and "new". My old file contains 10 lines and my new file contains 10 + "n" lines. The first field in both these files contain ID. I sort these two files on ID. I am interested in only the lines that are in the new file and not in old. I tried... (4 Replies)
Discussion started by: forumthreads
4 Replies

4. Shell Programming and Scripting

sed/awk: Delete matching words leaving only the first instance

I have an input text that looks like this (comes already sorted): on Caturday 22 at 10:15, some event on Caturday 22 at 10:15, some other event on Caturday 22 at 21:30, even more events on Funday 23 at 11:00, yet another event I need to delete all the matching words between the lines, from... (2 Replies)
Discussion started by: GrinningArmor
2 Replies

5. UNIX for Dummies Questions & Answers

Delete all rows but leaving first and last ones

Hello, Merry Christmas to all! I wish you the best for these holidays and the best for the next year 2011. I'd like your help please, I need to delete all the rows in the third column of my file, but without touching nor changing the first and last value position, this is an example of my... (2 Replies)
Discussion started by: Gery
2 Replies

6. Shell Programming and Scripting

Using Bash/Sed to delete between identical strings

Hi. I'm hoping that someone can help me with a bash script to delete a block of lines from a file. What I want to do is delete every line between two stings that are the same, including the line the first string is on but not the second. (Marked lines to match with !) For example if I... (2 Replies)
Discussion started by: Zykr
2 Replies

7. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

8. UNIX for Dummies Questions & Answers

more than 10 identical lines

I have a file that looks like this 10 user1s, 5 user2s and 10 users3. 10.10.1.1 user1 10.10.1.1 user1 10.10.1.1 user1 10.10.1.1 user1 10.10.1.1 user1 10.10.1.1 user1 10.10.1.1 user1 10.10.1.1 user1 10.10.1.1 user1 10.10.1.1 user1 10.10.1.2 user2 10.10.1.2 user2 10.10.1.2 user2... (7 Replies)
Discussion started by: lawsongeek
7 Replies

9. Shell Programming and Scripting

sed print all lines between second and third identical lines

I am trying to extract a table of data (mysql query output) from a log file. I need to print everything below the header and not past the end of the table. I have spent many hours searching with little progress. I am matching the regexp +-\{99\} with no problem. I just can't figure out how to print... (5 Replies)
Discussion started by: godfreydanials
5 Replies

10. Shell Programming and Scripting

Leaving last 30 lines

If we want to display lines from file leaving last 30 lines. i dont know the count of lines in file (14 Replies)
Discussion started by: mirwasim
14 Replies
DIFF3(1)						      General Commands Manual							  DIFF3(1)

NAME
diff3 - 3-way differential file comparison SYNOPSIS
diff3 [ -exEX3 ] file1 file2 file3 DESCRIPTION
Diff3 compares three versions of a file, and publishes disagreeing ranges of text flagged with these codes: ==== all three files differ ====1 file1 is different ====2 file2 is different ====3 file3 is different The type of change suffered in converting a given range of a given file to some other is indicated in one of these ways: f : n1 a Text is to be appended after line number n1 in file f, where f = 1, 2, or 3. f : n1 , n2 c Text is to be changed in the range line n1 to line n2. If n1 = n2, the range may be abbreviated to n1. The original contents of the range follows immediately after a c indication. When the contents of two files are identical, the contents of the lower-numbered file is suppressed. Under the -e option, diff3 publishes a script for the editor ed that will incorporate into file1 all changes between file2 and file3, i.e. the changes that normally would be flagged ==== and ====3. Option -x (-3) produces a script to incorporate only changes flagged ==== (====3). The following command will apply the resulting script to `file1'. (cat script; echo '1,$p') | ed - file1 The -E and -X are similar to -e and -x, respectively, but treat overlapping changes (i.e., changes that would be flagged with ==== in the normal listing) differently. The overlapping lines from both files will be inserted by the edit script, bracketed by "<<<<<<" and ">>>>>>" lines. For example, suppose lines 7-8 are changed in both file1 and file2. Applying the edit script generated by the command "diff3 -E file1 file2 file3" to file1 results in the file: lines 1-6 of file1 <<<<<<< file1 lines 7-8 of file1 ======= lines 7-8 of file3 >>>>>>> file3 rest of file1 The -E option is used by RCS merge(1) to insure that overlapping changes in the merged files are preserved and brought to someone's atten- tion. FILES
/tmp/d3????? /usr/libexec/diff3 SEE ALSO
diff(1) BUGS
Text lines that consist of a single `.' will defeat -e. 7th Edition October 21, 1996 DIFF3(1)
All times are GMT -4. The time now is 04:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy