Remove lines


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Remove lines
# 1  
Old 09-01-2015
Remove lines

I need to remove rows with the last update date (column 2nd) but when it equals (column1 & column 2) , just remove one of them

input.txt

Code:
6281271467892|2015-01-07
5555571618533|2015-01-01
5555571618533|2015-01-02
5555571800645|2015-01-12
5555567476389|2015-01-04
5555567972749|2015-01-12
5555567972749|2015-01-01
5555583587323|2015-01-15
5555583587323|2015-01-15


Output
Code:
5555571618533|2015-01-02
5555571800645|2015-01-12
5555567476389|2015-01-04
5555567972749|2015-01-12
5555583587323|2015-01-15

Thanks
# 2  
Old 09-01-2015
What have you tried?
# 3  
Old 09-01-2015
i did this

Code:
awk -F'|' '{print $1"|"$2}' input.txt | sort -k2,2nr | awk -F'|' '!a[$1]++'

but the selection was failed
# 4  
Old 09-01-2015
Code:
$ awk -F"|" ' ! arr[$1]++ ' OFS="|" file
6281271467892|2015-01-07
5555571618533|2015-01-01
5555571800645|2015-01-12
5555567476389|2015-01-04
5555567972749|2015-01-12
5555583587323|2015-01-15

Code:
$ sort -t'|' -kn1 -k2r file | awk -F"|" ' ! arr[$1]++ ' OFS="|"
5555567476389|2015-01-04
5555567972749|2015-01-12
5555571618533|2015-01-02
5555571800645|2015-01-12
5555583587323|2015-01-15
6281271467892|2015-01-07

# 5  
Old 10-26-2015
Hi Anbu

Can you explain this bit how it works exactly ? I have idea but not sure ..I think here we have not given actio part which comes under
Code:
 { action }

Hence default action is print here
Code:
 awk -F"|" ' ! arr[$1]++ ' OFS="|" file

# 6  
Old 10-26-2015
Hello krsnadasa,

Following is the explanation for above code, hope it may help you in same.
Code:
awk -F"|" '                     ######## Making field seprator as |
!arr[$1]++                      ######## creating an array named arr whose index is $1's value so ! arr[$1] means if any occurrence of $1 which is NOT present in 
                                        array arr then do it will make entry of that particular item into array an make it's value as one so next time it can't be taken 
                                        as we have put condition as ! arr[$1] 
                                        so if this condition is TRUE means when an entry is being added for $1 then no action is mentioned so it will print that line.
' OFS="|" file                  ######## mentioning Output field separator as | , reading Input_file as named file too

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
# 7  
Old 10-27-2015
Thanks Ravinder...

Very clear...
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Remove lines that are subsets of other lines in File

Hello everyone, Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done. My problem is the following: I have a file where each line is a sequence of IP addresses, example : 10.0.0.1 10.0.0.2 10.0.0.5 10.0.0.1 10.0.0.2... (5 Replies)
Discussion started by: MisterJellyBean
5 Replies

3. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies

4. UNIX for Dummies Questions & Answers

Want to remove all lines but not latest 50 lines from a file

Hi, I have a huge file which has Lacs of lines. File system got full. I want your guys help to suggest me a solution so that I can remove all lines from that file but not last 50,000 lines. I want solution which can remove lines from existing file so that I can have some space left with. (28 Replies)
Discussion started by: prashant2507198
28 Replies

5. Shell Programming and Scripting

Remove x lines form top and y lines form bottom using AWK?

How to remove x lines form top and y lines form bottom. This works, but like awk only cat file | head -n-y | awk 'NR>(x-1)' so remove last 3 lines and 5 firstcat file | head -n-3 | awk 'NR>4' (5 Replies)
Discussion started by: Jotne
5 Replies

6. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

7. Shell Programming and Scripting

Remove all lines except lines starting with [

Hello, I am trying to remove all the lines in file except lines starting with [ How can i accomplish this? Thank you very much in advance. (4 Replies)
Discussion started by: maxo
4 Replies

8. Shell Programming and Scripting

Remove lines

i want to remove the records starting with abc or xyz in the file without redirecting the original file. sample : file1.txt abc1234 124234 2020202 3242342 xyz2342 2afafa Expecting output 124234 2020202 3242342 2afafa (7 Replies)
Discussion started by: Jairaj
7 Replies

9. Shell Programming and Scripting

Remove All Lines Between Two Unique Lines

Hi all! Im wondering if its possible to remove all lines between two lines. Im working with a document like this: data1 data2 <Remove> data3 data4 </Remove> data5 data6 I need it to end up like this if that possible: data1 data2 data5 data6 There are multiple instances of... (2 Replies)
Discussion started by: Grizzly
2 Replies

10. Shell Programming and Scripting

Need to remove lines

Hi All, I have a xml file . And I need to remove the header inofrmation and the tralier information from the xml file. I have pasted the sample xml below: <?xml version="1.0" encoding="UTF-8" ?> - <!-- Generated by Ascential Software Corporation, DataStage - XMLOutput stage - Mon Nov... (5 Replies)
Discussion started by: kumar66
5 Replies
Login or Register to Ask a Question