Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grep records out of file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep records out of file
# 1  
Old 02-06-2012
Grep records out of file
Hi,

I have a file where there "Tab" seperated values are present.I need to identify duplicate entries based on column 1 & 6 only .

For e.g :
Quote:
Input file :

555002160 120203 230611 120204 390 T04DM
555002160 120203 230612 120204 391 T04D2
555002160 120203 230613 120204 392 T04DM
555002160 120203 230614 120204 393 T04D1

Output required :

555002160 120203 230611 120204 390 T04DM
555002160 120203 230613 120204 390 T04DM
I tried using uniq ..but the output is only having one duplicate entry, instead of both the entries.I need both the above entries .

Code:
uniq -f5 -d file1
 
Output :
 
555002160 120203 230613 120204 390 T04DM

# 2  
Old 02-06-2012
Code:
$ sort -t"      " -k6 file | uniq -f5 -D
555002160       120203  230611  120204  390     T04DM
555002160       120203  230613  120204  392     T04DM


Guru.
# 3  
Old 02-06-2012
Thanks Guru...still i was having problems with the sort suggested by you, as the columns in my file were bit more complicated.I just gave an example .

Thanks for ur time.

I managed to crack it using awk...

Code:
 
awk -F"|" '{if (x[$1$6]) { x_count[$1$6]++; print $0; if (x_count[$1$6] == 1) { print x[$1$6] } } x[$1$6] = $0}'  file1

---------- Post updated at 06:07 PM ---------- Previous update was at 06:06 PM ----------

Forgot to mention...i modified the delimiter to "|" instead of tab.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Want to grep records in alphabetical order from a file and split into other files

Hi All, I have one file containing thousands of table names in single column. Now I want that file split into multiple files e.g one file containing table names starting from A, other containing all tables starting from B...and so on..till Z. I tried below but it did not work. for i in... (6 Replies)
Discussion started by: shekhar_4_u
6 Replies

2. Shell Programming and Scripting

Grep 'time' in save records

Hi Team, Is there a way to grep time taken to save records. Its like there is one webpage where when I click save button taking so much time to save result. Therefore, I want to grep that time taken to save that record from file.log Thanks in advance. (1 Reply)
Discussion started by: TCS
1 Replies

3. Shell Programming and Scripting

Separate records of a file on 2 types of records

Hi I am new to shell programming in unix Please if I can provide help. I have a file structure of a header record and "N" detail records. The header record will be the total number of detail records I need to split the file in 2: One for the header Another for all detail records Could... (1 Reply)
Discussion started by: jamcogar
1 Replies

4. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies

5. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

6. Shell Programming and Scripting

Grep matched records from huge file

111111111100000000001111111111 123232323200000010001114545454 232435424200000000001232131212 342354234301000000002323423443 232435424200000000001232131212 2390898994200000000001238908092 This is the record format. From 11th position to 20th position in a record there are 0's occuring,and... (6 Replies)
Discussion started by: mjkreddy
6 Replies

7. Shell Programming and Scripting

grep all records in a file and get a word count -perl

Hi, I have a file .. file.txt .. i need to get a total record count in the files into a $variable.. im using perl script thanks (4 Replies)
Discussion started by: meghana
4 Replies

8. Shell Programming and Scripting

Count No of Records in File without counting Header and Trailer Records

I have a flat file and need to count no of records in the file less the header and the trailer record. I would appreciate any and all asistance Thanks Hadi Lalani (2 Replies)
Discussion started by: guiguy
2 Replies

9. UNIX for Dummies Questions & Answers

using cat and grep to display missing records

Gentle Unix users, Can someone tell me how I can use a combination of the cat and grep command to display records that are in FileA but missing in FileB. cat FileA one line at a time and grep to see if it is in fileB. If it is ignore. If line is not in fileB display the line. Thanks in... (4 Replies)
Discussion started by: jxh461
4 Replies

10. UNIX for Dummies Questions & Answers

how to grep for blank records (space,tab,/n)?

I want to exclude (-v) blank records from a file before analysing it. I know I can use '^]$' for spaces and tabs but how do you look for lines that have nothing (/n or line feed) ? (2 Replies)
Discussion started by: Browser_ice
2 Replies
Login or Register to Ask a Question