Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Filter out lines of a cvs from values from an other file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Filter out lines of a cvs from values from an other file
# 1  
Old 06-25-2008
Filter out lines of a cvs from values from an other file
Hi there,

I have a comma seperated file which looks like

Code:
16-Jun-08,KLM forwarder,,AMS,DXB,AMS,C,Y,G10,074-02580900,milestone failed - message not received,C,OK,,13/06/2008 00:00,KL427,13/06/2008 00:00,KL427,Rebooked,C,milestone failed - message not received,milestone failed - evented late,C,,milestone failed - evented late,C,,OK,
16-Jun-08,KLM forwarder,,AMS,HEL,AMS,C,Y,,074-02580911,milestone failed - message not received,C,OK,,13/06/2008 00:00,KL1169,13/06/2008 00:00,KL1169,Rebooked,C,milestone failed - message not received,milestone failed - message not received,C,,milestone failed - message not received,C,,milestone failed - message not received,C
17-Jun-08,KLM forwarder,,AMS,ZRH,AMS,C,Y,,074-02580922,milestone failed - message not received,C,OK,,14/06/2008 00:00,KL1957,14/06/2008 00:00,KL1957,Rebooked,C,milestone failed - message not received,OK,,,OK,,,OK,
16-Jun-08,KLM forwarder,,AMS,HEL,AMS,C,Y,,074-02580933,milestone failed - message not received,C,OK,,13/06/2008 00:00,KL1171,13/06/2008 00:00,KL1171,Rebooked,C,milestone failed - message not received,milestone failed - message not received,C,,milestone failed - message not received,C,,milestone failed - message not received,C

and an other file which has a list of number like:
Code:
074-26315295
074-26315155
074-26315240
074-26315354

so what I want to do is to get those lines from the csv file where the value of the 10th field is in the list of the numbers I have in the other file.

Can somebody help me with this?
# 2  
Old 06-25-2008
Code:
awk -F',' 'NR==FNR{_[$1];next} $10 in _{print}' filter data

# 3  
Old 06-25-2008
Works great, just I had to change my filter file to unix style only with LF. But can you please explain it to me because I don't get it how it works.

Thanks
# 4  
Old 06-25-2008
Here is the explanation:
Code:
awk -F',' '      # Set FieldSeparator to (,).
NR==FNR          # While the condition NR==FNR is true. 
{_[$1];next}     # build the array (_) with all the elements from field 1 of the filter file
$10 in _{print}  # For each RS having $10 in array (_), print RS
' filter data

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Shell script to filter certain lines from a file

hi~ i need script on AIX. and have a text file following : create aa 1 2 3 from a@a; create bb from b; create cc 3 4 5 6 6 7 from c@c; (7 Replies)
Discussion started by: tomato00
7 Replies

2. Shell Programming and Scripting

Filter file to remove duplicate values in first column

Hello, I have a script that is generating a tab delimited output file. num Name PCA_A1 PCA_A2 PCA_A3 0 compound_00 -3.5054 -1.1207 -2.4372 1 compound_01 -2.2641 0.4287 -1.6120 3 compound_03 -1.3053 1.8495 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

3. UNIX for Dummies Questions & Answers

Read in Multiple log files and output selected variables and values to cvs file

I have several problems with my problems: I hope you can help me. 1) the If else statement I am getting an error message. My syntax must be incorrect because the entire statement is throwing an error. For example in filew.log if these items don't exist Memsize, SASFoundation and also if... (0 Replies)
Discussion started by: dellanicholson
0 Replies

4. Shell Programming and Scripting

Filter lines based on values at specific positions

hi. I have a Fixed Length text file as input where the character positions 4-5(two character positions starting from 4th position) indicates the LOB indicator. The file structure is something like below: 10126Apple DrinkOmaha 10231Milkshake New Jersey 103 Billabong Illinois ... (6 Replies)
Discussion started by: kumarjt
6 Replies

5. Shell Programming and Scripting

UNIX command -Filter rows in fixed width file based on column values

Hi All, I am trying to select the rows in a fixed width file based on values in the columns. I want to select only the rows if column position 3-4 has the value AB I am using cut command to get the column values. Is it possible to check if cut -c3-4 = AB is true then select only that... (2 Replies)
Discussion started by: ashok.k
2 Replies

6. Shell Programming and Scripting

Filter all the lines with minimum specified length of words of a text file

Hi Can someone tell me which script will work best (in terms of speed and simplicity to write and run) for a large text file to filter all the lines with a minimum specified length of words ? A sample script with be definitely of great help !!! Thanks in advance. :) (4 Replies)
Discussion started by: my_Perl
4 Replies

7. Shell Programming and Scripting

awk to filter out lines containing unique values in a specified column

Hi, I have multiple files that each contain four columns of strings: File1: Code: 123 abc gfh 273 456 ddff jfh 837 789 ghi u4u 395 File2: Code: 123 abc dd fu 456 def 457 nd 891 384 djh 783 I want to compare the strings in Column 1 of File 1 with each other file and Print in... (3 Replies)
Discussion started by: owwow14
3 Replies

8. Linux

Filter a .CSV file based on the 5th column values

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013",""... (2 Replies)
Discussion started by: dhruuv369
2 Replies

9. Shell Programming and Scripting

Shell script to read lines in a text file and filter user data

hi all, I have this file with some user data. example: $cat myfile.txt FName|LName|Gender|Company|Branch|Bday|Salary|Age aaaa|bbbb|male|cccc|dddd|19900814|15000|20| eeee|asdg|male|gggg|ksgu|19911216||| aara|bdbm|male|kkkk|acke|19931018||23| asad|kfjg|male|kkkc|gkgg|19921213|14000|24|... (4 Replies)
Discussion started by: srimal
4 Replies

10. Shell Programming and Scripting

how do I filter double lines from a txt file

Hi, I have a file with the following: access-list.txt router1 access-list 1 permit any any access-list 1 deny any any router2 access-list 2 permit any any access-list 2 deny any any router3 access-list 3 permit any any access-list 3 deny any any I want to hava an output that... (10 Replies)
Discussion started by: I-1
10 Replies
Login or Register to Ask a Question