Select only the lines of a file starting with a field which is matcing a list. awk?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Select only the lines of a file starting with a field which is matcing a list. awk?
# 1  
Old 09-11-2014
Select only the lines of a file starting with a field which is matcing a list. awk?

Hello
I have a large file1 which has many events like "2014010420" and following lines under each event that start with text . It has this form:
Code:
2014010420 num --- ---  num   ....

NTE           num num --- num...      
EFA           num num ---  num  ...   
LASW         num num ---  num ... 
   
2014010450  num num num  ...                                    
   
STR           num num ---  num ..
KRP           num num --- num ...

2014012940  num num --- num  ... 
 
SARA       num num --- num ...
TREN        num num --- num.......

("-"=space)
and another file2 which is the list of the events that need to be selected, it has the form:
Code:
2014010420
2014012450...

I need using the list of the events at file 2 to select the events and their following lines (only until the next event) of file1. The format must remain the same. thanks in advance for yr help

Last edited by Scrutinizer; 09-12-2014 at 12:25 AM.. Reason: CODE tags
# 2  
Old 09-12-2014
Please use [code] and [/code] tags around any sample data to assist keeping the format untouched.

Is it true that blank lines appear before and after each event line? Are these the only blank lines that appear in the file and can these be used to distinguish the start/end of each event block?
# 3  
Old 09-12-2014
Try this...

Code:
awk 'FNR==NR{m[$1];next};/^2/{x=($1 in m)}x' file2 file1


Last edited by pilnet101; 09-12-2014 at 06:59 AM..
This User Gave Thanks to pilnet101 For This Post:
# 4  
Old 09-12-2014
Quote:
Originally Posted by pilnet101
Try this...

Code:
awk 'FNR==NR{m[$1]++;next};/^2/{x=($1 in m)}x' file2 file1

Can you explain your code, please ? i don't understand why detect the line which begin with the number 2 (/^2/) and why x ?
This User Gave Thanks to protocomm For This Post:
# 5  
Old 09-12-2014
Sure.

The first part of the code
Code:
awk 'FNR==NR{m[$1]++;next}

is reading and storing the contents of $1 from file2 into an array called m. So this will be made up of all the 2014* values.

The second part of the code
Code:
/^2/{x=($1 in m)}x'

is what is run for each line in file1. It basically says for each line beginning with a 2 (I made the assumption from the data provided by OP that each value in file2 would start with 2014, so in effect this can be elaborated to /^2014/ OR /^201/), check to see if this 2014* value (ie. $1) is in the array m. Set x to the return code of the 'in array' function which will exit 1 if true and 0 is false. Then pass x outside the action block to perform the default action of printing the line, when x is not 0.

Hope this helps, sorry if I have confused things more!
This User Gave Thanks to pilnet101 For This Post:
# 6  
Old 09-12-2014
Quote:
Originally Posted by pilnet101
Sure.

The first part of the code
Code:
awk 'FNR==NR{m[$1]++;next}

is reading and storing the contents of $1 from file2 into an array called m. So this will be made up of all the 2014* values.

The second part of the code
Code:
/^2/{x=($1 in m)}x'

is what is run for each line in file1. It basically says for each line beginning with a 2 (I made the assumption from the data provided by OP that each value in file2 would start with 2014, so in effect this can be elaborated to /^2014/ OR /^201/), check to see if this 2014* value (ie. $1) is in the array m. Set x to the return code of the 'in array' function which will exit 1 if true and 0 is false. Then pass x outside the action block to perform the default action of printing the line, when x is not 0.

Hope this helps, sorry if I have confused things more!
thx for your explanation, but i don't understand how the output print the line between the line which begin by 2
This User Gave Thanks to protocomm For This Post:
# 7  
Old 09-12-2014
x is reassigned every time a line starting with "2" is read, and is set to 1 (="on") if that line's $1 is found in the m array, to 0 (="off") if not.
ALL lines are printed as long as x is 1.

Last edited by RudiC; 09-12-2014 at 09:14 AM.. Reason: typo
These 2 Users Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to grep a line not starting with # from a file (there are two lines starting with # and normal)?

e.g. File name: File.txt cat File.txt Result: #INBOUND_QUEUE=FAQ1 INBOUND_QUEUE=FAQ2 I want to get the value for one which is not commented out. Thanks, (3 Replies)
Discussion started by: Tanu
3 Replies

2. Shell Programming and Scripting

awk to remove lines in file if specific field matches

I am trying to remove lines in the target.txt file if $5 before the - in that file matches sorted_list. I have tried grep and awk. Thank you :). grep grep -v -F -f targets.bed sort_list grep -vFf sort_list targets awk awk -F, ' > FILENAME == ARGV {to_remove=1; next} > ! ($5 in... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

awk repeat one field at all lines and modify field repetitions

Hello experts I have a file with paragraphs begining with a keeping date and ending with "END": 20120301 num num John num num A keepnum1 num num kathrin num num A keepnum1 num num kathrin num num B keepnum2 num num Pete num num A keepnum1 num num Jacob num... (2 Replies)
Discussion started by: phaethon
2 Replies

4. Shell Programming and Scripting

Using awk to select one field

Hi, I saw your post.. I have a dought in awk command... how to get the output from a file. i need a first column in etc/passwd file in a single column (in indivijual line)... i couldn't get with this command cat /etc/passwd | awk -F ":" '{printf $1}' Kindly help This thread was created... (3 Replies)
Discussion started by: Dheepak s
3 Replies

5. Shell Programming and Scripting

awk - mixed for and if to select particular lines in a data file

Hi all, I am new to AWK and I am trying to solve a problem that is probably easy for an expert. Suppose I have the following data file input.txt: 20 35 43 20 23 54 20 62 21 20.5 43 12 20.5 33 11 20.5 89 87 21 33 20 21 22 21 21 56 87 I want to select from all lines having the... (4 Replies)
Discussion started by: naska
4 Replies

6. Shell Programming and Scripting

select lines with certain values on certain fields with awk

I need a awk command to select from a log-file only the lines that have on the 2nd field (considering "|" separator) one of the values 10.216.22.XX or 10.216.22.YY or 10.216.22.ZZ and on the 4th field only values that contain strictly digits. I want the command to work parsing the file only once (I... (2 Replies)
Discussion started by: black_fender
2 Replies

7. Shell Programming and Scripting

Select lines in which column have value greater than some percent of total file lines

i have a file in following format 1 32 3 4 6 4 4 45 1 45 4 61 54 66 4 5 65 51 56 65 1 12 32 85 now here the total number of lines are 8(they vary each time) Now i want to select only those lines in which the values... (6 Replies)
Discussion started by: vaibhavkorde
6 Replies

8. Shell Programming and Scripting

Select some lines from a txt file and create a new file with awk

Hi there, I have a text file with several colums separated by "|;#" I need to search the file extracting all columns starting with the value of "1" or "2" saving in a separate file just the first 7 columns of each row maching the criteria, with replacement of the saparators in the nearly created... (4 Replies)
Discussion started by: capnino
4 Replies

9. UNIX for Dummies Questions & Answers

Awk - select from a list

Hi all, I am trying to select some columns from a file, based on the list of values. Would like to know how best I can achive this. If coulmn 1 has a value of 57 then print the ist column (This works) awk -F' ' '{if ( $1 == 57 ) {print $1}}' file.txt Now my requirement is that I have to... (14 Replies)
Discussion started by: simha77777
14 Replies

10. Shell Programming and Scripting

select last field from a file

hi everybody i would to select the last field of a file here as you can see i select the field number 8 y=`cat sortie2 | grep "^"| grep "starting"| awk '{ print $8}'` but line can containt more or less field in never know, i just know is the last one so i wondering to know if is... (3 Replies)
Discussion started by: kykyboss
3 Replies
Login or Register to Ask a Question