Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Find matched patterns in multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find matched patterns in multiple files
# 1  
Old 05-19-2014
Find matched patterns in multiple files
Hi,

I need help to find matched patterns in 30 files residing in a folder simultaneously. All these files only contain 1 column. For example,

File1
Code:
Gr_1
st-e34ss-11dd
bt-wwd-fewq
pt-wq02-ddpk
pw-xsw17-aqpp
Gr_2
srq-wy09-yyd9
sqq-fdfs-ffs9
Gr_3
etas-qqa-dfw
ddw-ppls-qqw
pft-78bn-htw0

File2
Code:
Gr_1
swq-ws09-fkd9
lpt-ddhg7-jjdk
klp-hdh76-ddl
Gr_2
etas-qqa-dfw
hfjs-90kj-hjs6
lks-gg77-jklo
Gr_3
pld-sgh09-qqw
dgf-23erf-we4
yur-aa23-fdfw
qoe-ewd8-kd9
Gr_4
st-e34ss-11dd
pw-xsw17-aqpp
pji-98hj-mkl

File3
Code:
Gr_1
pw-xsw17-aqpp
sad-sw09-dff
tdh-2kd-saww
add-aww2-qw9
Gr_5
swq-ws09-fkd9
hdjs-ss9-eew
lp0-89ks-2ww
Gr_10
ddf-da9-wwq
ffdw-2q1s-df
st-e34ss-11dd

The output file should be like below. It should print only the matched patterns (in blue) from those files:

outfile.txt
Code:
st-e34ss-11dd	Gr_1:File1	 Gr_4:File2	Gr_10:File3
pw-xsw17-aqpp	Gr_1:File1	 Gr_4:File2	Gr_1:File3
swq-ws09-fkd9	Gr_1:File2	 Gr_5:File3
etas-qqa-dfw	Gr_3:File1	 Gr_2:File2

I did the following codes. but this codes does not give me the results that i wanted as above. It did not give me the "Gr_x".
Code:
grep -w -f * >outfile.txt

Appreciate if someone could help me. Thanks.
Last edited by redse171; 05-19-2014 at 06:17 PM.. Reason: typo
# 2  
Old 05-20-2014
Code:
awk 'NR==FNR {a[$0]=$0;next}
	 /^Gr_[0-9]+$/ {key=$0} 
	 ($0 in a) { B[a[$0]]=B[a[$0]] "\t" key":"FILENAME} 
	 END { for (i in B){print i FS B[i]}
		}' ptr1 file[123]

This produces

Code:
pw-xsw17-aqpp   Gr_1:file1        Gr_4:file2        Gr_1:file3
st-e34ss-11dd   Gr_1:file1        Gr_4:file2        Gr_10:file3
etas-qqa-dfw    Gr_3:file1        Gr_2:file2
swq-ws09-fkd9   Gr_1:file2        Gr_5:file3



The ptr1 file contains the patterns you are looking for (the blue's) like this

Code:
$ cat ptr1
etas-qqa-dfw
pw-xsw17-aqpp
st-e34ss-11dd
swq-ws09-fkd9

Note: Look for the ^Gr_[0-9]+$ carefully and modify it accordingly if it doesn't match with your real data.
# 3  
Old 05-20-2014
Hi clx,

i tried your code but it gives me syntax error at line 2. :

Code:
awk: line 2: syntax error at or near ^
awk: line 2: runaway regular expression /{key=$0}

I understand your not there. as for the above syntax error, i got it when i ran the above sample input files.

Also, for my real input files, it does not mean that all the file name start with "File" and then followed by numbers. It could be any random names. thanks
# 4  
Old 05-20-2014
Which OS are you working on. If its SunOS, use nawk or /usr/xpg4/bin/awk


Naming of files is not an issue. You can provide wildcard also. Anything which you can list

Code:
*.txt 
*.csv
file_*.txt 
file1 tx.txt

# 5  
Old 05-20-2014
i am using linux (ubuntu). as for the file names, ok, i will use the wildcard. thanks
# 6  
Old 05-20-2014
Can you show the exact command you are trying?
# 7  
Old 05-20-2014
i am using the one that u gave me as i am testing on the sample input files that i gave.

Code:
awk 'NR==FNR {a[$0]=$0;next}      
/^Gr_[0-9]+$/ {key=$0}       
($0 in a) { B[a[$0]]=B[a[$0]] "\t" key":"FILENAME}       
END { for (i in B){print i FS B[i]}         
}' ptr1 file[123]
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash - Find files excluding file patterns and subfolder patterns

Hello. For a given folder, I want to select any files find $PATH1 -f \( -name "*" but omit any files like pattern name ! -iname "*.jpg" ! -iname "*.xsession*" ..... \) and also omit any subfolder like pattern name -type d \( -name "/etc/gconf/gconf.*" -o -name "*cache*" -o -name "*Cache*" -o... (2 Replies)
Discussion started by: jcdole
2 Replies

2. Shell Programming and Scripting

Find all matched lines from two files

Hello, everyone I have two files like this: File 1: A B C D E FFile 2:A B 1 A C 2 A K 3 B A 4 D E 3 W X 2 A B 2I want to print all lines (file2) that the first two columns are consist of elements from file1. So, what I expected is : A B 1 A B 2 (2 Replies)
Discussion started by: nengcheng
2 Replies

3. Shell Programming and Scripting

Find files not matching multiple patterns and then delete anything older than 10 days

Hi, I have multiple files in my log folder. e.g: a_m1.log b_1.log c_1.log d_1.log b_2.log c_2.log d_2.log e_m1.log a_m2.log e_m2.log I need to keep latest 10 instances of each file. I can write multiple find commands but looking if it is possible in one line. m file are monthly... (4 Replies)
Discussion started by: wahi80
4 Replies

4. Shell Programming and Scripting

Replacing matched patterns in multiple files with awk

Hello all, I have since given up trying to figure this out and used sed instead, but I am trying to understand awk and was wondering how someone might do this in awk. I am trying to match on the first field of a specific file with the first field on multiple files, and append the second field... (2 Replies)
Discussion started by: karlmalowned
2 Replies

5. Shell Programming and Scripting

Find matched patterns and print them with other patterns not the whole line

Hi, I am trying to extract some patterns from a line. The input file is space delimited and i could not use column to get value after "IN" or "OUT" patterns as there could be multiple white spaces before the next digits that i need to print in the output file . I need to print 3 patterns in a... (3 Replies)
Discussion started by: redse171
3 Replies

6. Shell Programming and Scripting

Matched multiple patterns that could be in a same line

Hi, I need help to match pattern started with "RW" in file 1 and with pattern in $1 in file 2 as follows:- File 1 BH /TOTAL=466(423); /POSITIVE=300(257); /UNKNOWN=25(25); BH /F_P=141(141); /F_N=136; /P=4; CC /TAX=!?; /MAX-R=2; CC /VER=2; RW P9610, AR_BSU , T; PAE25, AE_E57... (10 Replies)
Discussion started by: redse171
10 Replies

7. Shell Programming and Scripting

Find matched patterns in a column of 2 files with different size and merge them

Hi, i have input files like below:- input1 Name Seq_ID NewID Scores MT1 A0QZX3 1.65 277.4 IVO A0QZX3 1.65 244.5 HPO A0QZX3 1.65 240.5 RgP A0Q3PP 5.32 241.0 GX1 LPSZ3S 96.1 216.9 MEL LPSS3X 4.23 204.1 LDD LPSS3X 4.23 100.2 input2 Fac AddName NewID ... (9 Replies)
Discussion started by: redse171
9 Replies

8. Shell Programming and Scripting

Find common patterns in multiple file

Hi, I need help to find patterns that are common or matched in a specified column in multiple files. File1.txt ID1 555 ID23 8857 ID4 4454 ID05 555 File2.txt ID74 4454 ID96 555 ID322 4454 (4 Replies)
Discussion started by: redse171
4 Replies

9. Shell Programming and Scripting

How to group matched patterns in different files

Hi, I have a master file that i need to split into multiple files based on matched patterns. sample of my data as follows:- scaff_1 a e 123 130 c_scaff_100 scaff_1 a e 132 138 c_scaff_101 scaff_1 a e 140 150 ... (2 Replies)
Discussion started by: redse171
2 Replies

10. Shell Programming and Scripting

Find multiple patterns on multiple lines and concatenate output

I'm trying to parse COBOL code to combine variables into one string. I have two variable names that get literals moved into them and I'd like to use sed, awk, or similar to find these lines and combine the variables into the final component. These variable names are always VAR1 and VAR2. For... (8 Replies)
Discussion started by: wilg0005
8 Replies
Login or Register to Ask a Question