Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Copying section of file based on search criteria

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Copying section of file based on search criteria
# 1  
Old 04-14-2014
Copying section of file based on search criteria
Hi Guru's,

I am new to unix scripting. I have a huge file with user details in it(file2) and I have another file with a list of users(file1). Script has to search a user from file1 and get all the associated lines from file2.

Example:
Code:
fiel1:
cn=abc
cn=DEF
cn=xyx

File 2:
Code:
dn: cn=abc,ou=111,dc=111,dc=111,dc=com
uid: abc
userPassword:: lQ0E9sPsdfQ==

dn: cn=ghi,ou=111,dc=111,dc=111,dc=com
uid: ghi
userPassword:: PZHc9PsdfQ==

dn: cn=xyz,ou=111,dc=111,dc=111,dc=com
uid: xyz
userPassword:: PZHc9PsdfQ==

dn: cn=mno,ou=111,dc=111,dc=111,dc=com
uid: mno
userPassword:: PZHc9P23Q==

dn: cn=pqr,ou=111,dc=111,dc=111,dc=com
uid: pqr
userPassword:: PZHc9P23Q==

dn: cn=DEF,ou=111,dc=111,dc=111,dc=com
uid: DEF
userPassword:: PZHc9P23Q==



output file:
Code:
dn: cn=abc,ou=111,dc=111,dc=111,dc=com
uid: abc
userPassword:: lQ0E9sPsdfQ==

dn: cn=xyz,ou=111,dc=111,dc=111,dc=com
uid: xyz
userPassword:: PZHc9PsdfQ==

dn: cn=DEF,ou=111,dc=111,dc=111,dc=com
uid: DEF
userPassword:: PZHc9P23Q==

Thanks in advance for the help.
Sam
# 2  
Old 04-14-2014
Try this one liner, It will create a file(file_3) containing the lines from file_2 for id from file_1:
Code:
IFS='='; while read cn id;do perl -lne "print if /^dn: cn=${id}.*$/ .. /^$/" file_2; done <file_1 >file_3

# 3  
Old 04-15-2014
Code:
awk 'NR == FNR {split($0, a, FS); next} {split($0, c, " |,"); for(x in a) {if(a[x] == c[2]) print $0; continue}}' FS='\n' ORS='\n\n' RS= file1 file2

# 4  
Old 04-15-2014
Hi Srinishoo,

I am getting an empty file when I use the awk that you have provided.

Thanks,
Sam

---------- Post updated at 12:47 AM ---------- Previous update was at 12:45 AM ----------

Hi Spacebar,

Thanks for the update. The script is failing with the below error when I run it against the big file.

Code:
Bareword found where operator expected at -e line 1, near "/^dn: cn=n/a"
        (Missing operator before a?)
syntax error at -e line 1, near "/^dn: cn=n/a"
Execution of -e aborted due to compilation errors.

Thanks,
Sam
# 5  
Old 04-15-2014
You would make your life easier if you didn't put spaces in filenames (like File 2), the filename fiel1 looks like a typo, and your output show lines matching cn=xyz, but the corresponding line in fiel1 is cn=xyx; not cn=xyz.

You could try something like:
Code:
awk -F '[ ,]' '
FNR == NR {
	cn[$1]
	next
}
$1 == "dn:" {
	copy = ($2 in cn)
}
copy {	print
}' fiel1 "File 2"

which (with the sample input files you provided) produces the output:
Code:
dn: cn=abc,ou=111,dc=111,dc=111,dc=com
uid: abc
userPassword:: lQ0E9sPsdfQ==

dn: cn=DEF,ou=111,dc=111,dc=111,dc=com
uid: DEF
userPassword:: PZHc9P23Q==

If you want to use this on a Solaris/SunOS system, use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk instead of /usr/bin/awk.
These 2 Users Gave Thanks to Don Cragun For This Post:
Samingla SriniShoo
# 6  
Old 04-15-2014
Hi Don,

Thanks for the update. How do I execute this script? I tried by moving the code to a sh file but it failed.

Regards,
Sam

---------- Post updated at 03:15 AM ---------- Previous update was at 02:54 AM ----------

Hi Don,

I was able to execute the script. code works great. How can we ignore case sensitivity in the search. File1 may hold the value in small letter but file2 may have the value in capitol's. The output should be in all small letters.

Example:
Code:
File1:
cn=abc
cn=DEF
cn=xyz

Code:
File2:
dn: cn=ABC,ou=111,dc=111,dc=111,dc=com
uid: ABC
userPassword:: lQ0E9sPsdfQ==

dn: cn=ghi,ou=111,dc=111,dc=111,dc=com
uid: ghi
userPassword:: PZHc9PsdfQ==

dn: cn=xyz,ou=111,dc=111,dc=111,dc=com
uid: xyz
userPassword:: PZHc9PsdfQ==

dn: cn=mno,ou=111,dc=111,dc=111,dc=com
uid: mno
userPassword:: PZHc9P23Q==

dn: cn=pqr,ou=111,dc=111,dc=111,dc=com
uid: pqr
userPassword:: PZHc9P23Q==

dn: cn=def,ou=111,dc=111,dc=111,dc=com
uid: def
userPassword:: PZHc9P23Q==

Code:
Output:
dn: cn=abc,ou=111,dc=111,dc=111,dc=com
uid: abc
userPassword:: lQ0E9sPsdfQ==

dn: cn=xyz,ou=111,dc=111,dc=111,dc=com
uid: xyz
userPassword:: PZHc9PsdfQ==

dn: cn=def,ou=111,dc=111,dc=111,dc=com
uid: def
userPassword:: PZHc9P23Q==

Thanks for the help.
Srikanth
Last edited by Samingla; 04-15-2014 at 05:27 AM..
# 7  
Old 04-15-2014
Try this:
Code:
awk -F '[ ,]' '
FNR == NR {
	cn[tolower($1)]
	next
}
$1 == "dn:" {
	copy = (tolower($2) in cn)
}
copy {	print ($1 == "userPassword::" ? $0 : tolower($0))
}' File1 File2

This User Gave Thanks to Don Cragun For This Post:
Samingla
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match based on criteria to file

Trying to match $1 of target.txt to $5 of file.txt. If there is a match then in an output.txt file $1,$1 (row underneath),$6,$4,$7 from file.txt are printed on the same line as $1 of target.txt. The input is from excel and the output should be tab-deliminated. Thank you :). target.txt... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

Need To Delete Lines Based On Search Criteria

Hi All, I have following input file. I wish to retain those lines which match multiple search criteria. The search criteria is stored in a variable seperated from each other by comma(,). SEARCH_CRITERIA = "REJECT, DUPLICATE" Input File: ERROR,MYFILE_20130214_11387,9,37.75... (3 Replies)
Discussion started by: angshuman
3 Replies

3. Shell Programming and Scripting

Select lines from a file based on a criteria

Hi I need to select lines from a txt file, I have got a line starting with ZMIO:MSISDN= and after a few line I have another line starting with 'MOBILE STATION ISDN NUMBER' and another one starting with 'VLR-ADDRESS' I need to copy these three lines as three different columns in a separate... (3 Replies)
Discussion started by: Tlcm sam
3 Replies

4. Shell Programming and Scripting

Copying corresponding entries based on search

Hi Guru's I need some help to accomplish the below. I have two files, file 1 has ldap entries with one of its parameter cn. I have one more file with cn and corresponding row id. The out put should get the cn from file1 search for the cn in file2 and get the value of id and add the id entry in... (19 Replies)
Discussion started by: Samingla
19 Replies

5. Shell Programming and Scripting

Deleting a section based on search from other file

Hi Everyone, I need some help to accomplish the below. help is highly appriciated. I have a 45 mb file with ldap entries. Each user entry is separated by a string # entry-id: 1 and so on. Some of the entries has a string xyz: true. I want to delete the section if the user section has xyz: true... (6 Replies)
Discussion started by: Samingla
6 Replies

6. Shell Programming and Scripting

Extract data based on specific search criteria

I have a huge file (about 2 millions records) contains data separated by “,” (comma). As part of the requirement, I can't change the format. The objective is to remove some of the records with the following condition. If the 23rd field on each line start with 302 , I need to remove that from the... (4 Replies)
Discussion started by: jaygamini
4 Replies

7. Shell Programming and Scripting

Extract section of file based on word in section

I have a list of Servers in no particular order as follows: virtualMachines="IIBSBS IIBVICDMS01 IIBVICMA01"And I am generating some output from a pre-existing script that gives me the following (this is a sample output selection). 9/17/2010 8:00:05 PM: Normal backup using VDRBACKUPS... (2 Replies)
Discussion started by: jelloir
2 Replies

8. Shell Programming and Scripting

Delete new lines based on search criteria

Hi all! A bit of background: I am trying to create a script that formats SQL statements. I have gotten so far as to add new lines based on certain match criteria like commas, keywords etc. In the process, I end up adding newlines where I don't want. For example: substr(colName, 1, 10)... (3 Replies)
Discussion started by: jayarkay
3 Replies

9. Shell Programming and Scripting

Append specific lines to a previous line based on sequential search criteria

I'll try explain this as best I can. Let me know if it is not clear. I have large text files that contain data as such: 143593502 09-08-20 09:02:13 xxxxxxxxxxx xxxxxxxxxxx 09-08-20 09:02:11 N line 1 test line 2 test line 3 test 143593503 09-08-20 09:02:13... (3 Replies)
Discussion started by: jesse
3 Replies

10. UNIX for Dummies Questions & Answers

Select records based on search criteria on first column

Hi All, I need to select only those records having a non zero record in the first column of a comma delimited file. Suppose my input file is having data like: "0","01/08/2005 07:11:15",1,1,"Created",,"01/08/2005" "0","01/08/2005 07:12:40",1,1,"Created",,"01/08/2005"... (2 Replies)
Discussion started by: shashi_kiran_v
2 Replies
Login or Register to Ask a Question