find string and get the rest of the line in a pipe delimited file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers find string and get the rest of the line in a pipe delimited file
# 1  
Old 02-15-2012
find string and get the rest of the line in a pipe delimited file

Hi friends,

I have a file where I should search for a string and get the rest of the line but without the delimiter using awk.
for example I have the series of string in a file:
input_string.txt
Code:
bbb
ccc
aaa

and the mapping file looks like this.
mapping.txt
Code:
aaa|12
bbb|23
ccc|43

i need to get the mapped number and use it in a variable. the code looks like this:
Code:
while read line;
do
   mapped= ??AWK $line mapping.txt
   ....
   ....
done < input_string.txt

Is there a way to do this using awk or sed? Please help Smilie
# 2  
Old 02-15-2012
Code:
#! /bin/bash
while read x
do
    mapped=`grep $x output`
    echo ${mapped#*|}
done < input_string.txt


Last edited by balajesuri; 02-15-2012 at 12:35 PM..
This User Gave Thanks to balajesuri For This Post:
# 3  
Old 02-15-2012
I think this is closer to what you want, and far more efficient than running grep 9,000 times to process 9,000 different matches. It loads the entire mapping file in at once into an associative array, then reads input_string.txt line by line, printing the matching number if it finds it.

Code:
awk -F'|' 'BEGIN { while(getline<"mapping.txt") A[$1]=$2 }; A[$1] { print A[$1] }' input_string.txt

This User Gave Thanks to Corona688 For This Post:
# 4  
Old 02-15-2012
Quote:
Originally Posted by Corona688
I think this is closer to what you want, and far more efficient than running grep 9,000 times to process 9,000 different matches. It loads the entire mapping file in at once into an associative array, then reads input_string.txt line by line, printing the matching number if it finds it.

Code:
awk -F'|' 'BEGIN { while(getline<"mapping.txt") A[$1]=$2 }; A[$1] { print A[$1] }' input_string.txt

I may have to do a printf or some other operations once i got the mapped value from the file. How will the code look like if I use this awk. But you are correct, performance will be an issue if I have to search every time.
# 5  
Old 02-15-2012
awk separates the data file by itself, which is why it used it -- by telling it the separator is | with -F'|', it separates them natively...

It reads that file into an array, then uses that array to look it up later.

So when you run that awk program, it does something like this:

Code:
# Code in BEGIN block gets run before all lines are read
# Read from the map file, set array based on contents
A["aaa"]=12
A["bbb"]=23
A["ccc"]=43

# For every line in the array, print the array's contents
print A["bbb"]
print A["ccc"]
print A["aaaa"]

If that's not what your data actually looks like, post a better example.
# 6  
Old 02-15-2012
It does have the same type of data.
So the execution of the awk command will be before the while loop is declared right? so the code will look like this:
Code:
awk -F'|' 'BEGIN { while(getline<"mapping.txt") A[$1]=$2 }; A[$1] { print A[$1] }' input_string.txt

while read line; 
do    
   mapped=A["$line"]
   ....    .... 
done < input_string.txt

the search should not necessary to be in a specific order so it should be in an interactive sort of thing. The input_string.txt does not have to be in the same arrangement or order with mappint.txt.
# 7  
Old 02-15-2012
What while loop? That had nothing to do with my code. Setting variables inside awk wouldn't set variables outside it anyway. It's its own language, not part of the shell.

The awk oneliner does the entire thing.

Code:
$ cat mapping.txt
aaa|12
bbb|23
ccc|43

$ cat input_string.txt
bbb
ccc
aaa

$ awk -F'|' 'BEGIN { while(getline<"mapping.txt") A[$1]=$2 }; A[$1] { print A[$1] }' input_string.txt
23
43
12

$

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to remove new line characters from data rows in a Pipe delimited file?

I have a file as below Emp1|FirstName|MiddleName|LastName|Address|Pincode|PhoneNumber 1234|FirstName1|MiddleName2|LastName3| Add1 || ADD2|123|000000000 2345|FirstName2|MiddleName3|LastName4| Add1 || ADD2| 234|000000000 OUTPUT : ... (1 Reply)
Discussion started by: styris
1 Replies

2. UNIX for Dummies Questions & Answers

Need to convert a pipe delimited text file to tab delimited

Hi, I have a rquirement in unix as below . I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column. ex: Input Text file: 1|A|apple 2|B|bottle excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies

3. Shell Programming and Scripting

Replace pipe delimited column string to null

Hi All, I have a large dat file where each lines are pipe delimited values. I need to parse the file depending on the request. For example: sometimes I have told to remove all the values in the 7th column (this case remove values '3333' only from the first line and '3543' from the second line)... (4 Replies)
Discussion started by: express14
4 Replies

4. Shell Programming and Scripting

How to ignore Pipe in Pipe delimited file?

Hi guys, I need to know how i can ignore Pipe '|' if Pipe is coming as a column in Pipe delimited file for eg: file 1: xx|yy|"xyz|zzz"|zzz|12... using below awk command awk 'BEGIN {FS=OFS="|" } print $3 i would get xyz But i want as : xyz|zzz to consider as whole column... (13 Replies)
Discussion started by: rohit_shinez
13 Replies

5. Shell Programming and Scripting

Find for line with not null values at nth place in pipe delimited file

Hi, I am trying to find the lines in a pipe delimited file where 11th column has not null values. Any help is appreciated. Need help asap please. thanks in advance. (3 Replies)
Discussion started by: manikms
3 Replies

6. Shell Programming and Scripting

Find the text in the file and delete the rest of the line.

Hi, I have one requiremnet like this. I need to find some particular string (eg.IncludeDateTime = ) in a file. And wherever it finds the string the unix script has to delete the remaining text coming after the string (ie., 'IncludeDateTime = ' ) in the same line. I tried to write this script in... (5 Replies)
Discussion started by: jannusuresh
5 Replies

7. Shell Programming and Scripting

Help with converting Pipe delimited file to Tab Delimited

I have a file which was pipe delimited, I need to make it tab delimited. I tried with sed but no use cat file | sed 's/|//t/g' The above command substituted "/t" not tab in the place of pipe. Sample file: abc|123|2012-01-30|2012-04-28|xyz have to convert to: abc 123... (6 Replies)
Discussion started by: karumudi7
6 Replies

8. Shell Programming and Scripting

How to convert a space delimited file into a pipe delimited file using shellscript?

Hi All, I have space delimited file similar to the one as shown below.. I need to convert it as a pipe delimited, the values inside the pipe delimited file should be as highlighted... AA ATIU2345098809 009697 005374 BB ATIU2345097809 005445 006518 CC ATIU9685098809 003215 003571 DD... (7 Replies)
Discussion started by: nithins007
7 Replies

9. Shell Programming and Scripting

how to build a pipe delimited string

#! /bin/csh set delimiter = | foreach i (*) set str_deli="$i$delimiter" question: how to retain the value of str_deli so i can build a pipe delimited string? end (1 Reply)
Discussion started by: jdsignature88
1 Replies

10. Shell Programming and Scripting

convert a pipe delimited file to a':" delimited file

i have a file whose data is like this:: osr_pe_assign|-120|wg000d@att.com|4| osr_evt|-21|wg000d@att.com|4| pe_avail|-21|wg000d@att.com|4| osr_svt|-11|wg000d@att.com|4| pe_mop|-13|wg000d@att.com|4| instar_ready|-35|wg000d@att.com|4| nsdnet_ready|-90|wg000d@att.com|4|... (6 Replies)
Discussion started by: priyanka3006
6 Replies
Login or Register to Ask a Question