Using Awk to efficiently substitute values using 3 vectors


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using Awk to efficiently substitute values using 3 vectors
# 1  
Old 02-11-2010
Data Using Awk to efficiently substitute values using 3 vectors

I'm trying to efficiently combine the fields of two vectors (vectors b and c) into a new vector (vector d) as defined by instructions from a 3rd vector (vector a). So vector a has either a 1 or 2 in each field specifying which vector (b or c respectively) should go into that field. Vector a is space separated, while vectors b, c, and d are semicolon separated. I made a bash/awk script which does this, but it takes a while:
Code:
z=0
for y in `echo $a_vect`
do
  z=$(( $z + 1 ))            
  if [ "$y" -eq 1 ]; then 
    temp_var=`echo ${b_vect} | awk -F';' '{print $"'"${z}"'"}'`
    d_vect=`echo -ne "${d_vect};${temp_var}"`
  else
    temp_var=`echo ${c_vect} | awk -F';' '{print $"'"${z}"'"}'`
    d_vect=`echo -ne "${d_vect};${temp_var}"`
  fi
done


Last edited by Scott; 02-11-2010 at 04:29 PM.. Reason: Please use code tags
# 2  
Old 02-11-2010
Hammer & Screwdriver Sample of input/output

Can you provide a sample of the input and desired output? Seeing 5-10 lines of example makes coding much easier.
# 3  
Old 02-11-2010
Quote:
Originally Posted by joeyg
Can you provide a sample of the input and desired output? Seeing 5-10 lines of example makes coding much easier.
Code:
a_vect='1 2 1 2 1'
b_vect='0;1;2;3;4'
c_vect='5;6;7;8;9'

desired output:
Code:
d_vect='0;6;2;8;4'

Hope that helps Smilie, let me know if you need any further clarification

Last edited by Scott; 02-11-2010 at 04:42 PM.. Reason: Code tags
# 4  
Old 02-11-2010
Ok, I got something here for you. Its not awk, but it works and I belive its faster than yours because it does not loop three times. I am sure somebody will get a better solution, but here are my 2 cents.

Code:
#!/bin/bash

## JUST a lil FANCY STUFF for debug
function debug()  {
  if [[ $DEBUG ]]; then
    echo "$@"
  fi
}

if [[ "$1" == "d" ]]; then
  DEBUG=true
fi

## REAL USEFUL CODE STARTS HERE
inputfile='/tmp/1.ip' ## SET IT TO WHEREVER YOUR INPUT FILE IS
tmpfile='/tmp/2.ip'

## FORMAT THE INPUT A LITTLE SO ITS EASY TO HANDLE LATER.
sed "s/._vect='\(.*\)'/\1/g" ${inputfile}| sed "s/\([^ ;]*\)[ ;\$]/\1 /g" > $tmpfile

refarr=(`head -1 $tmpfile`)       ## THE FIRST  ROW OF INPUT
arr1=(`head -2 $tmpfile|tail -1`) ## THE SECOND ROW OF INPUT
arr2=(`tail -1 $tmpfile`)         ## THE THIRD  ROW OF INPUT
length=`echo ${arr2[@]}|awk '{print NF}'` ## NUMBER OF FIELDS IN EACH ROW

debug "REFARR = ${refarr[@]}"
debug "arr1 =   ${arr1[@]}"
debug "arr2 =   ${arr2[@]}"

typeset -a arr        ## FINAL RESULT ARRAY
i=0

## LOOP ONLY ONCE, ASSUME ALL ROWS HAVE SAME NUMBER OF FIELDS.
while [[ $i -lt length ]]; do
  if [[ "${refarr[i]}" == "1" ]]; then
    arr[i]=`echo "${arr1[i]}"`
  else
    arr[i]=`echo "${arr2[i]}"`
  fi
  debug "$i => Ref[${i}]: ${refarr[$i]}, arr1[$i]: ${arr1[$i]}, arr2[$i]: ${arr2[$i]}, arr[$i]: ${arr[$i]}"
  (( i = $i + 1 ))
done
echo "${arr[@]}"|sed "s/ /;/g"


I timed this script on my ubuntu (karmic koala) and for the given input it performed as below
Code:
time ./ux.sh 
0;6;2;8;4
./ux.sh  0.01s user 0.02s system 83% cpu 0.033 total

You might have noted that the input file is a variable in the script, you may parametrize it if you want, or just hard code it in there.
And as you can see, if you run this script with a argument "d", it should print what it is trying to do. Fore complete debug just set -x.

Last edited by linuxpenguin; 02-11-2010 at 06:51 PM.. Reason: Adding more info.
# 5  
Old 02-11-2010
MUCH faster, THANK YOU!!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Substitute from awk results

Hi, The following awk command : asmcmd lsdg | awk '{print $13;}' | grep -i ${SID} return the following output . An Empty line + two lines contain "/" at the end of the line INDEVDATA/ INDEVFRA/ I need to remove the "/" as well as the empty line. Please advise Thanks (3 Replies)
Discussion started by: Yoav
3 Replies

2. UNIX for Dummies Questions & Answers

Sort and vectors on awk

Well, i have a script and it makes a txt like this : Caps 12 cans 9 cols 10 my print line is something like this for(i in a) print i, a; i have to order the txt from higher to low like: (6 Replies)
Discussion started by: matius_88
6 Replies

3. Shell Programming and Scripting

awk to substitute ip without zero left padding

Hello All, I have this script to awk IP to new file. #awk '/myip|yourip/ {sub(/...\....\....\..../, newip)}1' newip=$IP existing.txt > new.txt When existing.txt has myip=192.168.123.123 and $IP has 192.168.12.12, the awk script is not working. But while I add zero left padding to $IP i.e,... (3 Replies)
Discussion started by: Shaan_Shaan
3 Replies

4. Shell Programming and Scripting

substitute with awk

When the line contains abc, it will goes to the next line and substitue the MM to NN bc 23 33 abc 23 33 ddd MM xx dff MM 33 cat xxx |awk '{if ($0~/abc/){getline;sub(/MM/,"NN")}{print}}', It doesn't show "abc 23 33 bc 23 33 ddd NN xx dff MM 33 bc 23 33 abc 23 33 ddd NN xx... (1 Reply)
Discussion started by: yanglei_fage
1 Replies

5. Shell Programming and Scripting

AWK variable substitute issue

dear, I have below file called folderlist.txt # ParentFolder environment_flag SubFolders triss 1 checksum bookstructure 1 fx 1 checksum_GMDB I have a script which which will create the folders under... (3 Replies)
Discussion started by: manas_ranjan
3 Replies

6. Shell Programming and Scripting

substitute variable for values in perl

hi all, how do i assign values passed in from command line to and sql statement in perl ?? e.g i want to assign :name1 and :Name2 to be whatever is passed into the perl script command line my $sqlStr = "select * from test_table where column1 = upper(nvl(:name1, name1 )) and column2... (1 Reply)
Discussion started by: cesarNZ
1 Replies

7. Shell Programming and Scripting

how to substitute filepaths with sed or awk?

I am having the following problem. I am having a lot of files (test_1_01.hea, test_1_02.hea, etc) with the content: project_directory /net/1/d_1/5/ tmp_directory /net/1/d_1/5/ material_directory /net/1/d_1/5/ And I have to substitute the filepaths with new counted ones where the... (3 Replies)
Discussion started by: ergy1983
3 Replies

8. Shell Programming and Scripting

AWK Multiple substitute

I want to reformat the following: ID1 ID001 0 0 2 1 GG TC GG CT GG AA AA AG ID2 ID002 0 0 2 2 GG 00 AG CC GG GG TC CC I want to replace only:... (1 Reply)
Discussion started by: genehunter
1 Replies

9. Shell Programming and Scripting

Using awk to substitute columns from one file into another

Hi, I am new to this forum and my script skills are very weak. I have this file (file 1) that contains 3 columns (tab delimited): kyle start stop john start stop joe start stop And I want to replace name (column 1) from another file. This other file has two columns, one is the... (4 Replies)
Discussion started by: kylle345
4 Replies

10. Shell Programming and Scripting

Substitute variable values

Hi, I am trying to redefine the value of a variable based on another variable value. And I want to read in my variables from a enviroment file in the end -- at least I think so. But 1st here's what I want I need to get working: var_1="11 10 9 8 7 6 5 4 3 2 1" var_2=3 var_3=4 So I want... (12 Replies)
Discussion started by: John Rihn
12 Replies
Login or Register to Ask a Question