Normalization Using Shell Scripting.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Normalization Using Shell Scripting.
# 1  
Old 03-02-2010
Normalization Using Shell Scripting.

Hi All, I am having a file having below three lines or maybe more than 3 lines. The first line will be always constant.

Code:
### Line 1 ####
Transformation||Transformation Mapplet Name||Transformation Group||Partition Index||Transformation Row ID||Error Sequence||Error Timestamp||Error UTC Time||Error Code||Error Message||Error Type||Transformation Data||Source Mapplet Name||Source Name||Source Row ID||Source Row Type||Source Data 
### Line 2 ####
exp_Concat||N/A||Input||1||4||1||03/01/2010 08:42:33||1267450953||11132||Transformation [exp_Concat] had an error evaluating output column [ID_OUT]. Error message is [<<Expression Fatal Error>> [ABORT]: ID CANT BE NULL VALUE\n... f:ABORT(u:'ID CANT BE NULL VALUE')].\n||3||N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000||N/A||SQ_EMP||4||0||N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000
### Line 3 ####
lkp_Concat||N/A||Input||1||3||1||03/01/2010 08:42:33||1267450953||11132||NULL||3||N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000||N/A||SQ_EMP||4||0||N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000

Out put expected :

Code:
Transformation = exp_Concat
Transformation Mapplet Name = N/A
.
.
.
Source Data = N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000

Transformation = lkp_Concat
Transformation Mapplet Name = N/A
.
.
.
Source Data = N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000

is it possible using shell script. Please if so let me know.

Last edited by Franklin52; 03-02-2010 at 03:25 AM.. Reason: Please use code tags!
# 2  
Old 03-02-2010
Post your script what you have tried so far.Then,we will help you.
# 3  
Old 03-02-2010
Code:
sed 's/||/|/g' file | awk 'BEGIN{FS="|"}NR==1{for(i=1;i<=NF;i++)head[i]=$i}NR>1{for(i=1;i<=NF;i++)print head[i],"=",$i}'


cheers,
Devaraj Takhellambam
# 4  
Old 03-02-2010
You try the following script which will do your requirement.

Code:
j=0
for line in `sed 's/ //g;s/||/|/g' Transinput`
do
if [[ `echo $line|grep -v "^#.*"` ]];then
if [[ $j -eq 0 ]]
then
for((k=1;k<=17;k++))
do
arr[$k]=`echo $line | cut -d '|' -f $k `
done
let j+=1
else
for((k=1;k<=17;k++))
do
echo "${arr[$k]} = `echo $line | cut -d '|' -f $k `"
done
echo -e "\n"
let j+=1
fi
fi
done

# 5  
Old 03-03-2010
Hi vivekraj and devtakh, thanks a lot for your reply.

Last edited by satyaranjon; 03-03-2010 at 02:35 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Shell script to read lines in a text file and filter user data Shell Programming and Scripting

sxsaaas (3 Replies)
Discussion started by: VikrantD
3 Replies

2. Shell Programming and Scripting

Data Normalization

Hi, there Need help on rearranging the data. I have data in the following format. LAC = 040 DN = 24001001 EQN = 920- 2- 0- 1 CAT = MS OPTRCL (7 Replies)
Discussion started by: meetsriharsha
7 Replies

3. Shell Programming and Scripting

Normalization using awk

I made my explanation precise in the CODE below. I can do this manually. But is there a way to automate this? If I give 4 or 10 or any number of inputs. It should calculate the CODE and print the different outputs with normalization value ? some thing like script.sh input1 input2 input3 input4... (12 Replies)
Discussion started by: quincyjones
12 Replies

4. Shell Programming and Scripting

Normalization using awk

Hi I have a file with chr22_190_200 XXY 0 0 chr22_201_210 XXY 0 30 chr22_211_220 XXY 3 0 chr22_221_230 XXY 0 0 chr22_231_240 XXY 5 0 chr22_241_250 ABC 0 0 chr22_251_260 ABC 22 11 ... (12 Replies)
Discussion started by: Diya123
12 Replies

5. Web Development

Perl scripting or shell scripting?

i am going to study any one of the scripting languages mentioned above(shell 0r perl scripting) . Which is having more scope for a fresher? (1 Reply)
Discussion started by: Anna Hussie
1 Replies

6. What is on Your Mind?

Shell Scripting vs Perl scripting

Gents, I have been working in a Solaris/Unix environment for about 9 months. I took some linux classses online before getting the job. But, I am not very good at scripting. I want to learn how to script. Do you think that I should start with Shell scripting or Perl? I wanted to continue with... (2 Replies)
Discussion started by: Pouchie1
2 Replies

7. What is on Your Mind?

Shell scripting vs Perl scripting

Hi all, I would like to start developping some good scripting skills. Do you think it would be best to start with shell scripting or Perl? I already got a fundation, really basics, in perl. but I am wondering what would be best to be good at first. Can you please help me determine which one to... (14 Replies)
Discussion started by: Pouchie1
14 Replies

8. Shell Programming and Scripting

Call Shell scripting from Perl Scripting.

Hi How to call a shell scripting through a Perl scripting? Actually I need some value from Shell scripting and passes in the Perl scripting. So how can i do this? (2 Replies)
Discussion started by: anupdas
2 Replies

9. Shell Programming and Scripting

difference between AIX shell scripting and Unix shell scripting.

please give the difference between AIX shell scripting and Unix shell scripting. (2 Replies)
Discussion started by: haroonec
2 Replies
Login or Register to Ask a Question