Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Generate a DML dynamically based off of header record

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Generate a DML dynamically based off of header record
# 15  
Old 06-13-2013
Here is an awk way:
Code:
awk '
        !/^record/ && !/^end/ {
                sub ( /;/, X, $NF )
                if ( $NF ~ /A.*/ )
                        s = s ? s OFS $NF : $NF
                if ( $NF ~ /Q.*/ )
                        Q[++j] = $NF
        }
        END {
                for ( k = 1; k <= j; k++ )
                        print s OFS Q[k] "(Question number),Response(value in the qn number col)"
        }
' OFS=, file

# 16  
Old 06-13-2013
Great start....
Yoda - Actually let me be more clear on this...since its tricky(atleast for me)!

Required Target DML:
Code:
record
string(',') A1;
string(',') A2;
date("YYYY-MM-DD")(',') A3;
string(',') A4;
date("YYYY-MM-DD")(',') A5;
string(',') Question;
string('\n') Response;
end;

--------------------------------------
Input Data:
Code:
ID,Key,keyCreateDate,EmailAddress,DepDate,EndDate,Q1,Q2,Q3,Q4,Q5,Q6,Q7,Q8,Q9,Q10,Q11,Q12,Q12_1,Q12_2,Q12_3,Q12_4,Q12_5,Q12_6,Q12_7
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,10,10,10,10,10,2,,10,10,10,11,10,10,10,10,10,10,10,1

Output Data:
Code:
ID,Key,keyCreateDate,EmailAddress,DepDate,EndDate,Question,Response
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q1,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q2,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q3,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q4,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q5,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q6,10
---
---
---
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q12_5,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q12_6,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q12_7,1

So....1 record would split out into 19 records since there are 19 questions!

And as the very nature the squence of questions is dynamic and the logic should be able to handle that and generate the output accordingly.

Thanks again!
~Anduzzi
# 17  
Old 06-14-2013
Based on some assumptions:
Code:
awk -F, -v C=6 '
        NR == 1 {
                for ( i = 1; i <= NF; i++ )
                {
                        if ( i <= C )
                                printf "%s", ( i == C ? $i OFS "Question,Response" RS : $i OFS )
                        else
                                Q[++n] = $i
                }
        }
        NR > 1 {
                for ( i = 1; i <= NF; i++ )
                {
                        if ( i <= C )
                                s = s ? s OFS $i : $i
                        else
                                A[++m] = $i
                }
        }
        END {
                for ( i = 1; i <= n; i++ )
                        print s, Q[i], A[i]
        }
' OFS=, file

This User Gave Thanks to Yoda For This Post:
anduzzi
# 18  
Old 06-14-2013
getting there....
Yoda - Don't bother about converting the DML in this script(or adding the header in output), just normalize the data....i.e break into number of times the total questions in the input(this case its 19) and assign the Question number and corresponding response in each record...

--It would be also helpful if you can post your results for this example after running the script so i can compare and see where I need to tweak the script....if required!

Code:
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q1,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q2,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q3,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q4,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q5,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q6,10
---
---
---
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q12_5,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q12_6,10
R_5dOLRNb9wp2V4KF,A8XZNR,2013-05-07,123@YAHOO.COM,2013-05-26,2013-05-29 08:03:51,Q12_7,1
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to print multiple required columns dynamically in a file using the header name?

Hi All, i am trying to print required multiple columns dynamically from a fie. But i am able to print only one column at a time. i am new to shell script, please help me on this issue. i am using below script awk -v COLT=$1 ' NR==1 { for (i=1; i<=NF; i++) { ... (2 Replies)
Discussion started by: balu1234
2 Replies

2. Shell Programming and Scripting

Parameterizing to dynamically generate the extract file from Oracle table using Shell Script

I have below 2 requirements for parameterize the generate the extract file from Oracle table using Shell Script. Could you please help me by modifying the script and show me how to execute it. First Requirement: I have a requirement where I need to parameterize to generate one... (0 Replies)
Discussion started by: hareshvikram
0 Replies

3. UNIX for Beginners Questions & Answers

Help in printing records where there is a 'header' in the first record ???

Hi, I have a backup report that unfortunately has some kind of hanging indent thing where the first line contains one column more than the others I managed to get the output that I wanted using awk, but just wanting to know if there is short way of doing it using the same awk Below is what... (2 Replies)
Discussion started by: newbie_01
2 Replies

4. Shell Programming and Scripting

Generate class path dynamically based on source path

Hi experts, I have multiple file names ending with .jsp located in $SOME_DIR, $SOME_DIR/f1/,$SOME_DIR/f2/test,$SOME_DIR/f3/fa and there are equivalent class files in $SOME_DIR/WEB-INF/classes/_pages,$SOME_DIR/WEB-INF/classes/_pages/_f1,... (0 Replies)
Discussion started by: oraclermanpt
0 Replies

5. UNIX for Dummies Questions & Answers

Dynamically merging 2 files on header values

Hi All, I have 2 files which i need to merge together based on the column names provided in the file. The first line in both files are header records. The first file has fixed columns but second file can have subset of the columns from file 1 File 1: ... (6 Replies)
Discussion started by: kushagra
6 Replies

6. Shell Programming and Scripting

Approach on Header record

All, I currently have a requirement to fetch a Date value from a table. And then insert a Header record into a file along with that date value. ex: echo "HDR"" "`date +%Y%j` `date +%Y%m%d` In the above example I used julian date and standard date using Current Date. But the requirement... (0 Replies)
Discussion started by: cmaroju
0 Replies

7. Shell Programming and Scripting

Insertion of Header record

A header record is to be inserted in the begining of a flat file without using extra file or new file. It should be inserted into same file. Advace thanks for all help... (7 Replies)
Discussion started by: shreekrishnagd
7 Replies

8. Shell Programming and Scripting

Skip parsing the header record - Awk

Guys.... Got a scenario in which I need to skip parsing the header record while I do an awk. Does awk has the flexibility to accomplish this?. If so, how do we do this?. Thanks !!! -Anduzzi :) (2 Replies)
Discussion started by: anduzzi
2 Replies

9. Shell Programming and Scripting

awk script to update header record

I am using HP UX and think this may be done with awk but bot sure. I have a file with a several header records and undeneath many detail records I need to put in the header record the number of detail records above this header record and number of detail records below this header record Header... (5 Replies)
Discussion started by: klut
5 Replies

10. UNIX for Dummies Questions & Answers

How to extract duplicate records with associated header record

All, I have a task to search through several hundred files and extract duplicate detail records and keep them grouped with their header record. If no duplicate detail record exists, don't pull the header. For example, an input file could look like this: input.txt HA D1 D2 D2 D3 D4 D4... (17 Replies)
Discussion started by: run_eim
17 Replies
Login or Register to Ask a Question