Request: How to Parse dynamic SQL query to pad extra columns to match the fixed number of columns


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Request: How to Parse dynamic SQL query to pad extra columns to match the fixed number of columns
# 1  
Old 01-08-2015
Request: How to Parse dynamic SQL query to pad extra columns to match the fixed number of columns

Hello All,

I have a requirement in which i will be given a sql query as input in a file with dynamic number of columns. For example some times i will get 5 columns, some times 8 columns etc up to 20 columns.
So my requirement is to generate a output query which will have 20 columns all the times irrespective of number of input columns.

Sample input file
Code:
SELECT
A AS COL1,
COALESCE(B ,'123') AS COL2,
CAST(C AS TIMESTAMP(0)) AS COL3
FROM TABLE
WHERE X = Y AND Z > (SELECT A FROM TABLE2);

Expected output
Code:
SELECT
A AS COL1,
COALESCE(B ,'123') AS COL2,
CAST(C AS TIMESTAMP(0)) AS COL3
,NULL AS COL4
,NULL AS COL5
..
..
,NULL AS COL20
FROM TABLE
WHERE X = Y AND Z > (SELECT A FROM TABLE2);


Can someone one please help me achieve this in UNIX KSH/BASH?

Thanks in Advance
# 2  
Old 01-08-2015
In shell or some flavor of procedural SQL?
# 3  
Old 01-08-2015
Quote:
Originally Posted by DGPickett
In shell or some flavor of procedural SQL?
Thanks for your reply.

Shell is the preferred. Because we are planning to use this script to work across multiple databases. But please provide PL SQL code also if you have any.
# 4  
Old 01-08-2015
Load an array of 20 entries, "NULL Default_01" through "NULL Default_20". The read your file and replace the first entries in the array with column names from the file. Now, generate a query string from the array, and run it. What does input look like?
# 5  
Old 01-08-2015
Identifying columns individually in the query to load into array is the problem I see here. Because I need to pick some delimiter for identifying columns. I can not choose ',' or 'AS' as they are used by other sql functions like coalesce and cast.
Also there is no guarantee that the columns will be one per line. They may include multiple columns in the same line.

Input file can be like this
Code:
SELECT
A AS COL1,COALESCE(B ,'123') AS COL2,CAST(C AS TIMESTAMP(0)) AS COL3
FROM TABLE
WHERE X = Y AND Z > (SELECT A FROM TABLE2);

or

Code:
SELECT
A AS COL1,
COALESCE(B ,'123') AS COL2,
CAST(C AS TIMESTAMP(0)) AS COL3
FROM TABLE
WHERE X = Y AND Z > (SELECT A FROM TABLE2);

# 6  
Old 01-08-2015
So, just do a:
Code:
ct=0
for col in `cat file`
do
 cols[$ct]=$col
 (( ct++ ))
done

# 7  
Old 01-08-2015
Quote:
Originally Posted by DGPickett
So, just do a:
Code:
ct=0
for col in `cat file`
do
 cols[$ct]=$col
 (( ct++ ))
done

If i use your code each row will go into one array element. But how can i know how many columns are there in the sql query actually?

For example if we consider below row, how can i select col1, col2 and col3 in to 3 array elements instead of 1?
Code:
A AS COL1,COALESCE(B ,'123') AS COL2,CAST(C AS TIMESTAMP(0)) AS COL3[

Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Adding columns from 2 files with variable number of columns

I have two files, file1 and file2 who have identical number of rows and columns. However, the script is supposed to be used for for different files and I cannot know the format in advance. Also, the number of columns changes within the file, some rows have more and some less columns (they are... (13 Replies)
Discussion started by: maya3
13 Replies

2. Programming

Query SQL get two values differents from the same columns

Hi, I have 2 different values in the same column and two different values in other column Query 1 ins name value 1 Test 12345 1 TestV1 12/10/2014 8 Test 85435 8 TestV1 11/11/2005 9 Test 42232 9 TestV1 19/10/2000 6 Test 54321... (6 Replies)
Discussion started by: faka
6 Replies

3. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

4. UNIX for Dummies Questions & Answers

Getting values of 2 columns from sql query in UNIX variables

Hi, I have connected to oracle database with sqlplus -s / <<EOF select ename, age from emp where empid=1234; EOF I want to save the values of ename and age in unix shell variables. Any pointers would be welcome.. Thanks in advance!!1 Cheers :):):):) (1 Reply)
Discussion started by: gonchusirsa
1 Replies

5. Shell Programming and Scripting

Match on columns and replace other columns

Hi Friends, I have the following input file cat input chr1 100 200 0.1 0.2 na 1 na nd chr1 105 200 0.1 0.2 1 1 na 98 chr1 110 290 nf 1 na nd na 1 chr2 130 150 12 3 na 1 na 1 chr3 450 600 nf nf na 10 na nd chr4 300 330 1 1 10 11 23 34 My requirement is 1. If $6 is na make $7 nd and... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

6. Shell Programming and Scripting

How to parse fixed-width columns which may include empty fields?

I am trying to selectively display several columns from a db2 query, which gives me a fixed-width output (partial output listed here): --------- -------------------------- ------------ ------ 000 0000000000198012 702 29 000 0000000000198013 ... (9 Replies)
Discussion started by: ahsh79
9 Replies

7. UNIX for Dummies Questions & Answers

Add extra columns help

Hi Gurus, This below script adds a column extra to my flat file..But how can i add another column, Say if i just put mention an other column beside the first column..it does get generated but as one column only while IFS="" read r; do printf "dummy\t%s\n" "$r" done < xxx.txt > zzz.txt ... (9 Replies)
Discussion started by: saggiboy10
9 Replies

8. Shell Programming and Scripting

splitting a huge line of file into multiple lines with fixed number of columns

Hi, I have a huge file with a single line. But I want to break that line into lines of with each line having five columns. My file is like this: code: "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you." I want it like this: code:... (1 Reply)
Discussion started by: rajsharma
1 Replies

9. Shell Programming and Scripting

Remove extra characters and sum the columns

I have data extracted like this: A=%123% B=%456% C=%789% A=%111% B=%222% C=%333% A=%777% B=%888% C=%999% Can someone please help me with a script to remove all the % signs and get the totals for A, B and C. So output will be: A=1368 B=666 C=2664 Thank you! (2 Replies)
Discussion started by: tatchel
2 Replies

10. Shell Programming and Scripting

Dynamic SQL query based on shell script parameters

Hi, I need a script that will run a dynamic Oracle SQL. Dynamic meaning the SQL statement depends on the parameter. For instance, something like this: #!/bin/ksh -x # Set environment . /home/mine/set_vars sqlplus $LOGINID <<! >> /home/mine/log.txt select count(1) from $1 where... (2 Replies)
Discussion started by: laiko
2 Replies
Login or Register to Ask a Question