Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Run sql query in shell script and output data save as delimited text

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Run sql query in shell script and output data save as delimited text
# 1  
Old 03-17-2017
Blade Run sql query in shell script and output data save as delimited text
I want to run sql query in shell script and output data save as delimited text (delimited text would be comma)

Code:
Code:
SPOOL_FILE=/pgedw/dan.txt
SQL=/pgedw/dan.sql
sqlplus -s username/password@myhost:port/servicename <<EOF
set head on
set COLSEP ,
set linesize 32767
SET TRIMSPOOL ON
SET TRIMOUT ON
SPOOL $SPOOL_FILE
@ $SQL
SPOOL OFF
EOF
echo "Note" |mail -s "Test" -a $SPOOL_FILE  org@mail.com
exit 0

For above code it returning result like below
Code:
Column 1,Column 2,Column 3,column 4,Column 5,Column 6,Column 7,Column 8  
--------,--------,--------,--------,--------,--------,--------,--------
03,           364,     364,     364,    1703  1925968,  031417, 0155732
03,           364,     364,     364,    1703  1925968,  031417, 0155754
03,           364,     364,     364,    1703  1925968,  031417, 0155708
03,           364,     364,     364,    1703  1925968,  031417, 0155707 
03,           364,     364,     364,    1703  1925968,  031417, 0155708  
Column 1,Column 2,Column 3,column 4,Column 5,Column 6,Column 7,Column 8 
--------,--------,--------,--------,--------,--------,--------,--------
03,           364,     364,     364,    1703  1925970,  031417, 0155732
03,           364,     364,     364,    1703  1925988,  031417, 0155754
03,           364,     364,     364,    1703  1925998,  031417, 0155708
03,           364,     364,     364,    1703  1925968,  031417, 0155707 
03,           364,     364,     364,    1703  1925968,  031417, 0155710

but actually output i want like this
Code:
Column 1,Column 2,Column 3,column 4,Column 5,Column 6,Column 7,Column 8  
03,364,364,364,1703,1925968,031417,0155732
03,364,364,364,1703,1925968,031417,0155754
03,364,364,364,1703,1925968,031417,0155708
03,364,364,364,1703,1925968,031417,0155707 
03,364,364,364,1703,1925968,031417,0155708  
03,364,364,364,1703,1925970,031417,0155732
03,364,364,364,17031,925988,031417,0155754
03,364,364,364,1703,1925998,031417,0155708
03,364,364,364,1703,1925968,031417,0155707 
03,364,364,364,1703,1925968,031417,0155710

Help help me

Thanks in advance



Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!
Last edited by RudiC; 03-17-2017 at 07:56 AM.. Reason: Added CODE tags.
# 2  
Old 03-17-2017
Hello!

In case you forgot to read the forum rules, here is quick copy.

Quote:
RULES OF THE UNIX AND LINUX FORUMS


(1) No flames, shouting (all caps), sarcasm, bullying, profanity or arrogant posts.

(2) No negative comments about others or impolite remarks. Be patient.

(3) Refrain from idle chatter that does not contribute to the knowledge base. This does not apply to the forums in The Unix Lounge which are for off-topic discussions.

(4) Do not 'bump up' questions if they are not answered promptly. No duplicate or cross-posting and do not report a post or send a private message where your goal is to get an answer more quickly.

(5) Search the forums database with your keywords before asking.

(6) Do not post classroom or homework problems.

(7) No job postings from headhunters or recruiters except in The Unix Forums Job Board. See How to Post to The UNIX Forums Job Board for information on using the Job Board.

(8) No BSD vs. Linux vs. Windows or similar threads.

(9) Edit your posts if you see spelling or grammar errors (don't write in cyberchat or cyberpunk style). English only.

(10) Don't post your email address and ask for an email reply. Don't send a private message with a technical question. The forums are for the benefit of all, so all Q&A should take place in the forums.

(11) Post questions with descriptive subjects. For example, do not post questions with subjects like "Help Me!", "Urgent!!" or "Doubt". Post subjects like "Execution Problems with Cron" or "Help with Backup Shell Script".

(12) These are not hacker boards so hacker related posts will be promptly deleted or moderated.

(13) The forum administrators reserve the right to prune, move or edit posts that do not adhere to the rules or are technically inaccurate.

(14) The forum administrators reserve the right to remove users or change their posting status to read only without notice if any rules are not followed.

(15) No smoking in the forums.
Cheers.

The UNIX and Linux Forums
This User Gave Thanks to Neo For This Post:
rbatte1
# 3  
Old 03-17-2017
Firstly I would recommend moving the credentials into the here document, else they will be publicly visible to anyone running a simple ps whilst your code is executing.

If the columns are of a fixed width, you may simply get away with adding in a few more statements to format the output column like this:-
Code:
COLUMN column-name FORMAT format-rule ;

You will need to know your column names and set them all individually.

Have a look at this page for more information. You may also look at setting number format or number width.

Ordinarily, the output will be presented in a nice tabular form with the default width being the length of the column name or the widest column content, whichever is greater.


If none of these work for you, perhaps you can capture the data into a file and use tr to delete all spaces, however this also may not be what you want.

Do you have any real data we can work with?





Robin
# 4  
Old 03-17-2017
The thing is that sqlplus started life as a command-line reporting tool in the early days of Oracle. The reports had fixed-width data because they were for end-users and humans find fixed width format more convenient to read than csv or delimited data. Delimited data is good for exchanging between systems or applications, but not so good for humans.
Over the years, sqlplus wasn't improved to remedy this particular issue.

In the recent years, Oracle has developed another command-line tool that is kind of a bridge between command-line sqlplus and the graphical tool - SQL Developer. This tool is called "SQLcl" and is at this location: Oracle SQLcl

It's free, it's just a zip file that you extract anywhere you like and all it needs is at least JRE version 8. So, if you are willing to try out this tool and explore it, you will find that it provides a lot of functionality out of the box.

Here are a few commands from my session of SQLcl:

Code:
SQL> 
SQL> show version
Oracle SQLDeveloper Command-Line (SQLcl) version: 4.2.0.17.073.1038
SQL> 
SQL> -- csv format: technique 1
SQL> 
SQL> select /*csv*/ * from scott.emp;
"EMPNO","ENAME","JOB","MGR","HIREDATE","SAL","COMM","DEPTNO"
7369,"SMITH","CLERK",7902,17-DEC-80,800,,20
7499,"ALLEN","SALESMAN",7698,20-FEB-81,1600,300,30
7521,"WARD","SALESMAN",7698,22-FEB-81,1250,500,30
7566,"JONES","MANAGER",7839,02-APR-81,2975,,20
7654,"MARTIN","SALESMAN",7698,28-SEP-81,1250,1400,30
7698,"BLAKE","MANAGER",7839,01-MAY-81,2850,,30
7782,"CLARK","MANAGER",7839,09-JUN-81,2450,,10
7788,"SCOTT","ANALYST",7566,19-APR-87,3000,,20
7839,"KING","PRESIDENT",,17-NOV-81,5000,,10
7844,"TURNER","SALESMAN",7698,08-SEP-81,1500,0,30
7876,"ADAMS","CLERK",7788,23-MAY-87,1100,,20
7900,"JAMES","CLERK",7698,03-DEC-81,950,,30
7902,"FORD","ANALYST",7566,03-DEC-81,3000,,20
7934,"MILLER","CLERK",7782,23-JAN-82,1300,,10

14 rows selected. 

SQL> 
SQL> -- csv format: technique 2
SQL> 
SQL> set sqlformat delimited ,
SQL> 
SQL> -- now all queries will return data in csv format
SQL> 
SQL> select * from scott.emp;
"EMPNO","ENAME","JOB","MGR","HIREDATE","SAL","COMM","DEPTNO"
7369,"SMITH","CLERK",7902,17-DEC-80,800,,20
7499,"ALLEN","SALESMAN",7698,20-FEB-81,1600,300,30
7521,"WARD","SALESMAN",7698,22-FEB-81,1250,500,30
7566,"JONES","MANAGER",7839,02-APR-81,2975,,20
7654,"MARTIN","SALESMAN",7698,28-SEP-81,1250,1400,30
7698,"BLAKE","MANAGER",7839,01-MAY-81,2850,,30
7782,"CLARK","MANAGER",7839,09-JUN-81,2450,,10
7788,"SCOTT","ANALYST",7566,19-APR-87,3000,,20
7839,"KING","PRESIDENT",,17-NOV-81,5000,,10
7844,"TURNER","SALESMAN",7698,08-SEP-81,1500,0,30
7876,"ADAMS","CLERK",7788,23-MAY-87,1100,,20
7900,"JAMES","CLERK",7698,03-DEC-81,950,,30
7902,"FORD","ANALYST",7566,03-DEC-81,3000,,20
7934,"MILLER","CLERK",7782,23-JAN-82,1300,,10

14 rows selected. 

SQL>

That was the easier method because it requires minimum effort on your part.

If you must use sqlplus, then you could use rbatte1's suggestions - use column formats for every column that you will be using, or use a two-step process where you spool the data as it is now and then use a scripting language or command to replace "<multiple_spaces>," by ",". The latter technique is a hit-or-miss because your data itself could have "<multiple_spaces>,".

Finally, the method that would require the most effort on your part would be to handcraft all your queries and concatenate all columns by commas. Something like the following in my sqlplus session:

Code:
SQL> 
SQL> set feedback off
SQL> set pagesize 0
SQL> 
SQL> select empno ||','|| ename ||','|| job ||','|| mgr ||','|| hiredate ||','|| sal ||','|| comm ||','|| deptno from scott.emp;
7369,SMITH,CLERK,7902,17-DEC-80,800,,20
7499,ALLEN,SALESMAN,7698,20-FEB-81,1600,300,30
7521,WARD,SALESMAN,7698,22-FEB-81,1250,500,30
7566,JONES,MANAGER,7839,02-APR-81,2975,,20
7654,MARTIN,SALESMAN,7698,28-SEP-81,1250,1400,30
7698,BLAKE,MANAGER,7839,01-MAY-81,2850,,30
7782,CLARK,MANAGER,7839,09-JUN-81,2450,,10
7788,SCOTT,ANALYST,7566,19-APR-87,3000,,20
7839,KING,PRESIDENT,,17-NOV-81,5000,,10
7844,TURNER,SALESMAN,7698,08-SEP-81,1500,0,30
7876,ADAMS,CLERK,7788,23-MAY-87,1100,,20
7900,JAMES,CLERK,7698,03-DEC-81,950,,30
7902,FORD,ANALYST,7566,03-DEC-81,3000,,20
7934,MILLER,CLERK,7782,23-JAN-82,1300,,10
SQL> 
SQL>

This User Gave Thanks to durden_tyler For This Post:
Jaganjag
# 5  
Old 03-18-2017
Blade Thank you
Thanks durden_tyler for your help . It is working Smilie

---------- Post updated at 09:06 AM ---------- Previous update was at 08:56 AM ----------

I have a SQL script that is selecting rows from a table with more than 50,000 records but I only need one header line at the beginning.

Can any one please suggest the code.

SET HEADING ON
SET PAGESIZE 50000

If I set the 'SET PAGESIZE' to 50000. it repeats, the column headers Evert 50000 lines.
# 6  
Old 03-18-2017
Quote:
Originally Posted by Jaganjag
...
I have a SQL script that is selecting rows from a table with more than 50,000 records but I only need one header line at the beginning.
...
...
If I set the 'SET PAGESIZE' to 50000. it repeats, the column headers Evert 50000 lines.
Set the pagesize to 0 and hard-code the header.
# 7  
Old 03-18-2017
Quote:
Originally Posted by durden_tyler
Set the pagesize to 0 and hard-code the header.
How to hard-code the header ? Where I want to hard-code the header
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Not able to write SQL query output in to .csv file with shell script.

I am trying to write SQL query output into a .csv file. But in the output columns are displaying in different lines instead of coming in one line. Main Code shell script: this is my code: #!/bin/bash file="db_detail.txt" . $file rm /batch/corpplan/bin/dan.csv... (6 Replies)
Discussion started by: sandeepgoli53
6 Replies

2. Shell Programming and Scripting

Shell script appending output of sql query

I am writing the following script to create the file v_out.txt. sqlplus -s /nolog << EOF CONNECT scott/tiger@orcl; whenever sqlerror exit sql.sqlcode; set newpage 0; SET PAGESIZE 0; SET ECHO OFF; SET FEEDBACK OFF; SET HEADING OFF; SET VERIFY OFF; SET LINESIZE 100; set tab off; set... (7 Replies)
Discussion started by: itzkashi
7 Replies

3. Shell Programming and Scripting

How can i run sql queries from UNIX shell script and retrieve data into text docs of UNIX?

Please share the doc asap as very urgently required. (1 Reply)
Discussion started by: 24ajay
1 Replies

4. Shell Programming and Scripting

SQL Query in Shell Script output formatting

Hi All, #!/bin/ksh call_sql () { sql=$1 sqlplus -s $sqlparam_sieb <<EOF SET ECHO OFF; SET NEWPAGE NONE; SET SQLBL OFF; SET VERIFY OFF; SET LINESIZE 2000; SET... (2 Replies)
Discussion started by: Kevin Tivoli
2 Replies

5. Shell Programming and Scripting

Shell script to run sql query having a long listing of parameters

Hi, I have a query regarding execution of a sql query having long listing of parameters ..I need to execute this query inside a shell script. The scenario is like.... Suppose I have a file abc.txt that has the card numbers..it could be in thousands.. then I need to fire a query like ... (12 Replies)
Discussion started by: vsachan
12 Replies

6. Shell Programming and Scripting

Run SQL thru shell script: how to get a new line when run sql query?

Hi, this's Pom. I'm quite a new one for shell script but I have to do sql on shell script to query some information from database. I found a concern to get a new line...When I run my script, it retrieves all data as wondering but it's shown in one line :( What should I do? I'm not sure that... (2 Replies)
Discussion started by: Kapom
2 Replies

7. Shell Programming and Scripting

Problem in formatting output of SQL query in excel sheet in shell script

Hi Guys.. Need your help to format the output of my shell script. I am using spool command to take out put in csv file. below is my code. (for example) col USERNAME for a15 col EMAIL for a30 col FULL_NAME for a20 col LAST_LOGIN for a40 col DATE_CREATED for a40 SPOOL 120.csv... (3 Replies)
Discussion started by: Agupte
3 Replies

8. Shell Programming and Scripting

How to run a SQL select query in Oracle database through shell script?

I need to run a SQL select query in Oracle database and have to capture the list of retrieved records in shell script. Also i would like to modify the query for certain condition and need to fetch it again. How can i do this? Is there a way to have a persistent connection to oracle database... (9 Replies)
Discussion started by: vel4ever
9 Replies

9. Shell Programming and Scripting

how to use data in unix text file as input to an sql query from shell

Hi, I have data in my text file something like this. adams robert ahmed gibbs I want to use this data line by line as input to an sql query which i run by connecting to an oracle database from shell. If you have code for similar scenario , please ehlp. I want the output of the sql query... (7 Replies)
Discussion started by: rdhanek
7 Replies

10. Shell Programming and Scripting

how to assign sql output data to shell script variable

Hi Guys ! I am new to unix and want to find out how we can make sql statement data to shell script variable? Any help/suggestion is greatly appreciated -Chandra (1 Reply)
Discussion started by: kattics
1 Replies
Login or Register to Ask a Question