Sponsored Content
Top Forums Shell Programming and Scripting Is there a way to handle commas inside the data when generating a csv file from shell script? Post 303029298 by patk625 on Thursday 24th of January 2019 01:13:48 PM
Old 01-24-2019
Is there a way to handle commas inside the data when generating a csv file from shell script?

I am extracting data via sql query and some of the data has commas. Output File must be csv and I cannot update the data in the db (as it is used by other application).

Example

table FavoriteThings
Code:
Person VARCHAR2(25),
Favorite VARCHAR2(100)

Sample Data
Code:
Greta        rain drop on roses
Liesl          whiskers on kitten
Kurt            chocolate,chocolate cookies

excerpt from code
Code:
sqlplus -s myuser/mypassword@mydb <<EOF

set feedback off
set linesize 999
whenever sqlerror exit failure;
set trimspool on
set lines 2000
set pages 0
set colsep ","
set serveroutput off


SPOOL favorites.csv
select Person, Favorite from FavoriteThings 
SPOOL_OFF
EXIT
EOF

Output needs to be
Code:
"Greta", "rain drop on roses"
"Liesl" ,"whiskers on kitten"
"Kurt "," chocolate,chocolate cookies"

currently I get
Code:
Greta, rain drop on roses
Liesl ,whiskers on kitten
Kurt , chocolate,chocolate cookies

which when imported into excel shows 4 columns for Kurt

Last edited by vbe; 01-24-2019 at 02:39 PM.. Reason: code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell Script to Load data into the database using a .csv file and .ctl file

Since i'm new to scripting i'm findind it difficult to code a script. The script has to be an executable with 2 paramters passed to it.The Parameters are 1. The Control file name(.ctl file) 2. The Data file name(.csv file) Does anybody have an idea about it? :confused: (3 Replies)
Discussion started by: Csmani
3 Replies

2. Shell Programming and Scripting

how to handle , in data where separator also commas in awk script

TEST_HEME,"SubNetwork=ONRM_RootMoR,SubNetwork=ARNC1",CELL when I split by FS="," then $0=TEST_HEME $1="SubNetwork=ONRM_RootMoR $2=SubNetwork=ARNC1" but I need this will be single value "SubNetwork=ONRM_RootMoR,SubNetwork=ARNC1" (4 Replies)
Discussion started by: Hemendra
4 Replies

3. Shell Programming and Scripting

Exporting data as a CSV file from Unix shell script

Friends...This is the first time i am trying the report generation using shell script... any suggestions are welcome. Is there a way to set the font size & color when i am exporting the data from unix shell script as a CSV file ? The following sample data is saved as a .csv file in the... (2 Replies)
Discussion started by: appu2176
2 Replies

4. Shell Programming and Scripting

Read data from .csv file through shell script & modify

I need to read data from a file called "test.csv" through shell script where the file contains values like name,price,descriptor etc. There are rows where descriptor (& in some rows name) are written as string & other characters like "car_+" OR "bike*" etc where it should contains strings like... (3 Replies)
Discussion started by: raj100
3 Replies

5. Shell Programming and Scripting

shell script to remove extra commas from CSV outp file

Name,,,,,,,,,,,,,,,,,,,,Domain,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Contact,Phone,Email,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Location -----------------------,------------------------------------------------,-------,-----,---------------------------------,------------------------------------ ----... (1 Reply)
Discussion started by: sreenath1037
1 Replies

6. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file based on certain conditions

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (1 Reply)
Discussion started by: Vivekit82
1 Replies

7. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 7 columns having values say column 1,column 2.....column 7 as below along with their values. Name, Address,... (7 Replies)
Discussion started by: Vivekit82
7 Replies

8. Shell Programming and Scripting

Shell script to extract data from csv file

Hi everyone, I have a csv file which has data with different heading and column names as below. Static Data Ingested ,,,,,,,,,,,,Known Explained Rejections Column_1,column_2,Column_3,Column_4,,Column_6,Column_7,,% Column_8,,Column_9 ,Column_10 ,... (14 Replies)
Discussion started by: Vivekit82
14 Replies

9. Shell Programming and Scripting

Generating CSV from Column data

Hi List, I have a chunk of data like so: User Account Control: User Account Control: User Account Control: User Account Control: Disabled User Account Control: User Account Control: User Account Control: Disabled User Account Control: User Account Control: ... (3 Replies)
Discussion started by: landossa
3 Replies

10. Shell Programming and Scripting

Shell script that should remove unnecessary commas between double quotes in CSV file

i have data as below 123,"paul phiri",paul@yahoo.com,"po.box 23, BT","Eco Bank,Blantyre,Malawi" i need an output to be 123,"paul phiri",paul@yahoo.com,"po.box 23 BT","Eco Bank Blantyre Malawi" (5 Replies)
Discussion started by: mathias23
5 Replies
DBIx::Abstract(3pm)					User Contributed Perl Documentation				       DBIx::Abstract(3pm)

NAME
DBIx::Abstract - DBI SQL abstraction VERSION
version 1.01 SYNOPSIS
use DBIx::Abstract; my $db = DBIx::Abstract->connect({ driver=>'mydriver', host=>'myhost.org', dbname=>'mydb', user=>'myuser', password=>'mypassword', }); if ($db->select('*','table')->rows) { while (my $data = $db->fetchrow_hashref) { # ... } } my $id = 23; my ($name) = $db->select('name','table',{id=>$id})->fetchrow_array; ### $db = DBIx::Abstract->connect( { driver=>'csv', f_name=>'foo/' } ); ### $db = DBIx::Abstract->connect({ dsn=>'dbi:someotherdb:so_db_name=mydb', user=>'myuser', password=>'mypassword', }); DESCRIPTION
This module provides methods for doing manipulating database tables This module provides methods retrieving and storing data in SQL databases. It provides methods for all of the more important SQL commands (like SELECT, INSERT, REPLACE, UPDATE, DELETE). It endeavors to produce an interface that will be intuitive to those already familiar with SQL. Notable features include: * data_source generation for some DBD drivers. * Can check to make sure the connection is not stale and reconnect if it is. * Controls statement handles for you. * Can delay writes. * Generates complex where clauses from hashes and arrays. * Shortcuts (convenience functions) for some common cases. (Like select_all_to_hashref.) DEPRICATED
We highly recommend that you use something like SQL::Abstract, which was inspired by this module. Or even DBIx::Class (which uses SQL::Abstract for it's query syntax). They're maintained and widely used. METHODS
Unless otherwise mentioned all methods return the database handle. connect "connect($connect_config | $dbihandle [,$options])" CONSTRUCTOR Open a connection to a database as configured by $connect_config. $connect_config can either be a scalar, in which case it is a DBI data source, or a reference to a hash with the following keys: dsn -- The data source to connect to your database OR, DBIx::Abstract will try to generate it if you give these instead: driver -- DBD driver to use (defaults to mysql) host -- Host of database server port -- Port of database server dbname -- Name of database Username and password are always valid. user -- Username to connect as password -- Password for user Alternatively you can pass in a DBI handle directly. This will disable the methods "reconnect" and "ensure_connection" as they rely on connection info not available on a DBI handle. Options is a hash reference. Each key/value pair is passed on to the opt method. clone This clones the object. For those times when you need a second connection to the same DB. If you need a second connection to a different DB, create a new object with 'connect'. This operation is logged at level 5 with the message "Cloned." connected Check to see if this object is connected to a database. It checks to see if it has a database handle and if that handle's "Active" attribute is true. reconnect If the object is not connected to a database it will reconnect using the same parameters connect was originally called with. ensure_connection Makes sure that the object is connect to a database. Makes sure that the connect is active (by sending a "SELECT 1"). If there is no connection, or the connection is not active then it tries to reconnect. If it fails to reconnect then it dies. opt ($key[,$value]) ({key=>$key[,value=>$value]) Set option $key to $value. Available keys are: loglevel (default 0) 0 -- Fatal errors only 1 -- Modifications 2 -- And selects 3 -- And user created queries 4 -- And results of queries 5 -- And other misc commands 6 -- Internals of commands logfile (default undef) Log file delaymods (default false) Delay making modifications to the database until run_delayed is run. useCached If this is true then prepare_cached is used instead of prepare. Checkout the DBI documentation on this feature before using this feature. saveSQL If this is true then with each query DBIx::Abstract will stuff the generated SQL into the 'lastsql' key in the self payload. Additionally you may use any valid DBI attribute. So, for instance, you can pass AutoCommit or LongReadLen. This operation is logged at level 5 with the message "Option Change" and the the key, the old value and new new value. query ($sql,@bind_params) ({sql=>$sql,bind_params=>[@bind_params]}) This sends $sql to the database object's query method. This should be used for applications where the existing methods are not able to generate flexible enough SQL for you. If you find yourself using this very often with things other then table manipulation (eg 'create table','alter table','drop table') then please let me know so I can extend DBIx::Abstract to include the functionality you are using. This operation is logged at level 3 run_delayed Execute delayed update/insert/delete queries. This operation is logged at level 5 with the message "Run delayed". delete ($table[,$where]) ({table=>$table[,where=>$where]}) Deletes records from $table. See also the documentation on "DBIx::Abstract Where Clauses". insert ($table,$fields) ({table=>$table,fields=>$fields}) $table is the name of the table to insert into. $fields is either a reference to a hash of field name/value or a scalar containing the SQL to insert after the "SET" portion of the statement. These all produce functionally equivalent SQL. $db->insert('foo',{bar=>'baz'}); $db->insert('foo',q|bar='baz'|); $db->insert({table=>'foo',fields=>{bar=>'baz'}}); $db->insert({table=>'foo',fields=>q|bar='baz'|}); We also support literals by making the value in the hash an arrayref: $db->insert('foo',{name=>'bar',date=>['substring(now(),1,10)']}); Would generate something like this: INSERT INTO foo (name,date) VALUES (?,substring(now(),1,10)) With "bar" bound to the first parameter. replace ($table,$fields) ({table=>$table,fields=>$fields}) $table is the name of the table to replace into. $fields is either a reference to a hash of field name/value or a scalar containing the SQL to insert after the "SET" portion of the statement. Replace works just like insert, except that if a record with the same primary key already exists then the existing record is replaced, instead of producing an error. update ($table,$fields[,$where]) ({table=>$table,fields=>$fields[,where=>$where]}) $table is the table to update. $fields is a reference to a hash keyed on field name/new value. See also the documentation on "DBIx::Abstract Where Clauses". select "select" ($fields,[$table,[$where[,$order]]]) ({ fields=>$fields, table=>$table [,where=>$where] [,order=>$order] [,join=>$join] [,group=>$group] }) The select method returns the DBIx::Abstract object it was invoked with. This allows you to chain commands. $fields can be either an array reference or a scalar. If it is an array reference then it should be a list of fields to include. If it is a scalar then it should be a literal to be inserted into the generated SQL after "SELECT". $table can be either an array reference or a scalar. If it is an array reference then it should be a list of tables to use. If it is a scalar then it should be a literal to be inserted into the generated SQL after "FROM". See also the documentation on "DBIx::Abstract Where Clauses". $order is the output order. If it is a scalar then it is inserted literally after "ORDER BY". If it is an arrayref then it is join'd with a comma and inserted. $join is there to make joining tables more convenient. It will takes one or more (as an arrayref) sets of statements to use when joining. For instance: $dbh->select({ fields=>'*', table=>'foo,bar', join=>'foo.id=bar.foo_id', where=>{'foo.dollars',['>',30]} }); Would produce: SELECT * FROM foo,bar WHERE (foo.dollars > ?) and (foo.id=foo_id) And put 30 into the bind_params list. $group is/are the field(s) to group by. It may be scalar or an arrayref. If it is a scalar then it should be a literal to be inserted after "GROUP BY". If it is an arrayref then it should be a list of fields to group on. select_one_to_hashref ($fields,$table[,$where]) ({fields=>$fields,table=>$table[,where=>$where]}) This returns a hashref to the first record returned by the select. Typically this should be used for cases when your where clause limits you to one record anyway. $fields is can be either a array reference or a scalar. If it is an array reference then it should be a list of fields to include. If it is a scalar then it should be a literal to be inserted into the generated SQL. $table is the table to select from. See also the documentation on "DBIx::Abstract Where Clauses". select_one_to_arrayref ($fields,$table[,$where]) ({fields=>$fields,table=>$table[,where=>$where]}) This returns a arrayref to the first record returned by the select. Typically this should be used for cases when your where clause limits you to one record anyway. $fields is can be either a array reference or a scalar. If it is an array reference then it should be a list of fields to include. If it is a scalar then it should be a literal to be inserted into the generated SQL. $table is the table to select from. See also the documentation on "DBIx::Abstract Where Clauses". select_one_to_array ($fields,$table[,$where]) ({fields=>$fields,table=>$table[,where=>$where]}) This returns a array to the first record returned by the select. Typically this should be used for cases when your where clause limits you to one record anyway. $fields is can be either a array reference or a scalar. If it is an array reference then it should be a list of fields to include. If it is a scalar then it should be a literal to be inserted into the generated SQL. $table is the table to select from. See also the documentation on "DBIx::Abstract Where Clauses". select_all_to_hashref ($fields,$table[,$where]) ({fields=>$fields,table=>$table[,where=>$where]}) This returns a hashref to all of the results of the select. It is keyed on the first field. If there are only two fields then the value is just the second field. If there are more then two fields then the value is set to an arrayref that contains all of the fields. $fields is can be either a array reference or a scalar. If it is an array reference then it should be a list of fields to include. If it is a scalar then it should be a literal to be inserted into the generated SQL. $table is the table to select from. See also the documentation on "DBIx::Abstract Where Clauses". fetchrow_hashref This is just a call to the DBI method. fetchrow_hash This calls fetchrow_hashref and dereferences it for you. fetchrow_array This method calls the database handle's method of the same name. fetchall_arrayref This method calls the database handle's method of the same name. rows This method calls the database handle's method of the same name. quote This method is passed to the database handle via AUTOLOAD. disconnect This method is passed to the database handle via AUTOLOAD. commit This method is passed to the database handle via AUTOLOAD. rollback This method is passed to the database handle via AUTOLOAD. trace This method is passed to the database handle via AUTOLOAD. finish This method is passed to the statement handle via AUTOLOAD. bind_col This method is passed to the statement handle via AUTOLOAD. bind_columns This method is passed to the statement handle via AUTOLOAD. Other things that need explanation DBIx::Abstract Where Clauses Where clauses in DBIx::Abstract can either be very simple, or highly complex. They are designed to be easy to use if you are just typing in a hard coded statement or have to build a complex query from data. Wheres are either a scalar, hash-ref or array-ref: If it is a scalar, then it is used as the literal where. If it is a hash-ref then the key is the field to check, the value is either a literal value to compare equality to, or an array-ref to an array of operator and value. { first=>'joe', age=>['>',26], last=>['like',q|b'%|] } Would produce: WHERE first = ? AND last like ? AND age > ? With joe, b'% and 26 passed as bind values. If it is an array-ref then it is an array of hash-refs and connectors: [ { first=>'joe', age=>['>',26] }, 'OR', { last=>['like',q|b'%|] } ] Would produce: WHERE (first = ? AND age > ?) OR (last like ?) With joe, 26 and b'% passed as bind values. [ { first=>'joe', last=>['like','%foo%'], }, 'AND', [ {age=>['>',26]}, 'OR', {age=>['<',30]} ] ] Would produce: WHERE (first = ? AND last like ?) AND ((age > ?) OR (age < ?)) With joe, %foo%, 26 and 30 passed as bind values. SUPPORTED DBD DRIVERS
These drivers have been reported to work: o mysql (development environment) o Pg (development environment) o Oracle o XBase Any driver that uses ODBC syntax should work using the hash ref method. With other drivers you should pass the DBI data source instead (this method will work with all drivers.) CHANGES SINCE LAST RELEASE
o Fixed a bug discovered by Jan Martin Mathiassen <reaper@mindriot.as>. If you pass in a password but no username it's supposed to ignore the password. Instead it was clearing it in the hash that you passed in but still using it. o Removed Test::More and Test::Builder from the distribution-- put Test::Simple in the prerequisites. AUTHOR
Rebecca Turner <winter@cpan.org> COPYRIGHT AND LICENSE
Portions copyright 2001-2002,2011 by Rebecca Turner Portions copyright 2000-2001 by Adelphia Business Solutions Copyright 1998-2000 by the Maine Internetworks (MINT) This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
http://sourceforge.net/projects/dbix-abstract/ DBI(3) AUTHOR
Becca <becca@referencethis.com> COPYRIGHT AND LICENSE
This software is copyright (c) 2011 by Rebecca Turner. This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself. perl v5.12.4 2011-09-24 DBIx::Abstract(3pm)
All times are GMT -4. The time now is 10:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy