Sponsored Content
Top Forums Shell Programming and Scripting Excel CSV with Double Quotes and Carriage Returns Post 302597258 by landossa on Thursday 9th of February 2012 07:44:15 PM
Old 02-09-2012
Excel CSV with Double Quotes and Carriage Returns

Hi List,

I am sure this issue must have been encountered before. I have saved an excel CSV and sent it to my linux box where I require to parse it as a CSV line by line. The file displays fine in Excel with csv headers and consistent records and fields. However when I look at it in linux I see multiple carriage returns and double quotes which don't appear when viewing in Excel.

Is there any unix/sed/awk script about which I can use to run from my linux box to convert this excel format into a workable CSV which is parse-able with unix tools?

I am sure there must be a simple substitution script which can achieve this.

Does anyone have any ideas?

Thanks,
Land
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

replace value with double quotes of specific coulmn value in csv file

Hi, I am trying to replace a specific column values in a csv file with double quotes. Example: SNO,NAME,ZIPCODE,RANK 1,Robert,74538,12 2,Sam,07564,13 3,Kim, Ed,12345,14 Desired Output: SNO,NAME,ZIPCODE,RANK 1,Robert Ken,74538,12 2,Sam Mik,"07564",13 3,"Kim, Ed",12345,14 I... (3 Replies)
Discussion started by: techmoris
3 Replies

2. Shell Programming and Scripting

Replacing comma with in double quotes in a csv file

Hello, I need to read a csv file and I am trying to replace a comma with a text DSEE?DSEE. Example Input "Chapter","NewTrains, "oldTrains","Delayed",10,"London" "Chapter","Newbuses,oldbuses","On Time",20,"London" Output "Chapter","NewTrainsDSEE?DSEE... (5 Replies)
Discussion started by: venkatvani
5 Replies

3. Shell Programming and Scripting

HELP with AWK or SED. Need to replace the commas between double quotes in CSV file

Hello experts, I need to validate a csv file which contains data like this: Sample.csv "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 I just need to check if all the records contain exactly the number of... (5 Replies)
Discussion started by: shell_boy23
5 Replies

4. Shell Programming and Scripting

Convert csv to pipe delimited except the ones in double quotes

I have a csv data file : A,B,C,D,"A,B",E,"GG,H" E,F,G,H,I,J,"S,P" I need to replace all "," with "|" except the ones between double quotes i.e A|B|C|D|"A,B"|E|"GG,H" E|F|G|H|I|J|"S,P" CAn someone assist? (8 Replies)
Discussion started by: Shivdatta
8 Replies

5. Shell Programming and Scripting

carriage returns within quotation marks causing new lines in csv

I have a csv file with 3 columns. Fields are comma delimited and strings are enclosed with quotation marks "". About 40% of the time, the line of values will start a new line thanks to carriage return characters within a string. Example: "apple","banana","orange" "pineapple","grape","straw... (6 Replies)
Discussion started by: koeji
6 Replies

6. Shell Programming and Scripting

How to delete the commas in a .CSV file that are enclosed in a string with double quotes?

Okay, I would like to delete all the commas in a .CSV file (TEST.CSV) or at least substitute them with empty space, that are enclosed in double quote. Please see the sample file as below: column 1,column 2,column 3,column 4,column 5,column 6,column 7,column 8,column 9,column 10... (8 Replies)
Discussion started by: dhruuv369
8 Replies

7. Shell Programming and Scripting

Shell script that should remove unnecessary commas between double quotes in CSV file

i have data as below 123,"paul phiri",paul@yahoo.com,"po.box 23, BT","Eco Bank,Blantyre,Malawi" i need an output to be 123,"paul phiri",paul@yahoo.com,"po.box 23 BT","Eco Bank Blantyre Malawi" (5 Replies)
Discussion started by: mathias23
5 Replies

8. Shell Programming and Scripting

Multiple carriage returns within quotation marks causing new lines in csv

There is a closed thread called "carriage returns within quotation marks causing new lines in csv" that I am unable to post to, so I am starting a new thread. The awk solution worked perfectly in most cases. We have some cases where there are multiple carriage returns within a single quoted... (9 Replies)
Discussion started by: Mary Roberts
9 Replies

9. Shell Programming and Scripting

Replace Double quotes within double quotes in a column with space while loading a CSV file

Hi All, I'm unable to load the data using sql loader where there are double quotes within the double quotes As these are optionally enclosed by double quotes. Sample Data : "221100",138.00,"D","0019/1477","44012075","49938","49938/15043000","Television - 22" Refurbished - Airwave","Supply... (6 Replies)
Discussion started by: mlavanya
6 Replies

10. Shell Programming and Scripting

Remove Carriage Return (CRLF) within double quotes

How to remove Carriage Return (CRLF) within double quotes in a file. There are multiple CRLFs within double quotes. We are on Ubuntu 14.04.2 LTS. The file that we are importing is a csv file from unix to windows and the file was formatted to unix2dos. Therefore all lines in the file all have... (12 Replies)
Discussion started by: covina
12 Replies
csv(n)								  CSV processing							    csv(n)

NAME
csv - Procedures to handle CSV data. SYNOPSIS
package require Tcl 8.3 package require csv ?0.3? ::csv::join values {sepChar ,} ::csv::joinlist values {sepChar ,} ::csv::read2matrix chan m {sepChar ,} {expand none} ::csv::read2queue chan q {sepChar ,} ::csv::report cmd matrix ?chan? ::csv::split line {sepChar ,} ::csv::split2matrix m line {sepChar ,} {expand none} ::csv::split2queue q line {sepChar ,} ::csv::writematrix m chan {sepChar ,} ::csv::writequeue q chan {sepChar ,} DESCRIPTION
The csv package provides commands to manipulate information in CSV FORMAT (CSV = Comma Separated Values). COMMANDS
The following commands are available: ::csv::join values {sepChar ,} Takes a list of values and returns a string in CSV format containing these values. The separator character can be defined by the caller, but this is optional. The default is ",". ::csv::joinlist values {sepChar ,} Takes a list of lists of values and returns a string in CSV format containing these values. The separator character can be defined by the caller, but this is optional. The default is ",". Each element of the outer list is considered a record, these are separated by newlines in the result. The elements of each record are formatted as usual (via ::csv::join). ::csv::read2matrix chan m {sepChar ,} {expand none} A wrapper around ::csv::split2matrix (see below) reading CSV-formatted lines from the specified channel (until EOF) and adding them to the given matrix. For an explanation of the expand argument see ::csv::split2matrix. ::csv::read2queue chan q {sepChar ,} A wrapper around ::csv::split2queue (see below) reading CSV-formatted lines from the specified channel (until EOF) and adding them to the given queue. ::csv::report cmd matrix ?chan? A report command which can be used by the matrix methods format 2string and format 2chan. For the latter this command delegates the work to ::csv::writematrix. cmd is expected to be either printmatrix or printmatrix2channel. The channel argument, chan, has to be present for the latter and must not be present for the first. ::csv::split line {sepChar ,} converts a line in CSV format into a list of the values contained in the line. The character used to separate the values from each other can be defined by the caller, via sepChar, but this is optional. The default is ",". ::csv::split2matrix m line {sepChar ,} {expand none} The same as ::csv::split, but appends the resulting list as a new row to the matrix m, using the method add row. The expansion mode specified via expand determines how the command handles a matrix with less columns than contained in line. The allowed modes are: none This is the default mode. In this mode it is the responsibility of the caller to ensure that the matrix has enough columns to contain the full line. If there are not enough columns the list of values is silently truncated at the end to fit. empty In this mode the command expands an empty matrix to hold all columns of the specified line, but goes no further. The overall effect is that the first of a series of lines determines the number of columns in the matrix and all following lines are truncated to that size, as if mode none was set. auto In this mode the command expands the matrix as needed to hold all columns contained in line. The overall effect is that after adding a series of lines the matrix will have enough columns to hold all columns of the longest line encountered so far. ::csv::split2queue q line {sepChar ,} The same as ::csv::split, but appending the resulting list as a single item to the queue q, using the method put. ::csv::writematrix m chan {sepChar ,} A wrapper around ::csv::join taking all rows in the matrix m and writing them CSV formatted into the channel chan. ::csv::writequeue q chan {sepChar ,} A wrapper around ::csv::join taking all items in the queue q (assumes that they are lists) and writing them CSV formatted into the channel chan. FORMAT
Each record of a csv file (comma-separated values, as exported e.g. by Excel) is a set of ASCII values separated by ",". For other lan- guages it may be ";" however, although this is not important for this case (The functions provided here allow any separator character). If a value contains itself the separator ",", then it (the value) is put between "". If a value contains ", it is replaced by "". EXAMPLE
The record 123,"123,521.2","Mary says ""Hello, I am Mary""" is parsed as follows: a) 123 b) 123,521.2 c) Mary says "Hello, I am Mary" SEE ALSO
matrix, queue KEYWORDS
csv, matrix, queue, package, tcllib csv 0.3 csv(n)
All times are GMT -4. The time now is 08:59 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy