Sponsored Content
Top Forums Shell Programming and Scripting carriage returns within quotation marks causing new lines in csv Post 302695335 by koeji on Sunday 2nd of September 2012 10:17:20 PM
Old 09-02-2012
carriage returns within quotation marks causing new lines in csv

I have a csv file with 3 columns. Fields are comma delimited and strings are enclosed with quotation marks "". About 40% of the time, the line of values will start a new line thanks to carriage return characters within a string.

Example:
Code:
"apple","banana","orange"
"pineapple","grape","straw
berry"
"apple","banana","cherry"

Should be:
Code:
"apple","banana","orange"
"pineapple","grape","strawberry"
"apple","banana","cherry"

How to create a bash script that will catch this issue and combine the line and the line below?

Last edited by Franklin52; 09-03-2012 at 03:34 AM.. Reason: Please use code tags for data and code samples
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

spaces and carriage returns in 'here documents'

As the title suggests, i am having some trouble figuring out how to pass spaces and carriage returns to a 'here document' ie #!/bin/bash /usr/local/install_script.sh <<SCRIPT yes no <pass carriage retun here> yes no <pass a space and then a carriage return here> exit SCRIPT any... (0 Replies)
Discussion started by: hcclnoodles
0 Replies

2. Shell Programming and Scripting

Removing carriage returns with sed

How do we delete all carriage returns after a particular string using sed inside a K Shell? e.g. I have a text file named file1 below: $ more file1 Group#=1 User=A Role=a1 Group#=2 User=B Role=a1 Role=b1 Group#=3 User=C Role=b1 I want the carriage returns to be delete on the... (12 Replies)
Discussion started by: stevefox
12 Replies

3. Shell Programming and Scripting

enclose a line with quotation marks

i have a file like this aaaa bbbb cccc aaa aaaa aa cccccccccccccccc aaaaaaa aaaa aaaa i want to enclose this lines with double quotation: "aaaa bbbb cccc" "aaa aaaa" "aa cccccccccccccccc" "aaaaaaa aaaa aaaa" any idea? (preferably without using sed) thanks in advance... (3 Replies)
Discussion started by: gfhgfnhhn
3 Replies

4. Shell Programming and Scripting

Count number of character occurence but not from quotation marks

I have the following string: 31-01-2012, 09:42:37;OK;94727132638;"Mozilla/5.0 (Linux; U; Android 2.2.1)";3G;WAP;I need a script which is counting the occurrence of semicolons ( ; ) but exclude the ones from the quotation marks. In the string given as example there are 8 semicolons but the script... (3 Replies)
Discussion started by: calinlicj
3 Replies

5. Shell Programming and Scripting

Excel CSV with Double Quotes and Carriage Returns

Hi List, I am sure this issue must have been encountered before. I have saved an excel CSV and sent it to my linux box where I require to parse it as a CSV line by line. The file displays fine in Excel with csv headers and consistent records and fields. However when I look at it in linux I see... (2 Replies)
Discussion started by: landossa
2 Replies

6. Shell Programming and Scripting

Why double quotation marks doesn't work in ksh function parameters passing?

I'm working on AIX 6, ksh shell. The parameters are some strings quotated by double quotation marks which from a file. They are quotated because there may be spaces in them. Example: "015607" "10" " " "A"I want to pass these parameters to a shell function by writing the following command: ... (4 Replies)
Discussion started by: Shimmey
4 Replies

7. Shell Programming and Scripting

TR not removing carriage returns

I have a CSV with carriage returns in place of newlines. I am trying to use tr to remove them, but it isn't working. Academic year,Term,Course name,Period,Last name,Nickname 2012-2013,First Semester,English 12,4th Period,Arnold,Adam 2012-2013,First Semester,English 12,4th Period,Adams,Jim... (1 Reply)
Discussion started by: nextyoyoma
1 Replies

8. Shell Programming and Scripting

Multiple carriage returns within quotation marks causing new lines in csv

There is a closed thread called "carriage returns within quotation marks causing new lines in csv" that I am unable to post to, so I am starting a new thread. The awk solution worked perfectly in most cases. We have some cases where there are multiple carriage returns within a single quoted... (9 Replies)
Discussion started by: Mary Roberts
9 Replies

9. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

10. Shell Programming and Scripting

Remove carriage returns from awk output

I'm on Linux version 2.6.32-696.3.1.el6.x86_64, using the Ksh shell. I'm working with the input file: John Daggett, 341 King Road, Plymouth MA Alice Ford, 22 East Broadway, Richmond VA Orville Thomas, 11345 Oak Bridge Road, Tulsa OK Terry Kalkas, 402 Lans Road, Beaver Falls PA Eric Adams,... (2 Replies)
Discussion started by: prooney
2 Replies
diff3(1)						      General Commands Manual							  diff3(1)

NAME
diff3 - Compares three files SYNOPSIS
diff3 [-e | -x | -E | -X | -3] file1 file2 file3 The diff3 command reads three versions of a file and writes to standard output the ranges of text that differ. OPTIONS
Creates an edit script for use with the ed command to incorporate into file1 all changes between file2 and file3 (that is, the changes that normally would be flagged ==== and ====3). Produces an edit script to incorporate only changes flagged ====. These are similar to -e and -x, respectively, but treat overlapping changes (that is, changes that are flagged ==== in the normal listing) differently. The overlap- ping lines from both files are inserted by the edit script, bracketed by <<<<<< and >>>>>> lines. The -E option is used by RCS merge to ensure that overlapping changes in the merged files are preserved and brought to someone's attention. Produces an edit script to incorpo- rate only changes flagged ====3. DESCRIPTION
The diff3 command reads three versions of a file and writes to standard output the ranges of text that differ, flagged with the following codes: All three files differ. file1 differs. file2 differs. file3 differs. The type of change needed to convert a given range of a given file to match another file is indicated in one of these two ways in the out- put: Text is to be added after line number number1 in file, where file is 1, 2, or 3. Text in the range line number1 to line number2 is to be changed. If number1 = number2, the range may be abbreviated to number1. The original contents of the range follow immediately after a c indication. When the contents of two files are identical, diff3 does not show the contents of the lower-numbered file, although it shows the location of the identical lines for each. NOTES
Editing scripts produced by the -e option cannot create lines consisting only of a single . (dot). EXAMPLES
To list the differences among three files, enter: diff3 fruit.a fruit.b fruit.c fruit.a, fruit.b, and fruit.c contain the following data: fruit.a: banana grape kiwi lemon mango orange peach pare fruit.b: apple banana grapefruit kiwi orange peach pear fruit.c: grape grapefruit kiwi lemon mango orange peach pear The output from diff3 shows the differences between these files as follows. (The comments on the right do not appear in the output.) ==== All three files are different. 1:1,2c - Lines 1 and 2 of the first file, fruit.a banana grape 2:1,3c - Lines 1 through 3 of fruit.b apple banana grapefruit 3:1,2c - Lines 1 and 2 of fruit.c grape grapefruit ====2 The second file, fruit.b, is different. 1:4,5c - Lines 4 and 5 are the same in fruit.a and fruit.c. 2:4a 3:4,5c - To make fruit.b look the same, add text after line 4. lemon mango ====1 The first file, fruit.a, is different. 1:8c pare 2:7c - Line 7 of fruit.b and line 8 of fruit.c are the same. 3:8c pear FILES
Helper program. SEE ALSO
Commands: bdiff(1), cmp(1), comm(1), diff(1), ed(1) diff3(1)
All times are GMT -4. The time now is 05:59 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy