Removing \n from .csv file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing \n from .csv file
# 1  
Old 01-23-2013
Removing \n from .csv file

Hi,

I have a requirement like my .csv file is generating from a db2 table using export command like below:

file format:
-----------

Code:
2011	4	0	0	N	S	C		C	"BHPC
BHPC"	0	0	0
2011	5	0	0	N	S	C		C	"BHPC
BHPC"	0	0	0

here BHPC is having new line character and because this when i am trying to load this file to a table it is loading in two rows and this should not be done.
please help me to delete the \n character in between "BHPC
BHPC"
with unix commands and at the same time it should load into the table with BHPC BHPC.
I have searched in the forun but couldn`t able to find out the solution.
Please let me know if you want anyother information on this.


Thanks In Advance
Varma.

Last edited by Scrutinizer; 01-23-2013 at 05:25 AM.. Reason: code tags
# 2  
Old 01-23-2013
Do you mean like this?
Code:
awk '$NF=="\"BHPC"{getline p; $0=$0 OFS p}1' file


Last edited by Scrutinizer; 01-23-2013 at 06:10 AM..
# 3  
Old 01-23-2013
Does that field have a newline stored in the database? If so wouldn't it be better to remove it using a database function?

Or if it doesn't have a newline in the database, is it just that you need to increase the line size?
# 4  
Old 01-23-2013
Hi,

Actually from the database the value is coming like C "BHPC
BHPC" 0 0
and if i looked it in unix environment by doing awk -F "," {print $35} with the file it is showing like

"BHPC
N
"BHPC
N
.
.
.
like this..

sorry I don`t have information on the database like how it was stored.

Regards,
pradeep.
# 5  
Old 01-23-2013
hey Varma,

please use the below code.

#This one-liner also starts with creating a named label "a". Then it tests to see if it is not the last line and appends the next line to the current one with "N" command. If the just appended line starts with a "BHPC", one-liner branches the label "a" to see if there are more lines starting with "BHPC". During this process a substitution gets executed which throws away the newline character which came from joining with "N" and the "BHPC". If the substitution fails, one-liner prints out the pattern space up to the newline character with the "P" command, and deletes the contents of pattern space up to the newline character with "D" command, and repeats the process.

Code:
sed -e :a -e '$!N;s/\nBHPC/ BHPC/;ta' -e 'P;D' $file

Moderator's Comments:
Mod Comment Please use code tags for code and data

Last edited by Scrutinizer; 01-24-2013 at 12:18 AM.. Reason: code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Removing commas from CSV file

Hi I'm creating a sh script to generate a csv file. The CSV contains the values from a sql table. The content looks this: a,b,c,c2,c3,,,,,,,,,,,d,e I have some code that can separate the fields using the comma as delimiter, but some values actually contain commas, such as... (2 Replies)
Discussion started by: preema
2 Replies

2. Shell Programming and Scripting

Removing string from CSV file by provide removal string from other file

What I need is to remove the text from Location_file.txt from each line matching all entries from Remove_location.txt Location_file.txt FlowPrePaid, h3nmg1cm2,Jamaica_MTAImageFileFlowPrePaid,h0nmg1cm1, Flow_BeatTest,FlowRockTest FlowNewTest,FlowNewTest,h0nmg1cm1 PartiallySubscribed,... (3 Replies)
Discussion started by: ketanraut
3 Replies

3. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

4. Shell Programming and Scripting

Egrep or awk for removing values within CSV file?

Hello, I have a large CSV file that contains values all on the same column, and in one very long row (e.g. no line breaks till end, with all data values separated by a comma). The file has two types of data for the values. One begins with the letters rs and some numbers. The other begins... (4 Replies)
Discussion started by: macurdy
4 Replies

5. Shell Programming and Scripting

Bash script help - removing certain rows from .csv file

Hello Everyone, I am trying to find a way to take a .csv file with 7 columns and a ton of rows (over 600,000) and remove the entire row if the cell in forth column is blank. Just to give you a little background on why I am doing this (just in case there is an easier way), I am pulling... (3 Replies)
Discussion started by: MrTuxor
3 Replies

6. Shell Programming and Scripting

removing duplicate records comparing 2 csv files

Hi All, I want to remove the rows from File1.csv by comparing a column/field in the File2.csv. If both columns matches then I want that row to be deleted from File1 using shell script(awk). Here is an example on what I need. File1.csv: RAJAK,ACTIVE,1 VIJAY,ACTIVE,2 TAHA,ACTIVE,3... (6 Replies)
Discussion started by: rajak.net
6 Replies

7. Shell Programming and Scripting

Removing comma "," in a field value in csv file

Hi, I have csv file with records as below. Now i have remove any comma in the filed value because that creates problem when i feed this file to an application. for example below are two sample records, the second record have a comma in "Salesforce.com, Inc." field, now i have to remove this... (13 Replies)
Discussion started by: anaga
13 Replies

8. Shell Programming and Scripting

Removing lines of a .csv file

Hello, Does anyone have a one-liner to remove lines of a csv file if the value in a specific column is zero? For example, I have this file, 12345,COM,5,0,N,29.95,Y 12345,MOM,1,0,N,29.95,Y 12345,COM,4,0,N,9.99,Y 12345,MOM,0,2,N,9.99,Y 12345,REN,0,1,N,9.99,Y and I want to remove lines... (4 Replies)
Discussion started by: palex
4 Replies

9. Linux

Removing non printing characters from a csv file

Hi, I have an csv file and there are some non printable characters(extended ascii) so I am trying to create a clean copy of the csv file . I am using this command: tr -cd "" < /opt/informatica/PowerCenter8.6.0/server/infa_shared/SrcFiles/ThirdParty/locations.csv > ... (4 Replies)
Discussion started by: gerkus
4 Replies
Login or Register to Ask a Question