Removing newline characters within DEL quotes.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing newline characters within DEL quotes.
# 1  
Old 09-01-2015
Removing newline characters within DEL quotes.

Hi,

Text file has DEL character(ASCII code 127) as quotes with comma as field delimiter. If any of the field contains new line character then I need to remove it. Please help me to achieve this.

Thanks
Vikram
# 2  
Old 09-01-2015
Samples, please (in code tags). Any attempts from your side?
# 3  
Old 09-01-2015
Firstly creating a test input file. I used this method:

Code:
$ DEL=$(printf '%b' "\x7F")
$ sed "s/~/$DEL/g" > infile <<EOF
> one,two,~Multi-line
> field
> with embedded ,~,four,five
> one,two,three,four,five
> one,two,~simple,string~,four,five
> EOF
$ sed "s/$DEL/|/g" infile
one,two,|Multi-line
field
with embedded ,|,four,five
one,two,three,four,five
one,two,|simple string|,four,five

And here is a GNU awk solution using a modified version of Lorance Freeshell's AWK CSV parser, Lorance hasn't updated this parser since 2009 and it has a few bugs corrected in the attached version:

Code:
awk -v DEL=$(printf "%b" "\x7F") '
@include "csv.awk"
{ num_fields = csv_parse($0, csv, ",", DEL, "", "\n", 1)
  for(i=1;i<=num_fields;i++) gsub("\n", " ", csv[i]);
  print csv_create(csv, num_fields, ",", DEL)
}' infile > outfile

If you don't have GNU awk you can insert the csv.awk file in the code directly instead of using the GNU @includefeature used above.

Result:
Code:
$ sed "s/$DEL/|/g" outfile
one,two,|Multi-line field with embedded ,|,four,five
one,two,three,four,five
one,two,simple string,four,five

# 4  
Old 09-15-2015
Hi,

Thanks for for your help.

I found this code to remove the new line characters present within the quotes.

Code:
awk '(NR-1)%2  {$1=$1}1' RS=\" ORS=\" infile

This is working fine. But is adding extra line with " in it. Please help to remove the last line with ".
Code:
 
 
Sample Data :
 
"Line1  RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake
View Way"
"Line 2 RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake
View Way"
 
Result :
 
"Line1 RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"
"Line 2 RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"
"
 
Expected:
 
"Line1 RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"
"Line 2 RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"

Thanks
Vikram
# 5  
Old 09-16-2015
try this:

Code:
awk '{for(i=1;i<=NF;i+=6){print $i,$(i+1),$(i+2),FS,$(i+3),$(i+4),$(i+5)}}' FS="\n"  RS=""

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Replace newline with comma and append quotes

Hi, I have below requirement. Apple Orange Banana Required O/p in bash 'Apple,Orange,Banana' Can you please help. Please wrap your samples, codes in CODE TAGS as per forum rules. (3 Replies)
Discussion started by: Rtk
3 Replies

2. Shell Programming and Scripting

Removing consecutive double quotes

Hi I have a .csv file and when opened in notepad looks like this gggg,nnnn,"last,first","llll""",nnn So, Here I would like the ouput as below gggg,nnnn,"last,first","llll",nnn i.e replace all two double quotes into one. How could I do that? This file is being processed by another... (5 Replies)
Discussion started by: dnat
5 Replies

3. Shell Programming and Scripting

Script for removing newline character from file

Hi below is my file. cat input.dat 101,abhilash,1000 102,prave en,2000 103,partha,4 000 10 4,naresh,5000 (its just a example file) and my output should be: 101,abhilash,1000 102,praveen,2000 103,partha,4000 104,naresh,5000 below is my code cat input.dat |tr -d '\n' >... (6 Replies)
Discussion started by: abhilash_nakka
6 Replies

4. Shell Programming and Scripting

Removing quotes in sed

I have a file and I want to remove quotes from the word " ` " through sed command but unable to remove I am using below command sed s/"`XYZ`"/"ZXY"/g file1.txt > file2.txt But this is not working. How can we remove "`" through sed command (2 Replies)
Discussion started by: kaushik02018
2 Replies

5. Shell Programming and Scripting

Removing quotes within quotes

Hello everyone, I am working on a file with thousands of lines and instead of manually removing them I need a script to remove quotes within quotes. For example a line may have something such as this: "Hey, I was ready to go on stage or "break a leg", but I failed miserably." So I need to... (15 Replies)
Discussion started by: tastybrownies
15 Replies

6. Shell Programming and Scripting

Replace newline character between a double quotes to a space

Hi Guys, I have a file with content as below aj.txt "Iam allfine" abcdef abcd "all is not well" What I'm trying to say is my data has some new line characters in between quoted text. I must get ride of the newline character that comes in between the quoted text. output must be:... (8 Replies)
Discussion started by: ajahuja
8 Replies

7. Shell Programming and Scripting

Removing ^M and the newline that follows it.

Hi Gurus, Apologies as I feel like this must be answered already on here somewhere but I just can't find it. I find many people looking to remove all \n and \r (CR and LF) or one or the other but the only times I've found someone trying to remove them only when both are together they've found... (7 Replies)
Discussion started by: Leedor
7 Replies

8. Shell Programming and Scripting

using awk removing newline and specific position

Hello Friends, Input File looks as follows: >FASTA Header1 line1 line2 line3 linen >FASTA Header2 Line1 Line2 linen >FASTA Header3 and so on ....... Output: Want something as: >FASTA Header1 line1line2line3linen >FASTA Header2 (5 Replies)
Discussion started by: Deep9000
5 Replies

9. Shell Programming and Scripting

Removing Embedded Newline from Delimited File

Hey there - a bit of background on what I'm trying to accomplish, first off. I am trying to load the data from a pipe delimited file into a database. The loading tool that I use cannot handle embedded newline characters within a field, so I need to scrub them out. Solutions that I have tried... (7 Replies)
Discussion started by: bbetteridge
7 Replies

10. Shell Programming and Scripting

sed removing carriage return and newline

Hi, I'm not very familiar with unix shell. I want to replace the combination of two carriage returns and one newline with one carriage return and one newline. I think the best way to do this is to use sed. I tried something like this: sed -e "s#\#\#g" file.txt but it doesn't work. Thanx... (2 Replies)
Discussion started by: mored
2 Replies
Login or Register to Ask a Question