Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replace double quotes with a single quote within a double quoted string

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace double quotes with a single quote within a double quoted string
# 22  
Old 05-05-2014
Quote:
Originally Posted by pchang
Thank you Subdeh - I decided to try your code (since it looks the most simplest) but I didn't get the expected output. Instead of a single quote, I got x27.
Try this instead:
Code:
sed "s/\([^,]\)\"\([^,]\)/\1'\2/g" file

# 23  
Old 05-05-2014
Quote:
Originally Posted by Subbeh
Try this instead:
Code:
sed "s/\([^,]\)\"\([^,]\)/\1'\2/g" file

WOW!!! - thank you.

Your code works great!!! The only thing is that it's only substituting the first occurence of a double quote, not all occurences within the string. See bold/underline data below. Could you enhance your code to replace all occurences of double quotes within the text string?

Input Data:
Code:
0000005335,"IBD","601725","6017257002503849","0430","153854","007907","0079070
00000","E0107725","2995 BL.DAGENAIS "H"   LAVAL        QCCA","0200","","","WD","
CH","","4001857090","","124","124",,60.00,60.00,60.00,0.00,0.45,60.45,0.037500,"
APP","00","EXC","5"

Output Data:
Code:
0000005335,"IBD","601725","6017257002503849","0430","153854","007907","0079070
00000","E0107725","2995 BL.DAGENAIS 'H"   LAVAL        QCCA","0200","","","WD","
CH","","4001857090","","124","124",,60.00,60.00,60.00,0.00,0.45,60.45,0.037500,"
APP","00","EXC","5"

# 24  
Old 05-05-2014
Hi.

In the case "H", the H has already participated in a regular expression match (the trailing [^,], and (AFAIK) the sed RE engine does not allow overlapping REs ... cheers, drl
# 25  
Old 05-05-2014
This could be done by using look-behind and look-ahead regexp.
A python example:
Code:
>>> import re
>>> text = '''0000005335,"IBD","601725","6017257002503849","0430","153854","007907","007907000000","E0107725","2995 BL.DAGENAIS "H"   LAVAL        QCCA","0200","","","WD","CH","","4001857090","","124","124",,60.00,60.00,60.00,0.00,0.45,60.45,0.037500,"APP","00","EXC","5"'''
>>> 
>>> print re.sub(r'(?<![,\A])"(?!(?:(,|\Z)))',"'",text)
0000005335,"IBD","601725","6017257002503849","0430","153854","007907","007907000000","E0107725","2995 BL.DAGENAIS 'H'   LAVAL        QCCA","0200","","","WD","CH","","4001857090","","124","124",,60.00,60.00,60.00,0.00,0.45,60.45,0.037500,"APP","00","EXC","5"

# 26  
Old 05-05-2014
Hi klashxx...

Just a couple of points here.
1) It looks as though you are using Python 2.x, (2.7.x), so your print statement would not
work on 3.x.x. Parentheses are needed to make 'print' a function 'print(your stuff here)'
2) Can you guarantee that all UNIX, Linux, CygWin etc, systems have a python version?

EDIT:-
To the OP is the part above the ^ character an error?
Code:
Last login: Mon May  5 19:09:15 on ttys000
AMIGA:barrywalker~> python
Python 2.7.1 (r271:86832, Aug  5 2011, 03:30:24) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> text='''1234,"","qwerty",," abcd efg "h" bcd"efg
...  dfg",","","5"'''
>>> print text
1234,"","qwerty",," abcd efg "h" bcd"efg
 dfg",","","5"
>>> 
>>> 
>>> print re.sub(r'(?<![,\A])"(?!(?:(,|\Z)))',"'",text)
1234,"","qwerty",," abcd efg 'h' bcd'efg
 dfg",","","5"
>>> # ^ is this an error?
... # Should it be a "'"?
... 
>>> exit()
AMIGA:barrywalker~> _

Last edited by wisecracker; 05-05-2014 at 03:21 PM.. Reason: See above...
# 27  
Old 05-05-2014
Hi pchang,
As has been stated many times, your simple request is ambiguous. With your stated requirements and the 4 line input sample:
Code:
0000005335,"IBD","601725","6017257002503849","0430","153854","007907","0079070
00000","E0107725","2995 BL.DAGENAIS "H"   LAVAL        QCCA","0200","","","WD","
CH","","4001857090","","124","124",,60.00,60.00,60.00,0.00,0.45,60.45,0.037500,"
APP","00","EXC","5"

there are at least 2**46 (i.e., 2 raised to the 46th power) different answers that meet your stated criteria. For example, one possible result that meets all of your stated requirements is:
Code:
0000005335,"IBD','601725','6017257002503849','0430','153854','007907','0079070
00000','E0107725','2995 BL.DAGENAIS "H"   LAVAL        QCCA','0200','','','WD','
CH','','4001857090','','124','124',,60.00,60.00,60.00,0.00,0.45,60.45,0.037500,'
APP','00','EXC','5"

You have to tell us what constitutes a quoted string. In most CSV format files (using comma as the field separator) with quoted strings, comma can appear as a regular character in a quoted string, newline can appear as a regular character in a quoted string, in fact anything except an unescaped quoting character and an unescaped escape character can appear in a quoted string. But you don't have an escape character and you have unescaped quoting characters in your quoted string. So we need unambiguous rules that specify which double-quote characters start a quoted field and which double-quote characters end a quoted field.

For example, if the following rules correctly state your requirements, I can give you an awk script that will do what you want:
  1. The input file is a text file. (By definition this means there are no null bytes in the file, there are no lines longer than LINE_MAX bytes, and (unless the file is an empty file) the last character in the file is a newline character.)
  2. The start of a double-quoted field occurs when the first character of a field is a double-quote character. (This character can be referred to as an opening double-quote.)
  3. The end of a double-quoted field is delimited by a double-quote character that is not an opening double-quote and that is immediately followed by a comma or a newline character. (This character can be referred to as a closing double-quote.)
  4. Fields shall be separated by a comma that is not in a double-quoted field.
  5. Any double-quote character in a double-quoted field other than the opening double-quote and the closing double-quote shall be converted to a single-quote character.
  6. Double-quote characters that are not an opening double-quote, not a closing double-quote, and not in a double-quoted field shall not changed.
  7. A record shall be terminated by a newline character that is not in a double-quoted field.
Do these rules accurately describe your input file format?

If they do, I'll clean up my awk script and post it.

If they don't, give us your set of UNAMBIGUOUS rules and maybe we'll be able to help you.
These 3 Users Gave Thanks to Don Cragun For This Post:
Chubler_XL drl jim mcnamara
# 28  
Old 05-05-2014
Quote:
Originally Posted by pchang
WOW!!! - thank you.

Your code works great!!! The only thing is that it's only substituting the first occurence of a double quote, not all occurences within the string. See bold/underline data below. Could you enhance your code to replace all occurences of double quotes within the text string?
This should fix it:

Code:
sed -e :a -e "s/\([^,]\)\"\([^,]\)/\1'\2/g;ta" file

This User Gave Thanks to Subbeh For This Post:
Chubler_XL
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace double quotes inside the string data for all the columns

Please use code tags Hi, I have input data is below format and n of column in the multiple flat files. the string data has any double quotes(") values replaced to double double quotes for all the columns{""). Also, my input flat file each column string data has carriage of new line too.... (14 Replies)
Discussion started by: SSrini
14 Replies

2. Shell Programming and Scripting

Replacing all but the first and last double quote in a line with a single quote with awk

From: 1,2,3,4,5,This is a test 6,7,8,9,0,"This, is a test" 1,9,2,8,3,"This is a ""test""" 4,7,3,1,8,"""" To: 1,2,3,4,5,This is a test 6,7,8,9,0,"This; is a test" 1,9,2,8,3,"This is a ''test''" 4,7,3,1,8,"''"Is there an easy syntax I'm overlooking? There will always be an odd number... (5 Replies)
Discussion started by: Michael Stora
5 Replies

3. Shell Programming and Scripting

Replace Double quotes within double quotes in a column with space while loading a CSV file

Hi All, I'm unable to load the data using sql loader where there are double quotes within the double quotes As these are optionally enclosed by double quotes. Sample Data : "221100",138.00,"D","0019/1477","44012075","49938","49938/15043000","Television - 22" Refurbished - Airwave","Supply... (6 Replies)
Discussion started by: mlavanya
6 Replies

4. Shell Programming and Scripting

sed command to replace string that contain blackslash,double quotes

Hi All, I have been trying to replace a string using the sed command string value contain blackslash and double quotes. I am not a expert writer of unix script but do try not to ask question. I have almost given up. Hope you all can give me some suggestion I want to replace a place string... (6 Replies)
Discussion started by: thanush9sep
6 Replies

5. Shell Programming and Scripting

Issue with Single Quotes and Double Quotes for prompt PS1

Hi, Trying to change the prompt. I have the following code. export PS1=' <${USER}@`hostname -s`>$ ' The hostname is not displayed <abc@`hostname -s`>$ uname -a AIX xyz 1 6 00F736154C00 <adcwl4h@`hostname -s`>$ If I use double quotes, then the hostname is printed properly but... (3 Replies)
Discussion started by: bobbygsk
3 Replies

6. Shell Programming and Scripting

Replace double double quotes using AWK/SED

Hi, I have data as "01/22/97-"aaaaaaaaaaaaaaaaa""aaa""aabbbbbbbbcccccc""zbcd""dddddddddeeeeeeeeefffffff" I want to remove only the Consequitive double quotes and not the one which occurs single. My O/P must be ... (2 Replies)
Discussion started by: Bhuvaneswari
2 Replies

7. Shell Programming and Scripting

Replace single quote with two single quotes in perl

Hi I want to replace single quote with two single quotes in a perl string. If the string is <It's Simpson's book> It should become <It''s Simpson''s book> (3 Replies)
Discussion started by: DushyantG
3 Replies

8. Shell Programming and Scripting

Regex in grep to match all lines ending with a double quote (") OR a single quote (')

Hi, I've been trying to write a regex to use in egrep (in a shell script) that'll fetch the names of all the files that match a particular pattern. I expect to match the following line in a file: Name = "abc" The regex I'm using to match the same is: egrep -l '(^) *= *" ** *"$' /PATH_TO_SEARCH... (6 Replies)
Discussion started by: NanJ
6 Replies

9. Shell Programming and Scripting

Double quotes or single quotes when using ssh?

I'm not very familiar with the ssh command. When I tried to set a variable and then echo its value on a remote machine via ssh, I found a problem. For example, $ ITSME=itsme $ ssh xxx.xxxx.xxx.xxx "ITSME=itsyou; echo $ITSME" itsme $ ssh xxx.xxxx.xxx.xxx 'ITSME=itsyou; echo $ITSME' itsyou $... (3 Replies)
Discussion started by: password636
3 Replies

10. Shell Programming and Scripting

single or double quote in SED

i m trying the following command but its not working: sed 's/find/\'replace\'/g' myFile but the sed enters into new line # sed 's/find/re\'place/g' myFile > I havn't any idea how to put single quote in my replace string. Your early help woud be appreciated. Thanx (2 Replies)
Discussion started by: asami
2 Replies
Login or Register to Ask a Question