Removal of multiple characters with in double quotes Post: 303011097

Sponsored Content

Top Forums Shell Programming and Scripting Removal of multiple characters with in double quotes Post 303011097 by Don Cragun on Friday 12th of January 2018 10:08:41 PM

01-12-2018

Registered User

With the sample input shown in post #9 stored in a file named file.csv, the following code:

Code:

awk -F'\n' -v dq='"' '
{	record = record $0
	#printf("record in:\n%s\n", record)
}
(n = split(record, f, dq)) % 2 {
	#printf("split into %d fields\n", n)
	for(i = 2; i <= n; i += 2) {
		gsub(/[|]/, "", f[i])
		#printf("f[%d] updated to: \"%s\"\n", i, f[i])
	}
	for(i = 1; i <= n; i++)
		printf("%s%s", f[i], (i == n) ? ORS : "")
	record = ""
}' file.csv

produces the output requested in post #9:

Code:

111|IKJA - SPORTS|00IIQ|Normal|100 Hall Road|
123|ABCD RENT-A-CAR XYZ LTD|00N0H|Enterprise Lake|100 View Way|
244|DEFG Travel  TourWorld LTD|AK0Q|Praire Lake|105 NE Main St|

But note that it removes <newline> and <vertical-bar> characters found between pairs of <double-quote> characters; it does NOT replace them with <NUL> characters. Replacing those characters with <NUL> characters would give you a binary file instead of a text file.

If someone else wants to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.

If you uncomment the commented out printf() statements, you can get an inside view at how it accumulates records and removes unwanted <vertical-bar>s and <newline>s.

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace multiple blanks within double quotes

I have various column names within double quotes, separated by commas. Example: "column one", "column number two", "this is column number three", anothercolumn, yetanothercolumn I need to eliminate the double quotes and replace the blanks within the double quotes by underscores, giving: ...

2. Shell Programming and Scripting

Removal of comma(,) present inbetween double quotes(" ")

Hi Experts, I have a file with some of the records contain double quotes. If I found a double quote(") in any particular record , I need to look for the next double quote in that particular record and in between these quotes, if any comma(,) is there I need to replace with Tilde (~) in the same...

3. Shell Programming and Scripting

Removal of new line character in double quotes

Hi, Could you please help me in removal of newline chracter present in between the double quotes and replacing it with space. For example ... Every field is wrapped with double quotes with comma delimiter, so I need to travese from first double quote occerence to till second double...

4. UNIX for Dummies Questions & Answers

grep single quotes or double quotes

Unix superusers, I am new to unix but would like to learn more about grep. I am very familiar with regular expressions as i have used them for searching text files in windows based text editors. Since I am not very familiar with Unix, I dont understand when one should use GREP with the...

5. UNIX for Dummies Questions & Answers

how to use grep: finding a string with double quotes and multiple digits

I have a file with a lot of lines (a lot!) that contain 10 digits between double quotes. ie "1726937489". The digits are random throughout, but always contain ten digits. I can not for the life of me, (via scouring the internet and grep how-to manuals) figure out how to find this when I search....

6. Shell Programming and Scripting

Multiple double quotes

hi Need to run below command on remote server: cmd -a "1 2" -b 3 If i run below, there's clash matching double quotes and fail. ssh $server "cmd -a "1 2" -b 3" I have few ideas which worked (like keeping the entire cmd in a file and copy it to remote server and then run that file)...

7. Shell Programming and Scripting

Issue with Single Quotes and Double Quotes for prompt PS1

Hi, Trying to change the prompt. I have the following code. export PS1=' <${USER}@`hostname -s`>$ ' The hostname is not displayed <abc@`hostname -s`>$ uname -a AIX xyz 1 6 00F736154C00 <adcwl4h@`hostname -s`>$ If I use double quotes, then the hostname is printed properly but...

8. Shell Programming and Scripting

Replace Double quotes within double quotes in a column with space while loading a CSV file

Hi All, I'm unable to load the data using sql loader where there are double quotes within the double quotes As these are optionally enclosed by double quotes. Sample Data : "221100",138.00,"D","0019/1477","44012075","49938","49938/15043000","Television - 22" Refurbished - Airwave","Supply...

9. Shell Programming and Scripting

Extract multiple columns base on double quotes as delimiter

Hi All, I have my data like below "1","abc,db","hac,aron","4","5" Now I need to extract 1,2,4th columns Output should be like "1",abc,db","4" Am trying to use cut command but not able to get the results. Thanks in advance.

10. Shell Programming and Scripting

Removal of comma within double quotes

Hi All, I am getting .csv file whenever there is a comma present between a field that field get enclosed with double quotes For eg as below abc,123,xxyy,2178 fgh,123,"x,x"yy",2178 ghi,123,"x,xyy",2178 jkl,123,xx"yy,2178 whereas I want my data as per below abc,123,xxyy,2178...

LEARN ABOUT MOJAVE

cap_mkdb

CAP_MKDB(1)						    BSD General Commands Manual 					       CAP_MKDB(1)

NAME

     cap_mkdb -- create capability database

SYNOPSIS

     cap_mkdb [-v] [-f outfile] file1 [file2 ...]

DESCRIPTION

     Cap_mkdb builds a hashed database out of the getcap(3) logical database constructed by the concatenation of the specified files .

     The database is named by the basename of the first file argument and the string ``.db''.  The getcap(3) routines can access the database in
     this form much more quickly than they can the original text file(s).

     The ``tc'' capabilities of the records are expanded before the record is stored into the database.

     The options as as follows:

     -f outfile
	     Specify a different database basename.

     -v      Print out the number of capability records in the database.

FORMAT

     Each record is stored in the database using two different types of keys.

     The first type is a key which consists of the first capability of the record (not including the trailing colon (``:'')) with a data field
     consisting of a special byte followed by the rest of the record.  The special byte is either a 0 or 1, where a 0 means that the record is
     okay, and a 1 means that there was a ``tc'' capability in the record that couldn't be expanded.

     The second type is a key which consists of one of the names from the first capability of the record with a data field consisting a special
     byte followed by the the first capability of the record.  The special byte is a 2.

     In normal operation names are looked up in the database, resulting in a key/data pair of the second type.	The data field of this key/data
     pair is used to look up a key/data pair of the first type which has the real data associated with the name.

RETURN VALUE

     The cap_mkdb utility exits 0 on success and >0 if an error occurs.

SEE ALSO

     dbopen(3), getcap(3), termcap(5)

BSD
								   May 31, 2019 							       BSD