Getting fields from a file having multiple delimiters

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Getting fields from a file having multiple delimiters
# 1  
Old 03-09-2017
Getting fields from a file having multiple delimiters

Hi All,

I have a file with a single row having the following text


I want an output in the following format


Note : the commas should be replaced with spaces, except for the one's between the double quotes. Also, the outer double quotes have to be replaced with a single quote
Lastly, the length of the 'functions'(statements between double quotes) is not fixed and can vary.

Looking forward for your expert advise. Thanks in Advance.Smilie

# 2  
Old 03-09-2017
Assuming that your input sample is representative:
awk -F, -v sq="'" '{match($0, /".*"/); print $1, $4, $6, sq substr($0, RSTART+1, RLENGTH-2) sq, $(NF-1)}' file

seems to do what you want.

If you are running this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 03-09-2017
Thanks Don, seems to work.Smilie

If you could please help me with the explanation as well.

# 4  
Old 03-09-2017
Hi dev.devil.1983,
Here is a copy of the awk script with comments...
# Invoke the awk utility with the input field separator set to a comma (-F,)
# and a variable named sq that is set to a string containing a single-quote
# character (-v sq="'").
awk -F, -v sq="'" '
{	# For each line read from the input file...
	# Search for the longest string in the current input line that starts
	# and ends with a double-quote character.  If a match is found, set
	# RSTART to the index of the 1st double-quote character and set RLENGTH
	# to the number of characters matched:
	match($0, /".*"/)
	# Print the:
	#	1st field from the input line ($1),
	#	print the output field separator (,), (Note that the default
	#	   OFS is a <space> character and the default is not overridden
	#	   in this script.)
	#	2nd field from the input line ($2),
	#	print the output field separator (,),
	#	4th field from the input line ($4),
	#	print the output field separator (,),
	#	6th field from the input line ($6),
	#	print the output field separator (,),
	#	print a single-quote character (sq),
	#	print the substring of the input line starting after the 1st
	#	   double-quote character up to, but not including, the last
	#	   double-quote character (substr(...)),
	#	print a single-quote character (sq),
	#	print the output field separator (,),
	#	print the next to the last field from the input line ($(NF-1)),
	#	and print the output record separator.  (Note that the default
	#	  ORS is a <newline> character and the default is not overridden
	#	  in this script.)
	print $1, $4, $6, sq substr($0, RSTART + 1, RLENGTH - 2) sq, $(NF - 1)
}' file	# Terminate the awk script and name the input file(s) to be processed.

Does this help?
This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell Script to Group by Based on Multiple Fields in a file

Hi, I want to know if there is any simple approach to SUM a field based on group by of different fields for e.g. file1.txt contains below data 20160622|XXX1||50.00||50.00|MONEY|Plan1| 20160622|XXX1||100.00||100.00|MONEY|Plan1| 20160623|XXX1||25.00||25.00|MONEY|Plan1|... (3 Replies)
Discussion started by: cnu_theprince
3 Replies

2. Shell Programming and Scripting

Cut counting consecutive delimiters as fields

When cut encounters consecutive delimiters it seems to count each instance as a field, at least with spaces. Is this typical behavior for any delimiter? #:~$ ifconfig eth0 | grep HWaddr eth0 Link encap:Ethernet HWaddr 94:de:80:a7:6d:e1 #:~$ ifconfig eth0 | grep HWaddr | cut -d " " -f... (6 Replies)
Discussion started by: Riker1204
6 Replies

3. UNIX for Dummies Questions & Answers

Cut fields between delimiters

I'm having bother getting both lines contained in a file to output as the same value. A simple example: john:123456:123:456:doe john:123456:123:doe cut -d: -f1,4 input file john:456 john:doe ^ first line should be same as second. trick one for me, i know why it's because of the... (4 Replies)
Discussion started by: landofus
4 Replies

4. Shell Programming and Scripting

awk multiple delimiters

Hi Folks, This is the first time I ever encountered this situation My input file is of this kind cat input.txt 1 PAIXAF 0 1 1 -9 0 0 0 1 2 0 2 1 2 1 7 PAIXEM 0 7 1 -9 1 0 2 0 1 2 2 1 0 2 9 PAKZXY 0 2 1 -9 2 0 1 1 1 0 1 2 0 1 Till the sixth column (which is -9), I want my columns to... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

5. Shell Programming and Scripting

awk: Print fields between two delimiters on separate lines and send to variables

I have email headers that look like the following. In the end I would like to accomplish sending each email address to its own variable, such as:'''''' etc... I know the sed to get rid of... (11 Replies)
Discussion started by: tay9000
11 Replies

6. Shell Programming and Scripting

Inserting additional comma delimiters in a csv file, after and before certian fields.

Hello I have a csv file which I need to insert addtional commas into. The csv is of the format field1,field2,field3,field4,...etc...,field13,field14 I need to add extra commas in each record so that the final output looks like ... (1 Reply)
Discussion started by: kamal_p_99
1 Replies

7. Shell Programming and Scripting

Sorting problem: Multiple delimiters, multiple keys

Hello If you wanted to sort a .csv file that was filled with lines like this: <Ticker>,<Date as YYYYMMDD>,<Time as H:M:S>,<Volume>,<Corr> (H : , M, S: ) by date, does anybody know of a better solution than to turn the 3rd and 4th colons of every line into commas, sorting on four keys,... (20 Replies)
Discussion started by: Ryan.
20 Replies

8. Shell Programming and Scripting

AWK with multiple delimiters

I have the following string sample: bla bla bla bla bla I would like to extract the "123" using awk. I thought about awk -F"]" '{ print $1 }' but it doesn't work Any ideas ? (7 Replies)
Discussion started by: gdub
7 Replies

9. Shell Programming and Scripting

Cutting a file with multiple delimiters into columns

Hi All I have recently had to start using Unix for work and I have hit brick wall with this prob.... I have a file that goes a little something like this.... EUR;EUR;EUR:USD:USD;USD;;;EUR/USD;XAU/AUD;XAU/EUR;XAU/AUD,GBP/BOB,UAD/XAU;;;1.11;2.22;3.33;4.44;5.55;6.66;;; is it possible to... (7 Replies)
Discussion started by: luckycharm
7 Replies

10. Shell Programming and Scripting

join on a file with multiple lines, fields

I've looked at the join command which is able to perform what I need on two rows with a common field, however if I have more than two rows I need to join all of them. Thus I have one file with multiple rows to be joined on an index number: 1 randomtext1 2 rtext2 2 rtext3 3 rtext4 3 rtext5... (5 Replies)
Discussion started by: crimper
5 Replies
Login or Register to Ask a Question

Featured Tech Videos