Xml to csv (again)


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Xml to csv (again)
# 1  
Old 05-22-2019
Xml to csv (again)

Hello,
I have copied .xml code for a single item below. I am trying to extract three items (field indices*b244 (second occurrence), b203, and j151), so the desired output would be:

Code:
9780323013543	Manual of Natural Veterinary Medicine: Science and Tradition, 1e	68.95

A parallel solution, based upon a similar question I had made, would be of the following form, but this is not working. A solution of this form would be ideal:

Code:
awk ‘/b244|b203|j151/ {L=/a001/; gsub (/ *<[^>]*> */, _); printf "%s%s", L?TRS:"\t", $0; TRS=ORS} END {printf RS}' file

Here is the .xml data code:

Code:
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE ONIXmessage SYSTEM "http://www.editeur.org/onix/2.1/03/short/onix-international.dtd">
<ONIXmessage>
	<header>
		<m174>Elsevier Health Sciences</m174>
		<m175>Tony Cardinale</m175>
		<m182>20190405</m182>
	</header>
	<product>
		<a001>1084320</a001>
		<a002>03</a002>
		<productidentifier>
			<b221>02</b221>
			<b244>0323013546</b244>
		</productidentifier>
		<productidentifier>
			<b221>03</b221>
			<b244>9780323013543</b244>
		</productidentifier>
		<productidentifier>
			<b221>15</b221>
			<b244>9780323013543</b244>
		</productidentifier>
		<b246>03</b246>
		<b012>BC</b012>
		<title>
			<b202>01</b202>
			<b203>Manual of Natural Veterinary Medicine: Science and Tradition, 1e</b203>
			<b029>Science and Tradition</b029>
		</title>
		<contributor>
			<b035>A01</b035>
			<b039>Susan G.</b039>
			<b040>Wynn</b040>
			<b042>DVM</b042>
			<professionalaffiliation>
				<b046>Wynn Clinic for Therapeutic Alternatives, Marietta, GA</b046>
			</professionalaffiliation>
		</contributor>
		<contributor>
			<b035>A01</b035>
			<b039>Steve</b039>
			<b040>Marsden</b040>
			<b042>DVM, ND, MSOM, LAc, Dipl C.H.</b042>
			<professionalaffiliation>
				<b046>Co-founder, Edmonton Holistic Veterinary Clinic, The Natural Path Clinic, Edmonton, Alberta; Instructor, American Association of Veterinary Acupuncture, International Veterinary Acupuncture Society, and Academy of Veterinary Acupuncturists of Canada</b046>
			</professionalaffiliation>
		</contributor>
		<language>
			<b253>01</b253>
			<b252>eng</b252>
		</language>
		<b061>768</b061>
		<b064>MED089000</b064>
		<subject>
			<b067>24</b067>
			<b171>Landing</b171>
			<b069>5</b069>
			<b070>Veterinary Medicine</b070>
		</subject>
		<subject>
			<b067>24</b067>
			<b171>MSC</b171>
			<b069>079</b069>
			<b070>Veterinary - General</b070>
		</subject>
		<b073>06</b073>
		<othertext>
			<d102>01</d102>
			<d104>This handy reference provides users with an understanding of complementary and alternative treatment options for more than 130 common disease states. A practical manual, it describes a variety of possible approaches to small animal disorders. Concentrating on nutrition, herbs, traditional Chinese medicines, and physical therapies, the authors present both tradition- and evidence-based therapies for disorders not always responsive to conventional therapies. Each monograph-style discussion of natural therapies for disorders common to specific body systems presents therapeutic rationales with the goals of treatment, alternative therapies with conventional bases, paradigmatic options, and authors' suggestions from which they've experienced success. Key references are also included at the conclusion of each chapter.&lt;ul&gt;&lt;li&gt;Presents new and alternative therapies with scientific support, encouraging veterinarians explore new therapies with confidence.&lt;/li&gt;&lt;li&gt;Helps veterinarians develop treatment plans - a vast improvement over large texts that simply introduce the therapies.&lt;/li&gt;&lt;li&gt;Clearly explains esoteric concepts of traditional Chinese medicine in updated language.&lt;/li&gt;&lt;li&gt;Practical, user-friendly pocket manual format allows for quick access in the clinical setting.&lt;/li&gt;&lt;li&gt;Chapters are organized logically by body system and disorders are alphabetized within each chapter.&lt;/li&gt;&lt;li&gt;Each body system chapter includes a case report that describes the history, physical examination, assessment, treatment, and outcome of a specific patient to further illustrate how to develop a treatment plan.&lt;/li&gt;&lt;li&gt;Each appendix offers practical backup for designing treatment plans, from homemade diets and Chinese food therapy to oral herb doses and a valuable herb cross-reference table.&lt;/li&gt;&lt;/ul&gt;</d104>
		</othertext>
		<othertext>
			<d102>04</d102>
			<d104>Part One	Fundamentals of Chinese Medicine&lt;br&gt;1. The meat and potatoes of Chinese medicine: the cooking pot analogy&lt;br&gt;2. Chinese medicine as a basis for an alternative medical approach&lt;br&gt;Part Two Clinical Strategies by Organ System&lt;br&gt;3. Therapies for behavior disorders&lt;br&gt;4. Therapies for cardiovascular disorders&lt;br&gt;5. Therapies for dermatologic disorders&lt;br&gt;6. Therapies for digestive disorders&lt;br&gt;7. Therapies for ear diseases&lt;br&gt;8. Therapies for endocrine disorders&lt;br&gt;9. Therapies for hematologic and immunologic disorders&lt;br&gt;10. Therapies for infectious diseases&lt;br&gt;11. Therapies for liver diseases &lt;br&gt;12. Therapies for musculoskeletal disorders&lt;br&gt;13. Therapies for neoplastic disorders&lt;br&gt;14. Therapies for neurologic disorders&lt;br&gt;15. Therapies for ophthalmologic disorders&lt;br&gt;16. Therapies for respiratory disorders&lt;br&gt;17. Therapies for reproductive disorders&lt;br&gt;18. Therapies for urologic disorders&lt;br&gt;Appendix A 	Guidelines for Homemade Diets&lt;br&gt;Appendix B 	Chinese Food Therapy &lt;br&gt;Appendix C 	Suggested Oral Herb Doses&lt;br&gt;Appendix D 	Chinese Herb Cross Reference Table&lt;br&gt;Appendix E 	Acupuncture Points</d104>
		</othertext>
		<imprint>
			<b241>02</b241>
			<b243>Mosby17</b243>
			<b079>Mosby</b079>
		</imprint>
		<b081>Elsevier Health Sciences</b081>
		<b394>04</b394>
		<b003>20021015</b003>
		<copyrightstatement>
			<b087>2003</b087>
			<copyrightowner>
				<copyrightowneridentifier>
					<b392>02</b392>
					<b244>Elsevier Health Sciences</b244>
				</copyrightowneridentifier>
			</copyrightowner>
		</copyrightstatement>
		<salesrights>
			<b089>01</b089>
			<b388>WORLD</b388>
		</salesrights>
		<measure>
			<c093>01</c093>
			<c094>7.32</c094>
			<c095>in</c095>
		</measure>
		<measure>
			<c093>02</c093>
			<c094>4.84</c094>
			<c095>in</c095>
		</measure>
		<relatedproduct>
			<h208>06</h208>
			<productidentifier>
				<b221>03</b221>
				<b244>9780323070096</b244>
			</productidentifier>
		</relatedproduct>
		<relatedproduct>
			<h208>12</h208>
			<productidentifier>
				<b221>02</b221>
				<b244>0323029981</b244>
			</productidentifier>
			<productidentifier>
				<b221>03</b221>
				<b244>9780323029988</b244>
			</productidentifier>
		</relatedproduct>
		<relatedproduct>
			<h208>12</h208>
			<productidentifier>
				<b221>03</b221>
				<b244>9780702047435</b244>
			</productidentifier>
		</relatedproduct>
		<supplydetail>
			<j137>Elsevier Health Sciences</j137>
			<j397>WORLD</j397>
			<j141>MD</j141>
			<j396>23</j396>
			<j145>20</j145>
			<price>
				<j148>01</j148>
				<discountcoded>
					<j363>02</j363>
					<j378>Elsevier Health Sciences</j378>
					<j364>REF</j364>
				</discountcoded>
				<j151>68.95</j151>
				<j152>USD</j152>
			</price>
		</supplydetail>
	</product>

# 2  
Old 05-22-2019
Hi, I don't understand what is happening, I just tried to correct some mistakes.
Code:
awk '/b244|b203|j151/ {L=/a001/; gsub (/ *<[^>]*> */, _); printf "%s%s", L? rs:"\t", $0; rs=RS } END {printf RS}' file

--- Post updated at 23:22 ---

Code:
awk -F"<[^>]*>" '/b244|b203|j151/ {print $2}' file

--- Post updated at 23:36 ---

Then further just need an explanation on what principle should be selected line with tag <b244>?
Temporarily remove the field separator -F
Code:
awk '/b244|b203|j151/ {print $0}' file.xml
			<b244>0323013546</b244>
			<b244>9780323013543</b244>
			<b244>9780323013543</b244>
			<b203>Manual of Natural Veterinary Medicine: Science and Tradition, 1e</b203>
					<b244>Elsevier Health Sciences</b244>
				<b244>9780323070096</b244>
				<b244>0323029981</b244>
				<b244>9780323029988</b244>
				<b244>9780702047435</b244>
				<j151>68.95</j151>

--- Post updated at 23:56 ---

@palex, sorry, I was not attentive
Code:
awk -F"<[^>]*>" '/b244/ {cn++; if(cn == 2) printf $2}; /b203|j151/ {printf "\t" $2} END {print ""}' file


Last edited by nezabudka; 05-22-2019 at 05:44 PM..
# 3  
Old 05-23-2019
Code:
sed -n '/b244/! b; x;s/$/\n/; /^\n\n$/! {x;b}; :1; ${x;s/[ \t]*<[^>]*>//g;s/\n/\t/gp}; n; /b203\|j151/ H;b1' file

This User Gave Thanks to nezabudka For This Post:
# 4  
Old 05-23-2019
Hi,

here's a solution with xmlstarlet, not shorter but a bit more data-structure-oriented approach, which I find a lot easier to understand:

Code:
xmlstarlet sel -t -m "//product" \ 
    -v "concat(current()/productidentifier[2]/b244,' ',current()/title/b203,' ',current()/supplydetail/price/j151)" -n  data.xml

Output:

Code:
9780323013543 Manual of Natural Veterinary Medicine: Science and Tradition, 1e 68.95

Note: I added the closing XML-Tag "</ONIXmessage>" at the end. Without this my solution won't work. I assume it is there in the original file, even if your sample data does not contain it.

Hmm. How do I get a tab character in the command?

The following works only within bash:
Code:
xmlstarlet sel -t -m "//product"     \ 
   -v "concat(current()/productidentifier[2]/b244,'"$'\t'"',current()/title/b203,'"$'\t'"',current()/supplydetail/price/j151)" -n  data.xml

For a posix shell, I know no other way than this:
Code:
tab="$(echo -ne "\t")"
xmlstarlet sel -t -m "//product"     \ 
   -v "concat(current()/productidentifier[2]/b244,'$tab',current()/title/b203,'$tab',current()/supplydetail/price/j151)" -n  data.xml

As an alternative you can type a literal TAB at the command line. Here at my linux box, I get a literal tab on the command line with pressing Ctrl + V and then TAB.

Last edited by stomp; 05-23-2019 at 04:46 PM..
# 5  
Old 05-23-2019
That is great.... thanks so much! The only issue is that each line of the output file runs together. How can I add a line separator?
# 6  
Old 05-23-2019
The -n should be responsible for inserting newlines.
# 7  
Old 05-23-2019
My apologies, I was referring to the previous solution:

Code:
awk -F"<[^>]*>" '/b244/ {cn++; if(cn == 2) printf $2}; /b203|j151/ {printf "\t" $2} END {print ""}' file

How can I separate the output lines?

Thanks again!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Converting XML to CSV

Hello, For i while i have been using XMLStarlet to convert several XML files to CSV files. So far this always went fine. Today however i got a new XML format however but i cannot find out how to get the data i need. Below is part of the code where it shows the different format. What... (10 Replies)
Discussion started by: SDohmen
10 Replies

2. UNIX and Linux Applications

Xml to csv

Hello, Does anyone know of a way to convert an .xml file (ONIX) to something more workable, like a .csv (or even .xls) file? Ideally something on the command line would be ideal, but not absolutely necessary. I would be dealing with .xml files of 125 MB+. I am using XQuartz in El Capitan. ... (17 Replies)
Discussion started by: palex
17 Replies

3. Shell Programming and Scripting

XML to CSV

I want to pharse below Xml Using Shell Scripting . Thanks in Advance <md> <neid> <neun>1523</neun> <nedn>XXX1212</nedn> <nesw>fffff12515</nesw> </neid> <mi> <mts>20141128001500</mts> <gp>550</gp> <mt>pmct1</mt> <mt>pmNo2</mt> <mt>pmNo3S</mt> <mv> <moid>Ma=1,Rn=1,Ul=311C</moid>... (6 Replies)
Discussion started by: pareshkp
6 Replies

4. Shell Programming and Scripting

How to convert xml to csv ?

I am in need of converting billions of XML into csv file to load data to DB, i have found the below code in perl but not sure why it's not working properly. CODE: #!/usr/bin/perl # Script to illustrate how to parse a simple XML file # and pick out all the values for a specific element, in... (1 Reply)
Discussion started by: rspwilliam
1 Replies

5. Shell Programming and Scripting

Convert xml to csv

I need to convert below xml code to csv. I searched other posts as well but this post (_https://www.unix.com/shell-programming-scripting/174417-extract-parse-xml-data-statistic-value-csv.html) gives "sed command garbled" error. As of now I have written a long script to do it, but can it be done with... (7 Replies)
Discussion started by: dineshydv
7 Replies

6. Shell Programming and Scripting

XML to CSV specific

Hi , Please any one to help on ,extract this xml code into csv columns list. <SOURCEFIELD BUSINESSNAME ="" DATATYPE ="date" DESCRIPTION ="" FIELDNUMBER ="1" FIELDPROPERTY ="0" FIELDTYPE ="ELEMITEM" HIDDEN ="NO" KEYTYPE ="NOT A KEY" LENGTH ="19" LEVEL ="0" NAME ="BUSINESS_DATE"... (4 Replies)
Discussion started by: mohan705
4 Replies

7. Shell Programming and Scripting

XML to csv transformation

Hi, I want to write a perl script. Which should accept the xml file, one xsl file and the loaction. The perl script should process the xml file using the xsl file and puts the out put in specified location. For example: My.perl is perls cript. my.xml is like this <?xml version="1.0"... (2 Replies)
Discussion started by: siba.s.nayak
2 Replies

8. Shell Programming and Scripting

CSV processing to XML

Hi, i am really fresh with shell scripting and programming, i have an issue i am not able to solve to populate data on my server for Cisco IP phones. I have CSV file within the following format: ;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;... (9 Replies)
Discussion started by: angel2008
9 Replies

9. Shell Programming and Scripting

CSV to XML

Iam pretty new to UNIX and would like to convert a CSV to an XML file using AWK scripts. Can anybody suggest a solution? My CSV file looks something like this : Serial No Growth% Annual % Commission % Unemployed % 1 35% 29% 59% 42% 2 61% ... (15 Replies)
Discussion started by: pjanakir
15 Replies

10. Shell Programming and Scripting

Help to convert XML to CSV

Apologies if this has already been covered in this site somewhere, I did try looking but without any success. I am new to the whole XML thing, very late starter, and have a requirement to convert an XML fiule to a CSV fomat. I am crrently working on a Solaris OS. Does anyone have any suggestions,... (2 Replies)
Discussion started by: rossingi_33
2 Replies
Login or Register to Ask a Question