Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Duplicates in an XML file
Top Forums Shell Programming and Scripting Duplicates in an XML file Post 302551686 by TasosARISFC on Wednesday 31st of August 2011 09:57:51 AM
Old 08-31-2011
Duplicates in an XML file
Hi All,

I have an xml file that contains information like this

Code:
 
<ID>574922<COMMENT>TEXT
TEXT
TEXT</COMMENT></ID>
<ID>574922<COMMENT>TEXT
TEXT
TEXT</COMMENT></ID>
<ID>412659<COMMENT>TEXT
TEXT
TEXT
TEXT
TEXT</COMMENT></ID>
<ID>873520<COMMENT>TEXT</COMMENT></ID>
<ID>480622<COMMENT>TEXT</COMMENT></ID>
<ID>873520<COMMENT>TEXT
TEXT</COMMENT></ID>
<ID>480622<COMMENT>TEXT</COMMENT></ID>

I want to remove duplicate entries, the problem is I cannot sort, as due to comment stracture for some entries, they form a new line.

I tried the following code
(called as awk -f script file)
Code:
/^end:/ {   if (! (Record in Records)) {      
Records[Record];      
print RecordLabel ":";      
print Record;      
print $0;        
Record = "";   
}   
next;}$1 ~ /^.*:/ {   sub(/:.*/, "", $1);   
RecordLabel = $1;   
next;}{   Record = (Record ? Record "\n" : "") $0;}

provided by Aigles in another post, which for some reason does not work (modified to my input), it does not change anything. I tried to variate it unsuccsfully so far.

any ideas would be much appreciated

many thanks
Last edited by TasosARISFC; 09-08-2011 at 05:34 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to parse a XML file using PERL and XML::DOm

I need to know the way. I have got parsing down some nodes. But I was unable to get the child node perfectly. If you have code please send it. It will be very useful for me. (0 Replies)
Discussion started by: girigopal
0 Replies

2. Shell Programming and Scripting

How to remove xml namespace from xml file using shell script?

I have an xml file: <AutoData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Table1> <Data1 10 </Data1> <Data2 20 </Data2> <Data3 40 </Data3> <Table1> </AutoData> and I have to remove the portion xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" only. I tried using sed... (10 Replies)
Discussion started by: Gary1978
10 Replies

3. Shell Programming and Scripting

How to add the multiple lines of xml tags before a particular xml tag in a file

Hi All, I'm stuck with adding multiple lines(irrespective of line number) to a file before a particular xml tag. Please help me. <A>testing_Location</A> <value>LA</value> <zone>US</zone> <B>Region</B> <value>Russia</value> <zone>Washington</zone> <C>Country</C>... (0 Replies)
Discussion started by: mjavalkar
0 Replies

4. Shell Programming and Scripting

Help required in Splitting a xml file into multiple and appending it in another .xml file

HI All, I have to split a xml file into multiple xml files and append it in another .xml file. for example below is a sample xml and using shell script i have to split it into three xml files and append all the three xmls in a .xml file. Can some one help plz. eg: <?xml version="1.0"?>... (4 Replies)
Discussion started by: ganesan kulasek
4 Replies

5. Shell Programming and Scripting

XML: parsing of the Google contacts XML file

I am trying to parse the XML Google contact file using tools like xmllint and I even dived into the XSL Style Sheets using xsltproc but I get nowhere. I can not supply any sample file as it contains private data but you can download your own contacts using this script: #!/bin/sh # imports... (9 Replies)
Discussion started by: ripat
9 Replies

6. Shell Programming and Scripting

Comparing delta values of one xml file in other xml file

Hi All, I have two xml files. One is having below input <NameValuePair> <name>Daemon</name> <value>tcp:7474</value> </NameValuePair> <NameValuePair> <name>Network</name> <value></value> </NameValuePair> ... (2 Replies)
Discussion started by: sharsour
2 Replies

7. Shell Programming and Scripting

Split xml file into multiple xml based on letterID

Hi All, We need to split a large xml into multiple valid xml with same header(2lines) and footer(last line) for N number of letterId. In the example below we have first 2 lines as header and last line as footer.(They need to be in each split xml file) Header: <?xml version="1.0"... (5 Replies)
Discussion started by: vx04
5 Replies

8. Shell Programming and Scripting

Splitting a single xml file into multiple xml files

Hi, I'm having a xml file with multiple xml header. so i want to split the file into multiple files. Sample.xml consists multiple headers so how can we split these multiple headers into multiple files in unix. eg : <?xml version="1.0" encoding="UTF-8"?> <ml:individual... (3 Replies)
Discussion started by: Narendra921631
3 Replies

9. UNIX for Beginners Questions & Answers

Grepping multiple XML tag results from XML file.

I want to write a one line script that outputs the result of multiple xml tags from a XML file. For example I have a XML file which has below XML tags in the file: <EMAIL>***</EMAIL> <CUSTOMER_ID>****</CUSTOMER_ID> <BRANDID>***</BRANDID> Now I want to grep the values of all these specified... (1 Reply)
Discussion started by: shubh752
1 Replies

10. UNIX for Beginners Questions & Answers

How to pull multiple XML tags from the same XML file in Shell.?

I'm searching for the names of a TV show in the XML file I've attached at the end of this post. What I'm trying to do now is pull out/list the data from each of the <SeriesName> tags throughout the document. Currently, I'm only able to get data the first instance of that XML field using the... (9 Replies)
Discussion started by: hungryd
9 Replies

LEARN ABOUT SUSE

create_text_search_template

CREATE TEXT SEARCH 
TEMPLATE(7)					   SQL Commands 				    CREATE TEXT SEARCH TEMPLATE(7)

NAME

       CREATE TEXT SEARCH TEMPLATE - define a new text search template

SYNOPSIS

       CREATE TEXT SEARCH TEMPLATE name (
	   [ INIT = init_function , ]
	   LEXIZE = lexize_function
       )

DESCRIPTION

       CREATE  TEXT SEARCH TEMPLATE creates a new text search template. Text search templates define the functions that implement text search dic-
       tionaries. A template is not useful by itself, but must be instantiated as a dictionary to be  used.  The  dictionary  typically  specifies
       parameters to be given to the template functions.

       If a schema name is given then the text search template is created in the specified schema. Otherwise it is created in the current schema.

       You  must  be a superuser to use CREATE TEXT SEARCH TEMPLATE. This restriction is made because an erroneous text search template definition
       could confuse or even crash the server.	The reason for separating  templates  from  dictionaries  is  that  a  template  encapsulates  the
       ``unsafe''  aspects of defining a dictionary.  The parameters that can be set when defining a dictionary are safe for unprivileged users to
       set, and so creating a dictionary need not be a privileged operation.

       Refer to in the documentation for further information.

PARAMETERS

       name   The name of the text search template to be created. The name can be schema-qualified.

       init_function
	      The name of the init function for the template.

       lexize_function
	      The name of the lexize function for the template.

       The function names can be schema-qualified if necessary. Argument types are not given, since the argument list for each type of function is
       predetermined. The lexize function is required, but the init function is optional.

       The arguments can appear in any order, not only the one shown above.

COMPATIBILITY

       There is no CREATE TEXT SEARCH TEMPLATE statement in the SQL standard.

SEE ALSO

       ALTER TEXT SEARCH TEMPLATE [alter_text_search_template(7)], DROP TEXT SEARCH TEMPLATE [drop_text_search_template(7)]

SQL - Language Statements					    2010-05-14					    CREATE TEXT SEARCH TEMPLATE(7)
All times are GMT -4. The time now is 11:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy