Sponsored Content
Top Forums Shell Programming and Scripting remove duplicated xml record in a file under unix Post 302089631 by happyv on Wednesday 20th of September 2006 06:34:51 AM
Old 09-20-2006
Is the Perl can run under ksh Unix?

Also, the record is a bit difference...it look like

record1:
this is testing
my id is 2001
end:
record2:
this is testing2
my id is 2002
end:
record3:
this is testing
my id is 2002
end:
record4:
this is testing2
my id is 2002
end:

For the above, record 2 and 4 is duplicated. Because of the "id" and "testing2" is the same. if only one line is the same which is not called duplicated..

So Perl or any friend can help for the script?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove duplicated columns

hi all, i have a file contain multicolumns, this file is sorted by col2 and col3. i want to remove the duplicated columns if the col2 and col3 are the same in another line. example fileA AA BB CC DD CC XX CC DD BB CC ZZ FF DD FF HH HH the output is AA BB CC DD BB CC ZZ FF... (6 Replies)
Discussion started by: kamel.seg
6 Replies

2. UNIX for Dummies Questions & Answers

how to read record by record from a file in unix

Hi guys, i have a big file with the following format.This includes header(H),detail(D) and trailer(T) information in the file.My problem is i have to search for the character "6h" at 14 th and 15 th position in all the records .if it is there i have to write all those records into a... (1 Reply)
Discussion started by: raoscb
1 Replies

3. UNIX for Advanced & Expert Users

How to read an Xml record contained in a file--urgent

Hi I have an xml file which has multiple xml records.. I don't know how to read those records and pipe them to another shell command the file is like <abc>z<def>y<ghi>x........</ghi></def></abc> (1st record) <jkl>z<mno>y<pqr>x........</pqr></mno></jkl> (2nd record) Each record end... (4 Replies)
Discussion started by: aixjadoo
4 Replies

4. Shell Programming and Scripting

How to remove xml namespace from xml file using shell script?

I have an xml file: <AutoData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Table1> <Data1 10 </Data1> <Data2 20 </Data2> <Data3 40 </Data3> <Table1> </AutoData> and I have to remove the portion xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" only. I tried using sed... (10 Replies)
Discussion started by: Gary1978
10 Replies

5. Shell Programming and Scripting

Help with remove duplicated content

Input file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hcmv-UL70-3p hsa-4486 hcms-US25 hsa-360-5 hcms-US25 hsa-4 hcms-US25 hsa-458 hcms-US25 hsa-44812 . . Desired Output file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hsa-4486... (3 Replies)
Discussion started by: perl_beginner
3 Replies

6. UNIX for Dummies Questions & Answers

Delete a record in a xml file using shell scripting

find pattern, delete line with pattern and 3 lines above and 8 lines below the pattern. The pattern is "isup". The entire record with starting tag <record> and ending tag </record> containing the pattern is to be deleted and the rest to be retained. <record> ... (4 Replies)
Discussion started by: sdesstp
4 Replies

7. Shell Programming and Scripting

How to remove duplicated lines?

Hi, if i have a file like this: Query=1 a a b c c c d Query=2 b b b c c e . . . (7 Replies)
Discussion started by: the_simpsons
7 Replies

8. Shell Programming and Scripting

Extract timestamp from first record in xml file and it checks if not it will replace first record

I have test.xml <emp><id>101</id><name>AAA</name><date>06/06/14 1811</date></emp> <Join><id>101</id><city>london</city><date>06/06/14 2011</date></join> <Join><id>101</id><city>new york</city><date>06/06/14 1811</date></join> <Join><id>101</id><city>sydney</city><date>06/06/14... (2 Replies)
Discussion started by: vsraju
2 Replies

9. Shell Programming and Scripting

How to remove duplicated column in a text file?

Dear all, How can I remove duplicated column in a text file? Input: LG10_PM_map_19_LEnd 1000560 G AA AA AA AA AA GG LG10_PM_map_19_LEnd 1005621 G GG GG GG AA AA GG LG10_PM_map_19_LEnd 1011214 A AA AA AA AA GG GG LG10_PM_map_19_LEnd 1011673 T TT TT TT TT CC CC... (1 Reply)
Discussion started by: huiyee1
1 Replies

10. Shell Programming and Scripting

Remove duplicated records and update last line record counts

Hi Gurus, I need to remove duplicate line in file and update TRAILER (last line) record count. the file is comma delimited, field 2 is key to identify duplicated record. I can use below command to remove duplicated. but don't know how to replace last line 2nd field to new count. awk -F","... (11 Replies)
Discussion started by: green_k
11 Replies
srec_motorola(5)						File Formats Manual						  srec_motorola(5)

NAME
srec_motorola - Motorola S-Record hexadecimal file format DESCRIPTION
This format is also known as the Exorciser, Exormacs or Exormax format. Motorola's S-record format allows binary files to be uploaded and downloaded between two computer systems. This type of format is widely used when transferring programs and data between a computer system (such as a PC, Macintosh, or workstation) and an emulator or evaluation board for Motorola microcontrollers and microprocessors. The Lines Most S-Record file contain only S-Record lines (see the next section), which always start with a capital S character. Some systems gener- ate various "extensions[rq] which usually manifest as lines which start with something else. These "extension[rq] lines may or may not break other systems made by other vendors. Caveat emptor. The Fields The S-Record format consists of 5 fields. These are the type field, length field, address field, data field, and the checksum. The lines always start with a capital S character. +--+------+---------------+---------+------+----------+ |S | Type | Record Length | Address | Data | Checksum | +--+------+---------------+---------+------+----------+ Type The type field is a 1 character field that specifies whether the record is an S0, S1, S2, S3, S5, S6, S7, S8 or S9 field. Record Length The record length field is a 2 character (1 byte) field that specifies the number of character pairs (bytes) in the record, exclud- ing the type and record length fields. Address This is a 2-, 3- or 4-byte address that specifies where the data in the S-Record is to be loaded into memory. Data The data field contains the executable code, memory-loadable data or descriptive information to be transferred. Checksum The checksum is an 8-bit field that represents the least significant byte of the one's complement of the sum of the values repre- sented by the pairs of characters making up the record's length, address, and data fields. Record Types S0 This type of record is the header record for each block of S-Records. The data field may contain any descriptive information iden- tifying the following block of S-Records. (It is commonly "HDR[rq] on many systems.) The address field is normally zero. S1 A record containing data and the 2-byte address at which the data is to reside. S2 A record containing data and the 3-byte address at which the data is to reside. S3 A record containing data and the 4-byte address at which the data is to reside. S5 An optional record containing the number of S1, S2 and S3 records transmitted in a particular block. The count appears in the two- byte address field. There is no data field. This record is optional, you do not have to use it. Nobody knows if you can have more than one in a file; and if you do, nobody knows whether or not the line count resets after each one. The srec_cat command will only ever use one, provided the number of lines fits in 16 bits, otherwise it will use S6. S6 An optional record containing the number of S1, S2 and S3 records transmitted in a particular block. The count appears in the three-byte address field. There is no data field. This record is optional, you do not have to use it. Nobody knows if you can have more than one in a file; and if you do, nobody knows whether or not the line count resets after each one. Nobody knows what happens if you mix S5 and S6 records in a file. The srec_cat command will only ever use one, provided the number of lines fits in 24 bits. S7 A termination record for a block of S3 records. The address field may contain the 4-byte address of the instruction to which control is passed. There is no data field. S8 A termination record for a block of S2 records. The address field may optionally contain the 3-byte address of the instruction to which control is passed. There is no data field. S9 A termination record for a block of S1 records. The address field may optionally contain the 2-byte address of the instruction to which control is passed. If not specified, the first entry point specification encountered in the object module input will be used. There is no data field. Size Multiplier In general, binary data will expand in sized by approximately 2.4 times when represented with this format. EXAMPLE
Here is an example S-Record file. It contains the data "Hello, World[rq] to be loaded at address 0. S00600004844521B S110000048656C6C6F2C20576F726C640A9D S5030001FB S9030000FC COPYRIGHT
srec_cat version 1.58 Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Peter Miller The srec_cat program comes with ABSOLUTELY NO WARRANTY; for details use the 'srec_cat -VERSion License' command. This is free software and you are welcome to redistribute it under certain conditions; for details use the 'srec_cat -VERSion License' command. AUTHOR
Peter Miller E-Mail: pmiller@opensource.org.au //* WWW: http://miller.emu.id.au/pmiller/ Reference Manual SRecord srec_motorola(5)
All times are GMT -4. The time now is 02:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy