Sponsored Content
Top Forums Shell Programming and Scripting Merge files and remove duplicated rows Post 302915690 by gull04 on Thursday 4th of September 2014 09:31:20 AM
Old 09-04-2014
Hi,

The simple way would be to build a header file as required, strip the individual headers from your files and output the data to a second file using "sort" append the data to the header file that you created.

Regards

Dave
This User Gave Thanks to gull04 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove duplicated lines without sort

Hi Just wondering whether or not I can remove duplicated lines without sort For example, I use the command who, which shows users who are logging on. In some cases, it shows duplicated lines of users who are logging on more than one terminal. Normally, I would do who | cut -d" " -f1 |... (6 Replies)
Discussion started by: lalelle
6 Replies

2. Shell Programming and Scripting

remove duplicated columns

hi all, i have a file contain multicolumns, this file is sorted by col2 and col3. i want to remove the duplicated columns if the col2 and col3 are the same in another line. example fileA AA BB CC DD CC XX CC DD BB CC ZZ FF DD FF HH HH the output is AA BB CC DD BB CC ZZ FF... (6 Replies)
Discussion started by: kamel.seg
6 Replies

3. Shell Programming and Scripting

Help with remove duplicated content

Input file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hcmv-UL70-3p hsa-4486 hcms-US25 hsa-360-5 hcms-US25 hsa-4 hcms-US25 hsa-458 hcms-US25 hsa-44812 . . Desired Output file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hsa-4486... (3 Replies)
Discussion started by: perl_beginner
3 Replies

4. Shell Programming and Scripting

Remove rows with first 4 fields duplicated in awk

Hi, I am trying to use awk to remove all rows where the first 4 fields are duplicates. e.g. in the following data lines 6-9 would be removed, leaving one copy of the duplicated row (row 5) Borgarhraun FH9822 ol24 FH9822_ol24_m20 ol Deformed c Borgarhraun FH9822 ol24 ... (3 Replies)
Discussion started by: tomahawk
3 Replies

5. UNIX for Dummies Questions & Answers

Merge two files with common IDs but unequal number of rows

Hi, I have two files that I would like to merge and think that there should be a solution using awk. The files look something like this: file 1 IDX1 IDY1 IDX2 IDY2 IDX3 IDY3 file 2 IDY1 dataA data1 IDY2 dataB data2 IDY3 dataC data3 Desired output IDX1 IDY1 dataA data1 IDX2 ... (5 Replies)
Discussion started by: katie8856
5 Replies

6. UNIX for Dummies Questions & Answers

Merge/Remove 2 files

Hi, I would like to get some advice here. Basically I have 2 diff files, I need to add/comment FileA (old) contents as compared with FileB (new). The file has almost 100k lines so it would be great if someone can advise me on the easiest way out. For example FileB has this parameter but... (1 Reply)
Discussion started by: tech.dummy
1 Replies

7. Shell Programming and Scripting

How to remove duplicated lines?

Hi, if i have a file like this: Query=1 a a b c c c d Query=2 b b b c c e . . . (7 Replies)
Discussion started by: the_simpsons
7 Replies

8. Shell Programming and Scripting

Merge files based on both common and uncommon rows

Hi, I have two files A (2190 rows) and file B (1100 rows). I want to merge the contents of two files based on common field, also I need the unmatched rows from file A file A: ABC XYZ PQR file B: >LMN|chr1:11000-12456: >ABC|chr15:176578-187678: >PQR|chr3:14567-15866: output... (3 Replies)
Discussion started by: Diya123
3 Replies

9. Shell Programming and Scripting

Removing duplicated first field rows

Hello, I am trying to eliminate rows where the first field is duplicated, leaving the row where the last field is "NET". Data file: 345234|22.34|LST 546543|55.33|LST 793929|98.23|LST 793929|64.69|NET 149593|49.22|LST Desired output: 345234|22.34|LST 546543|55.33|LST... (2 Replies)
Discussion started by: palex
2 Replies

10. AIX

Remove duplicated bootlist entries

Hello. I have a server with 2 boot disk but in the bootlist there are 5 paths of one disk but no path of the other. How can I remove paths from one disk to insert paths from the other disk? Thanks in advance. server074:root:/# bootlist -om normal hdisk0 blv=hd5 pathid=0 hdisk0... (7 Replies)
Discussion started by: Gabriander
7 Replies
inet6_rth_space(3N)													       inet6_rth_space(3N)

NAME
inet6_rth_add(), inet6_rth_getaddr(), inet6_rth_init(), inet6_rth_reverse(), inet6_rth_segments(), inet6_rth_space() - IPv6 Routing header options manipulation functions. SYNOPSIS
DESCRIPTION
These functions can be used by an application to build and examine an IPv6 Routing header. The Routing header can be used by an IPv6 source to list one or more intermediate nodes to be visited on the way to a packet's destination. These three functions build a Routing header: returns the number of bytes required for a Routing header. initializes the buffer data for a Routing header. adds one IPv6 address to the Routing header. Three functions deal with a returned Routing header: reverses a Routing header. returns the number of segments in a Routing header. fetches one address from a Routing header. These functions are described below: This function returns the number of bytes required to hold a routing header of the specified type containing the specified number of segments (addresses). For an IPv6 Type 0 Routing header, the number of segments must be between 0 and 127, inclusive. The return value is just the space for the Routing header. When the application uses ancillary data, it must pass the returned length to to determine how much memory is needed for the ancillary data object (includ- ing the structure). If the return value is 0, then either the type of the Routing header is not supported by this implementation or the number of segments is invalid for this type of Routing header. This function returns the size but does not allocate the space required for the ancillary data. Note: If type is 2 (Routing Header Type 2), segments must be 1 (RFC 4584). This function initializes the buffer pointed to by bp to contain a Routing header of the specified type. bp_len is only used to verify if the buffer is large enough. The caller must allocate the buffer, and its size can be determined by calling Upon success, the return value is the pointer to the buffer (bp), and the pointer is then used as the first argument to the function. Upon an error, the return value is NULL. Note: If type is 2 (Routing Header Type 2), segments must be 1 (RFC 4584). This function adds the IPv6 address pointed to by addr to the end of the Routing header being constructed. If successful, the member of the Routing header is updated to account for the new address in the Routing header and the return value of the function is 0. Upon an error the return value of the function is -1. Note: If the Routing header is type 2, this function can only be called once. This function takes a Routing header extension header pointed to by the first argument in and writes a new Routing header. The new Routing header sends datagrams along the reverse of that route. The function reverses the order of the addresses and sets the member in the new routing header to the number of segments. Both arguments are allowed to point to the same buffer (that is, the reversal can occur in place). The return value of the function is 0 on success, or -1 upon an error. Note: If Routing header is type 2, this function returns -1 (RFC 4584). This function returns the number of segments (addresses) contained in the Routing header described by bp which can be 0 or greater. The return value of the function is -1 upon an error. This function returns a pointer to the IPv6 address specified by index (which must be a value between 0 and one less than the value returned by in the Routing header described by bp. An application should first call to obtain the number of segments in the Routing header. Upon an error, the return value of the function is NULL. Note: If Routing header is type 2, this function returns a pointer to the IPv6 home address specified by the ip6r2_homeaddr field in which is pointed by bp. The index must be zero. Notes To use these functions, the application must be compiled with: To receive a Routing header, the application must enable the socket option: To send a Routing header, the application specifies it either as ancillary data in a call to or using (see send(2) and getsockopt(2), respectively). EXAMPLES
gives a comprehensive example in Appendix B. SEE ALSO
send(2), getsockopt(2), IPv6(7P), xopen_networking(7). inet6_rth_space(3N)
All times are GMT -4. The time now is 03:00 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy