Sponsored Content
Top Forums Shell Programming and Scripting Removing duplicated first field rows Post 302906182 by palex on Tuesday 17th of June 2014 07:04:23 PM
Old 06-17-2014
Removing duplicated first field rows

Hello,
I am trying to eliminate rows where the first field is duplicated, leaving the row where the last field is "NET".


Data file:
Code:
345234|22.34|LST
546543|55.33|LST
793929|98.23|LST
793929|64.69|NET
149593|49.22|LST


Desired output:
Code:
345234|22.34|LST
546543|55.33|LST
793929|64.69|NET
149593|49.22|LST

Thanks very much!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help removing lines with duplicated columns

Hi Guys... Please Could you help me with the following ? aaaa bbbb cccc sdsd aaaa bbbb cccc qwer as you can see, the 2 lines are matched in three fields... how can I delete this pupicate ? I mean to delete the second one if 3 fields were duplicated ? Thanks (14 Replies)
Discussion started by: yahyaaa
14 Replies

2. Shell Programming and Scripting

Removing rows from a file

I have a file like below and want to use awk to solve this problem. The record separator is ">". I want to look at each record section enclosed within ">". Find the row with the 2nd and 3rd columns being 0, such as 10 0 0 I need to take the first number which in this case is 10. Then... (15 Replies)
Discussion started by: kristinu
15 Replies

3. Shell Programming and Scripting

Removing rows based on a another file

Hi, I am not sure if this has already been asked (I tried the search but the search was too broad). Basically I want to remove rows based on another file. So file1 looks like this (tab seperated): HHN 3 5 5 HUJ 2 2 1 JJJ 3 1 1 JUN 2 1 3 I have another file (file2)... (2 Replies)
Discussion started by: kylle345
2 Replies

4. Shell Programming and Scripting

Remove rows with first 4 fields duplicated in awk

Hi, I am trying to use awk to remove all rows where the first 4 fields are duplicates. e.g. in the following data lines 6-9 would be removed, leaving one copy of the duplicated row (row 5) Borgarhraun FH9822 ol24 FH9822_ol24_m20 ol Deformed c Borgarhraun FH9822 ol24 ... (3 Replies)
Discussion started by: tomahawk
3 Replies

5. UNIX for Dummies Questions & Answers

Removing duplicated lines??

Hi Guys.. I have a problem for some reason my database has copied everything 4 times. My Database looks like this: >BAC233456 rhjieaheiohjteo tjtjrj6jkk6k6 j54ju54jh54jh >ANI124365 afrhtjykulilil htrjykuk rtkjryky ukrykyrk >BAC233456 rhjieaheiohjteo tjtjrj6jkk6k6 j54ju54jh54jh... (6 Replies)
Discussion started by: Iifa
6 Replies

6. Shell Programming and Scripting

Removing rows and chars from text file

Dear community, maybe I'm asking the moon :rolleyes:, but I'm scratching my head to find a solution for it. :wall: I have a file called query.out (coming from Oracle query), the file is like this: ADDR TOTAL -------------------- ---------- TGROUPAGGR... (16 Replies)
Discussion started by: Lord Spectre
16 Replies

7. Shell Programming and Scripting

Removing Duplicate Rows in a file

Hello I have a file with contents like this... Part1 Field2 Field3 Field4 (line1) Part2 Field2 Field3 Field4 (line2) Part3 Field2 Field3 Field4 (line3) Part1 Field2 Field3 Field4 (line4) Part4 Field2 Field3 Field4 (line5) Part5 Field2 Field3 Field4 (line6) Part2 Field2 Field3 Field4... (7 Replies)
Discussion started by: ekbaazigar
7 Replies

8. Shell Programming and Scripting

Merge files and remove duplicated rows

In a folder I'll several times daily receive new files that I want to combine into one big file, without any duplicate rows. The file name in the folder will look like e.q: MissingData_2014-08-25_09-30-18.txt MissingData_2014-08-25_09-30-14.txt MissingData_2014-08-26_09-30-12.txt The content... (9 Replies)
Discussion started by: Bergans
9 Replies

9. UNIX for Dummies Questions & Answers

Removing rows that contain non-unique column entry

Background: I have a file of thousands of potential SSR primers from Batch Primer 3. I can't use primers that will contain the same sequence ID or sequence as another primer. I have some basic shell scripting skills, but not enough to handle this. What you need to know: I need to remove the... (1 Reply)
Discussion started by: msatseqs
1 Replies

10. Shell Programming and Scripting

Removing top few and bottom few rows in a file

Hi, I have a requirement where I need to delete given number of top and bottom rows in a flat file which has new line as its delimiter. For ex: if top_rows=2 & bottom_rows=1 Then in a given file which looks like: New York DC LA London Tokyo Prague Paris Bombay Sydney... (7 Replies)
Discussion started by: calredd
7 Replies
NETROM(4)						     Linux Programmer's Manual							 NETROM(4)

NAME
AF_NETROM - NET/ROM amateur packet radio protocol family DESCRIPTION
NET/ROM is a protocol used extensively by radio amateurs. The Linux NET/ROM protocol family permits access to these protocols via the stan- dard networking socket metaphor. The NET/ROM protocol layer only supports connected mode. IP traffic may be stacked on top of NET/ROM frames using a non-standard extension to the NET/ROM protocol. The only mode of operation is connected mode which is the mode used for a socket of type SOCK_SEQPACKET (stream sockets are not available in NET/ROM). This requires that the user ensures output data is suitably packetised, and that input data is read a packet at a time into a buffer of suitable size. NET/ROM addresses consist of 6 ascii characters and a number called the SSID. These are encoded into a sockaddr_ax25 structure which is provided to the relevant system calls. NET/ROM has some unusual properties. Notably in a multi-user system an AX.25 address is often associated with a user, and some users may not have such an association. a set of ioctl calls are provided to manage an association table. NET/ROM supports the following socket options for SOL_NETROM. NETROM_T1 is the T1 timer in 1/10ths of a second, NETROM_T2 is the T2 timer in 1/10ths of a second. NETROM_N2, the retry counter is also configurable. There is no 'infinite retry' option supported however. It is possible for an application to request that the NET/ROM layer return the NET/ROM header as well as the application data, this is done via the NETROM_HDRINCL socket option. SEE ALSO
call(1), socket(2), setsockopt(2), getsockopt(2), nrbroadcast(5), nrports(5), netromd(8), noderest(8), nodesave(8), nrparms(8). AUTHOR
Jonathan Naylor G4KLX <g4klx@g4klx.demon.co.uk> Linux 25 July 1996 NETROM(4)
All times are GMT -4. The time now is 07:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy