Sponsored Content
Full Discussion: Delete duplicate rows
Top Forums Shell Programming and Scripting Delete duplicate rows Post 302629379 by Toiday on Tuesday 24th of April 2012 02:42:13 PM
Old 04-24-2012
Code:
awk '{if (Previous_Line!=$1$2$3$4$5) print; Previous_Line=$1$2$3$4$5}' file

This is assuming the duplicated lines are already together like you have in your example. Otherwise, sort the file then pipe to the awk statement.

Moderator's Comments:
Mod Comment Link: How to use [code] tags

Last edited by Scrutinizer; 04-24-2012 at 04:05 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

duplicate rows in a file

hi all can anyone please let me know if there is a way to find out duplicate rows in a file. i have a file that has hundreds of numbers(all in next row). i want to find out the numbers that are repeted in the file. eg. 123434 534 5575 4746767 347624 5575 i want 5575 please help (3 Replies)
Discussion started by: infyanurag
3 Replies

2. Shell Programming and Scripting

how to delete duplicate rows in a file

I have a file content like below. "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","",""... (5 Replies)
Discussion started by: vamshikrishnab
5 Replies

3. Shell Programming and Scripting

How to extract duplicate rows

I have searched the internet for duplicate row extracting. All I have seen is extracting good rows or eliminating duplicate rows. How do I extract duplicate rows from a flat file in unix. I'm using Korn shell on HP Unix. For.eg. FlatFile.txt ======== 123:456:678 123:456:678 123:456:876... (5 Replies)
Discussion started by: bobbygsk
5 Replies

4. HP-UX

How to get Duplicate rows in a file

Hi all, I have written one shell script. The output file of this script is having sql output. In that file, I want to extract the rows which are having multiple entries(duplicate rows). For example, the output file will be like the following way. ... (7 Replies)
Discussion started by: raghu.iv85
7 Replies

5. Shell Programming and Scripting

how to delete duplicate rows based on last column

hii i have a huge amt of data stored in a file.Here in this file i need to remove duplicates rows in such a way that the last column has different data & i must check for greatest among last colmn data & print the largest data along with other entries but just one of other duplicate entries is... (16 Replies)
Discussion started by: reva
16 Replies

6. Ubuntu

delete duplicate rows with awk files

Hi every body I have some text file with a lots of duplicate rows like this: 165.179.568.197 154.893.836.174 242.473.396.153 165.179.568.197 165.179.568.197 165.179.568.197 154.893.836.174 how can I delete the repeated rows? Thanks Saeideh (2 Replies)
Discussion started by: sashtari
2 Replies

7. Shell Programming and Scripting

How to extract duplicate rows

Hi! I have a file as below: line1 line2 line2 line3 line3 line3 line4 line4 line4 line4 I would like to extract duplicate lines (not unique, triplicate or quadruplicate lines). Output will be as below: line2 line2 I would appreciate if anyone can help. Thanks. (4 Replies)
Discussion started by: chromatin
4 Replies

8. Shell Programming and Scripting

Median and max of duplicate rows

Hi all, plz help me with this, I want to to extract the duplicate rows (column 1) in a file which at least repeat 4 times. then I want to summarize them by getting the max , mean, median and min. The file is sorted by column 1, all the repeated rows appear together. If number of elements is... (5 Replies)
Discussion started by: ritakadm
5 Replies

9. Shell Programming and Scripting

Extract duplicate rows with conditions

Gents Can you help please. Input file 5490921425 1 7 1310342 54909214251 5490921425 2 1 1 54909214252 5491120937 1 1 3 54911209371 5491120937 3 1 1 54911209373 5491320785 1 ... (4 Replies)
Discussion started by: jiam912
4 Replies

10. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies
GAPPLETVIEWER(1)                                                        GNU                                                       GAPPLETVIEWER(1)

NAME
gappletviewer - Load and runs an applet SYNOPSIS
appletviewer [OPTION]... URL... appletviewer [OPTION]... -code CODE appletviewer [OPTION]... -plugin INPUT,OUTPUT DESCRIPTION
The appletviewer tool loads and runs an applet. Use the first form to test applets specified by tag. The URL should resolve to an HTML document from which the appletviewer will extract applet tags. The APPLET, EMBED and OBJECT tags are supported. If a given document contains multiple applet tags, all the applets will be loaded, with each applet appearing in its own window. Likewise, when multiple URLs are specified, each applet tag instance is given its own window. If a given document contains no recognized tags the appletviewer does nothing. appletviewer http://www.gnu.org/software/classpath/ Use the second form to test an applet in development. This form allows applet tag attributes to be supplied on the command line. Only one applet may be specified using the -code option. The -code option overrides the URL form -- any URLs specified will be ignored. appletviewer -code Test.class -param datafile,data.txt gcjwebplugin uses the third form to communicate with the appletviewer through named pipes. OPTIONS
URL OPTIONS -debug This option is not yet implemented but is provided for compatibility. -encoding CHARSET Use this option to specify an alternate character encoding for the specified HTML page. APPLET TAG OPTIONS -code CODE Use the -code option to specify the value of the applet tag CODE attribute. -codebase CODEBASE Use the -codebase option to specify the value of the applet tag CODEBASE attribute. -archive ARCHIVE Use the -archive option to specify the value of the applet tag ARCHIVE attribute. -width WIDTH Use the -width option to specify the value of the applet tag WIDTH attribute. -height HEIGHT Use the -height option to specify the value of the applet tag HEIGHT attribute. -param NAME,VALUE Use the -param option to specify values for the NAME and VALUE attributes of an applet PARAM tag. PLUGIN OPTION -plugin INPUT,OUTPUT gcjwebplugin uses the -plugin option to specify the named pipe the appletviewer should use for receiving commands (INPUT) and the one it should use for sending commands to gcjwebplugin (OUTPUT). DEBUGGING OPTION -verbose Use the -verbose option to have the appletviewer print debugging messages. STANDARD OPTIONS -help Use the -help option to have the appletviewer print a usage message, then exit. -version Use the -version option to have the appletviewer print its version, then exit. -JOPTION Use the -J option to pass OPTION to the virtual machine that will run the appletviewer. Unlike other options, there must not be a space between the -J and OPTION. BUGS
SEE ALSO
AUTHOR
0.98 2010-07-05 GAPPLETVIEWER(1)
All times are GMT -4. The time now is 07:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy