Sponsored Content
Top Forums Shell Programming and Scripting awk script to find duplicate values Post 302912125 by aramacha on Wednesday 6th of August 2014 05:44:04 AM
Old 08-06-2014
Thanks RudiC, its working as expected. But I am struggling a bit to convert into a script file. Can you please help me.

Thanks
 

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find and replace duplicate column values in a row

I have file which as 12 columns and values like this 1,2,3,4,5 a,b,c,d,e b,c,a,e,f a,b,e,a,h if you see the first column has duplicate values, I need to identify (print it to console) the duplicate value (which is 'a') and also remove duplicate values like below. I could be in two... (5 Replies)
Discussion started by: nuthalapati
5 Replies

2. Shell Programming and Scripting

Find duplicate based on 'n' fields and mark the duplicate as 'D'

Hi, In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'. In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies

3. Shell Programming and Scripting

[Solved] Find duplicate and add pattern in sed/awk

<Update> I have the solution: sed 's/\{3\}/&;&;---;4/' The thread can be marked as solved! </Update> Hi There, I'm working on a script processing some data from a website into cvs format. There is only one final problem left I can't find a solution. I've processed my file... (0 Replies)
Discussion started by: lolworlds
0 Replies

4. Shell Programming and Scripting

How to find the X highest values in a list depending on the values of another list with bash/awk?

Hi everyone, This is an exemple of inpout.txt file (a "," delimited text file which can be open as csv file): ID, Code, Value, Store SP|01, AABBCDE, 15, 3 SP|01, AABBCDE, 14, 2 SP|01, AABBCDF, 13, 2 SP|01, AABBCDE, 16, 3 SP|02, AABBCED, 15, 2 SP|01, AABBCDF, 12, 3 SP|01, AABBCDD,... (1 Reply)
Discussion started by: jeremy589
1 Replies

5. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

6. Shell Programming and Scripting

Sum duplicate values in text file through awk between dates

I need to sum values in text file in case duplicate row are present with same name and different value below is example of data in file i have and format i need. Data in text file 20170308 PM,U,2 PM,U,113 PM,I,123 DA,U,135 DA,I,113 DA,I,1 20170309 PM,U,2 PM,U,1 PM,I,123 PM,I,1... (3 Replies)
Discussion started by: Adfire
3 Replies

7. Shell Programming and Scripting

Do replace operation and awk to sum multiple columns if another column has duplicate values

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (12 Replies)
Discussion started by: as7951
12 Replies

8. UNIX for Beginners Questions & Answers

Find lines with duplicate values in a particular column

I have a file with 5 columns. I want to pull out all records where the value in column 4 is not unique. For example in the sample below, I would want it to print out all lines except for the last two. 40991764 2419 724 47182 Cand A 40992936 3591 724 47182 Cand B 40993016 3671 724 47182 Cand C... (5 Replies)
Discussion started by: kaktus
5 Replies
DBIx::Class::Manual::DocMap(3)				User Contributed Perl Documentation			    DBIx::Class::Manual::DocMap(3)

NAME
DBIx::Class::Manual::DocMap - What documentation do we have? Manuals DBIx::Class::Manual - User's Manual overview. DBIx::Class::Manual::Intro - Introduction to setting up and using DBIx::Class. DBIx::Class::Manual::Example - Full example Schema. DBIx::Class::Manual::SQLHackers::TOC - How to use DBIx::Class if you know SQL (external, available on CPAN) DBIx::Class::Manual::Glossary - What do all those terms mean? DBIx::Class::Manual::Cookbook - Various short recipes on how to do things. DBIx::Class::Manual::FAQ - Frequently Asked Questions, gathered from IRC and the mailing list. DBIx::Class::Manual::Troubleshooting - What to do if things go wrong (diagnostics of known error messages). DBIx::Class::Manual::Component - How to write your own DBIx::Class components. Setting up DBIx::Class::Schema - Overall schemas, and connection container. DBIx::Class::ResultSource - Source/Table definition functions. DBIx::Class::Relationship - Simple relationships. DBIx::Class::Relationship::Base - Relationship details. DBIx::Class::PK::Auto - Magically retrieve auto-incrementing fields. DBIx::Class::Core - Set of standard components to load. DBIx::Class::InflateColumn - Making objects out of your columns. DBIx::Class::InflateColumn::DateTime - Magically turn your datetime or timestamp columns into DateTime objects. DBIx::Class::PK - Dealing with primary keys. DBIx::Class::ResultSourceProxy::Table - Turns the resultsource into a table. DBIx::Class::AccessorGroup - Accessor grouping. Retrieving and creating data DBIx::Class::ResultSet - Selecting and manipulating sets. DBIx::Class::ResultSetColumn - Perform operations on entire columns of a ResultSet. DBIx::Class::Row - Dealing with actual data. DBIx::Class::Storage - Basic Storage stuff. DBIx::Class::Storage::DBI - Storage using DBI and SQL::Abstract. perl v5.16.2 2012-08-16 DBIx::Class::Manual::DocMap(3)
All times are GMT -4. The time now is 09:45 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy