Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to Remove duplicate value from file?
Top Forums Shell Programming and Scripting How to Remove duplicate value from file? Post 302770728 by mohan sharma on Monday 18th of February 2013 06:09:56 AM
Old 02-18-2013
How to Remove duplicate value from file?
if different branch code is available for same BIC code and one of the branch code is XXX.only one row will be stored and with branch code as XXX .rest of the rows for the BIC code will not be stored.

for example if $7 is BIC code and $8 is branch code
INPUT file are following
Code:
BI	M	IT056868	UNICREDI	VIA	LIBHIA	UNCRIT2B	XXX	UNCR
BI	M	US001165	NEUBERGE	LLC	NEWYRK	NEUBUS33	253	NEUB
HH	M	IND90909	SBILIFES	HNI	NANANN	GGGGGGGG	142	UICC
HH	M	MNOOOO	        98989089	IIC	UMNKSS	MOHAN844	XXX	KLKL
HH	M	MNKKKKKK	90909090	JMV	MNJKMN	MOHAN844	256	LOPD
HH	M	MKLJHJKK	KIKIKIKI	JKJ	NMHMNM	MOHAN844	456	LOPS

here $7 is having 3 bic code with same value"MOHAN844" with diff branch code in $8 and i want only BIC code with XXX value ...and if XXX is not present in $8 then i need all record..

please Help me..

Moderator's Comments:
Mod Comment Use code tags, thanks.
Last edited by zaxxon; 02-18-2013 at 07:35 AM.. Reason: code tags
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Duplicate Lines in File

I am doing KSH script to remove duplicate lines in a file. Let say the file has format below. FileA 1253-6856 3101-4011 1827-1356 1822-1157 1822-1157 1000-1410 1000-1410 1822-1231 1822-1231 3101-4011 1822-1157 1822-1231 and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies

2. UNIX for Dummies Questions & Answers

Remove Duplicate lines from File

I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Discussion started by: Nysif Steve
18 Replies

3. Shell Programming and Scripting

Sort and Remove Duplicate on file

How do we sort and remove duplicate on column 1,2 retaining the record with maximum date (in feild 3) for the file with following format. aaa|1234|2010-12-31 aaa|1234|2010-11-10 bbb|345|2011-01-01 ccc|346|2011-02-01 bbb|345|2011-03-10 aaa|1234|2010-01-01 Required Output ... (5 Replies)
Discussion started by: mabarif16
5 Replies

4. Shell Programming and Scripting

Formatting a file - Remove Duplicate

Hi I have a file in the following format. Basically the file contains tablename and their aliases: TABLE1 TABLE1 A TABLE2 TABLE2 B TABLE3 TABLE4 TABLE4 C TABLE4 Upon formatting an sql statement I am getting such output. Problem: Whenever a tablename appears with alias, it has... (5 Replies)
Discussion started by: freakygs
5 Replies

5. Shell Programming and Scripting

remove of duplicate line from a file

I have a file a.txt having content like deepak ram sham deepram sita kumar I Want to delete the first line containing "deep" ... I tried using... grep -i 'deep' a.txt It gives me 2 rows...I want to delete the first one.. + need to know the command to delete the line from... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

6. Shell Programming and Scripting

How do I remove the duplicate lines in this file?

Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file. I wrote the following script, but does not work. What is the problem? The output file should only contain five lines: Later! (5 Replies)
Discussion started by: Ernst
5 Replies

7. Shell Programming and Scripting

Remove the duplicate content in a file

Here is the contents of test.txt Dependencies Resolved Changes in packages about to be updated: ChangeLog for: 1:perl-Archive-Extract-0.38-131.el6_4.x86_64, - Resolves: #915692 - CVE-2013-1667 (DoS in rehashing code) Dependencies Resolved Changes in packages about to be updated: ... (5 Replies)
Discussion started by: ashokvpp
5 Replies

8. Shell Programming and Scripting

Remove duplicate lines from a file

Hi, I have a csv file which contains some millions of lines in it. The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line). I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies

9. Shell Programming and Scripting

Remove duplicate values in a column(not in the file)

Hi Gurus, I have a file(weblog) as below abc|xyz|123|agentcode=sample code abcdeeess,agentcode=sample code abcdeeess,agentcode=sample code abcdeeess|agentadd=abcd stereet 23343,agentadd=abcd stereet 23343 sss|wwq|999|agentcode=sample1 code wqwdeeess,gentcode=sample1 code... (4 Replies)
Discussion started by: ratheeshjulk
4 Replies

LEARN ABOUT MOJAVE

datetime::locale::gez_er5.18

DateTime::Locale::gez_ER(3)				User Contributed Perl Documentation			       DateTime::Locale::gez_ER(3)

NAME

       DateTime::Locale::gez_ER

SYNOPSIS

	 use DateTime;

	 my $dt = DateTime->now( locale => 'gez_ER' );
	 print $dt->month_name();

DESCRIPTION

       This is the DateTime locale package for Geez Eritrea.

DATA

       This locale inherits from the DateTime::Locale::gez locale.

       It contains the following data.

   Days
       Wide (format)

	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXXX
	 XXX

       Abbreviated (format)

	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX

       Narrow (format)

	 X
	 X
	 X
	 X
	 X
	 X
	 X

       Wide (stand-alone)

	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXXX
	 XXX

       Abbreviated (stand-alone)

	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX

       Narrow (stand-alone)

	 X
	 X
	 X
	 X
	 X
	 X
	 X

   Months
       Wide (format)

	 XXX
	 XXX
	 XXX
	 XXX
	 XXXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX

       Abbreviated (format)

	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX

       Narrow (format)

	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X

       Wide (stand-alone)

	 XXX
	 XXX
	 XXX
	 XXX
	 XXXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX

       Abbreviated (stand-alone)

	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX
	 XXX

       Narrow (stand-alone)

	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X

   Quarters
       Wide (format)

	 Q1
	 Q2
	 Q3
	 Q4

       Abbreviated (format)

	 Q1
	 Q2
	 Q3
	 Q4

       Narrow (format)

	 1
	 2
	 3
	 4

       Wide (stand-alone)

	 Q1
	 Q2
	 Q3
	 Q4

       Abbreviated (stand-alone)

	 Q1
	 Q2
	 Q3
	 Q4

       Narrow (stand-alone)

	 1
	 2
	 3
	 4

   Eras
       Wide

	 BCE
	 CE

       Abbreviated

	 X/X
	 X/X

       Narrow

	 X/X
	 X/X

   Date Formats
       Full

	  2008-02-05T18:30:30 = XXXX 05 XXX XXXX 2008 X/X
	  1995-12-22T09:05:02 = XXXX 22 XXX XXXX 1995 X/X
	 -0010-09-15T04:44:23 = XXXXX 15 XXX XXXX -10 X/X

       Long

	  2008-02-05T18:30:30 = 05 XXX 2008
	  1995-12-22T09:05:02 = 22 XXX 1995
	 -0010-09-15T04:44:23 = 15 XXX -10

       Medium

	  2008-02-05T18:30:30 = 05-XXX-2008
	  1995-12-22T09:05:02 = 22-XXX-1995
	 -0010-09-15T04:44:23 = 15-XXX--10

       Short

	  2008-02-05T18:30:30 = 05/02/08
	  1995-12-22T09:05:02 = 22/12/95
	 -0010-09-15T04:44:23 = 15/09/-10

       Default

	  2008-02-05T18:30:30 = 05-XXX-2008
	  1995-12-22T09:05:02 = 22-XXX-1995
	 -0010-09-15T04:44:23 = 15-XXX--10

   Time Formats
       Full

	  2008-02-05T18:30:30 = 6:30:30 XXX UTC
	  1995-12-22T09:05:02 = 9:05:02 XXX UTC
	 -0010-09-15T04:44:23 = 4:44:23 XXX UTC

       Long

	  2008-02-05T18:30:30 = 6:30:30 XXX UTC
	  1995-12-22T09:05:02 = 9:05:02 XXX UTC
	 -0010-09-15T04:44:23 = 4:44:23 XXX UTC

       Medium

	  2008-02-05T18:30:30 = 6:30:30 XXX
	  1995-12-22T09:05:02 = 9:05:02 XXX
	 -0010-09-15T04:44:23 = 4:44:23 XXX

       Short

	  2008-02-05T18:30:30 = 6:30 XXX
	  1995-12-22T09:05:02 = 9:05 XXX
	 -0010-09-15T04:44:23 = 4:44 XXX

       Default

	  2008-02-05T18:30:30 = 6:30:30 XXX
	  1995-12-22T09:05:02 = 9:05:02 XXX
	 -0010-09-15T04:44:23 = 4:44:23 XXX

   Datetime Formats
       Full

	  2008-02-05T18:30:30 = XXXX 05 XXX XXXX 2008 X/X 6:30:30 XXX UTC
	  1995-12-22T09:05:02 = XXXX 22 XXX XXXX 1995 X/X 9:05:02 XXX UTC
	 -0010-09-15T04:44:23 = XXXXX 15 XXX XXXX -10 X/X 4:44:23 XXX UTC

       Long

	  2008-02-05T18:30:30 = 05 XXX 2008 6:30:30 XXX UTC
	  1995-12-22T09:05:02 = 22 XXX 1995 9:05:02 XXX UTC
	 -0010-09-15T04:44:23 = 15 XXX -10 4:44:23 XXX UTC

       Medium

	  2008-02-05T18:30:30 = 05-XXX-2008 6:30:30 XXX
	  1995-12-22T09:05:02 = 22-XXX-1995 9:05:02 XXX
	 -0010-09-15T04:44:23 = 15-XXX--10 4:44:23 XXX

       Short

	  2008-02-05T18:30:30 = 05/02/08 6:30 XXX
	  1995-12-22T09:05:02 = 22/12/95 9:05 XXX
	 -0010-09-15T04:44:23 = 15/09/-10 4:44 XXX

       Default

	  2008-02-05T18:30:30 = 05-XXX-2008 6:30:30 XXX
	  1995-12-22T09:05:02 = 22-XXX-1995 9:05:02 XXX
	 -0010-09-15T04:44:23 = 15-XXX--10 4:44:23 XXX

   Available Formats
       d (d)

	  2008-02-05T18:30:30 = 5
	  1995-12-22T09:05:02 = 22
	 -0010-09-15T04:44:23 = 15

       EEEd (d EEE)

	  2008-02-05T18:30:30 = 5 XXX
	  1995-12-22T09:05:02 = 22 XXX
	 -0010-09-15T04:44:23 = 15 XXX

       Hm (H:mm)

	  2008-02-05T18:30:30 = 18:30
	  1995-12-22T09:05:02 = 9:05
	 -0010-09-15T04:44:23 = 4:44

       hm (h:mm a)

	  2008-02-05T18:30:30 = 6:30 XXX
	  1995-12-22T09:05:02 = 9:05 XXX
	 -0010-09-15T04:44:23 = 4:44 XXX

       Hms (H:mm:ss)

	  2008-02-05T18:30:30 = 18:30:30
	  1995-12-22T09:05:02 = 9:05:02
	 -0010-09-15T04:44:23 = 4:44:23

       hms (h:mm:ss a)

	  2008-02-05T18:30:30 = 6:30:30 XXX
	  1995-12-22T09:05:02 = 9:05:02 XXX
	 -0010-09-15T04:44:23 = 4:44:23 XXX

       M (L)

	  2008-02-05T18:30:30 = 2
	  1995-12-22T09:05:02 = 12
	 -0010-09-15T04:44:23 = 9

       Md (M-d)

	  2008-02-05T18:30:30 = 2-5
	  1995-12-22T09:05:02 = 12-22
	 -0010-09-15T04:44:23 = 9-15

       MEd (E, M-d)

	  2008-02-05T18:30:30 = XXX, 2-5
	  1995-12-22T09:05:02 = XXX, 12-22
	 -0010-09-15T04:44:23 = XXX, 9-15

       MMdd (dd/MM)

	  2008-02-05T18:30:30 = 05/02
	  1995-12-22T09:05:02 = 22/12
	 -0010-09-15T04:44:23 = 15/09

       MMM (LLL)

	  2008-02-05T18:30:30 = XXX
	  1995-12-22T09:05:02 = XXX
	 -0010-09-15T04:44:23 = XXX

       MMMd (MMM d)

	  2008-02-05T18:30:30 = XXX 5
	  1995-12-22T09:05:02 = XXX 22
	 -0010-09-15T04:44:23 = XXX 15

       MMMEd (E MMM d)

	  2008-02-05T18:30:30 = XXX XXX 5
	  1995-12-22T09:05:02 = XXX XXX 22
	 -0010-09-15T04:44:23 = XXX XXX 15

       MMMMd (MMMM d)

	  2008-02-05T18:30:30 = XXX 5
	  1995-12-22T09:05:02 = XXX 22
	 -0010-09-15T04:44:23 = XXX 15

       MMMMdd (dd MMMM)

	  2008-02-05T18:30:30 = 05 XXX
	  1995-12-22T09:05:02 = 22 XXX
	 -0010-09-15T04:44:23 = 15 XXX

       MMMMEd (E MMMM d)

	  2008-02-05T18:30:30 = XXX XXX 5
	  1995-12-22T09:05:02 = XXX XXX 22
	 -0010-09-15T04:44:23 = XXX XXX 15

       ms (mm:ss)

	  2008-02-05T18:30:30 = 30:30
	  1995-12-22T09:05:02 = 05:02
	 -0010-09-15T04:44:23 = 44:23

       y (y)

	  2008-02-05T18:30:30 = 2008
	  1995-12-22T09:05:02 = 1995
	 -0010-09-15T04:44:23 = -10

       yM (y-M)

	  2008-02-05T18:30:30 = 2008-2
	  1995-12-22T09:05:02 = 1995-12
	 -0010-09-15T04:44:23 = -10-9

       yMEd (EEE, y-M-d)

	  2008-02-05T18:30:30 = XXX, 2008-2-5
	  1995-12-22T09:05:02 = XXX, 1995-12-22
	 -0010-09-15T04:44:23 = XXX, -10-9-15

       yMMM (y MMM)

	  2008-02-05T18:30:30 = 2008 XXX
	  1995-12-22T09:05:02 = 1995 XXX
	 -0010-09-15T04:44:23 = -10 XXX

       yMMMEd (EEE, y MMM d)

	  2008-02-05T18:30:30 = XXX, 2008 XXX 5
	  1995-12-22T09:05:02 = XXX, 1995 XXX 22
	 -0010-09-15T04:44:23 = XXX, -10 XXX 15

       yMMMM (y MMMM)

	  2008-02-05T18:30:30 = 2008 XXX
	  1995-12-22T09:05:02 = 1995 XXX
	 -0010-09-15T04:44:23 = -10 XXX

       yQ (y Q)

	  2008-02-05T18:30:30 = 2008 1
	  1995-12-22T09:05:02 = 1995 4
	 -0010-09-15T04:44:23 = -10 3

       yQQQ (y QQQ)

	  2008-02-05T18:30:30 = 2008 Q1
	  1995-12-22T09:05:02 = 1995 Q4
	 -0010-09-15T04:44:23 = -10 Q3

       yyMM (MM/yy)

	  2008-02-05T18:30:30 = 02/08
	  1995-12-22T09:05:02 = 12/95
	 -0010-09-15T04:44:23 = 09/-10

       yyQ (Q yy)

	  2008-02-05T18:30:30 = 1 08
	  1995-12-22T09:05:02 = 4 95
	 -0010-09-15T04:44:23 = 3 -10

       yyyyMMMM (MMMM y)

	  2008-02-05T18:30:30 = XXX 2008
	  1995-12-22T09:05:02 = XXX 1995
	 -0010-09-15T04:44:23 = XXX -10

   Miscellaneous
       Prefers 24 hour time?

       No

       Local first day of the week

       XXXX

SUPPORT

       See DateTime::Locale.

AUTHOR

       Dave Rolsky <autarch@urth.org>

COPYRIGHT

       Copyright (c) 2008 David Rolsky. All rights reserved. This program is free software; you can redistribute it and/or modify it under the
       same terms as Perl itself.

       This module was generated from data provided by the CLDR project, see the LICENSE.cldr in this distribution for details on the CLDR data's
       license.

perl v5.18.2							    2017-10-06					       DateTime::Locale::gez_ER(3)
All times are GMT -4. The time now is 08:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy