Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: deleting dupes in a row
Top Forums Shell Programming and Scripting deleting dupes in a row Post 302698195 by itkamaraj on Sunday 9th of September 2012 12:39:15 AM
Old 09-09-2012
Code:
$ awk -F= '{for(i=1;i<=NF;i++){a[$i]++;}for(i in a){b=b"="i}{sub("=","",b);$0=b;b="";delete a}}1' OFS=\= input.txt
md=mohammed=mohd=muhammed=muhmd
mahndra=mahendra=mahendera

---------- Post updated at 10:09 AM ---------- Previous update was at 10:04 AM ----------

Code:
$ perl -F= -lane 'print join "=",keys %{{ map { $_ => 1 } @F}};' input.txt
muhammed=muhmd=md=mohammed=mohd
mahendra=mahendera=mahndra

This User Gave Thanks to itkamaraj For This Post:
gimley
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Deleting all occurences of a duplicate row

Hi, I need to delete all occurences of the repeated lines from a file and retain only the lines that is not repeated elsewhere in the file. As seen below the first two lines are same except that for the string "From BaseLine" and "From SMS".I shouldn't consider the string "From SMS" and "From... (7 Replies)
Discussion started by: ragavhere
7 Replies

2. Shell Programming and Scripting

deleting blank line and row containing certain words in single sed command

Hi Is it possible to do the following in a single command /usr/xpg4/bin/sed -e '/rows selected/d' /aemu/CALLAUTO/callauto.txt > /aemu/CALLAUTO/callautonew.txt /usr/xpg4/bin/sed -e '/^$/d' /aemu/CALLAUTO/callautonew.txt > /aemu/CALLAUTO/callauto_new.txt exit (1 Reply)
Discussion started by: aemunathan
1 Replies

3. UNIX for Dummies Questions & Answers

deleting a row if a certain column is below a certain number

How can you delete a row if a certain column is bigger than a certain number? I have the following input: 20080709 20081222 95750 1 0 0.02 94.88 20080709 20081222 95750 2 0 0.89 94.88 20080709 20081222 9575 1 0 0 94.88 20080709 20081222 9575 2 0 0 94.88 20080709 20081222 9587.5 1 0 0... (1 Reply)
Discussion started by: Pep Puigvert
1 Replies

4. UNIX for Dummies Questions & Answers

deleting a row if a certain column is below a certain number

How can you delete a row if a certain column is bigger than a certain number? I have the following input: 20080709 20081222 95750 1 0 0.02 94.88 20080709 20081222 95750 2 0 0.89 94.88 20080709 20081222 9575 1 0 0 94.88 20080709 20081222 9575 2 0 0 94.88 20080709 20081222 9587.5 1 0 0... (6 Replies)
Discussion started by: Pep Puigvert
6 Replies

5. Shell Programming and Scripting

Insert row without deleting previous data using sed

Hello, I want to add a new row to a file to insert data without deleting the previous data there. Example: file a b c d Output a b newtext c (6 Replies)
Discussion started by: joseamck
6 Replies

6. Shell Programming and Scripting

Script for identifying and deleting dupes in a line

I am compiling a synonym dictionary which has the following structure Headword=Synonym1,Synonym2 and so on, with each synonym separated by a comma. As is usual in such cases manual preparation of synonyms results in repeating the synonym which results in dupes as in the example below:... (3 Replies)
Discussion started by: gimley
3 Replies

7. Shell Programming and Scripting

Deleting a row based on fetched value of column

Hi, I have a file which consists of two columns but the first one can be varying in length like 123456789 0abcd 123456789 0abcd 4015 0 0abcd 5000 0abcd I want to go through the file reading each line, count the number of characters in the first column and delete... (2 Replies)
Discussion started by: swasid
2 Replies

8. Shell Programming and Scripting

Deleting row if all column values are a particular string

Hello, I have a very large file for which I would like to remove all rows for which the value of columns 2-5 is zero. For instance I would like this file: contig1, 0, 0, 0, 0 contig2, 1, 3, 5, 0 contig3, 0, 0, 0, 0 contig4, 0, 5, 6, 7 To become this file: contig2, 1, 3, 5,0 ... (17 Replies)
Discussion started by: mouchkam
17 Replies

9. Shell Programming and Scripting

Moving new row and deleting old row to another table

Hi, I want to move a new row to another table if the field from new row doesn't have the first word that I categorized (like: IRC blablabla, PTM blablabla, ADM blablabla, BS blablabla). I already use this script but doesn't work as I expected. CHECK_KEYWORD="$( mysql -uroot -p123456 smsd -N... (7 Replies)
Discussion started by: jazzyzha
7 Replies

10. Shell Programming and Scripting

In php, Moving a new row to another table and deleting old row

Hi, I already succeed moving a new row to another table if the field from new row doesn't have the first word that I categorized (like: IRC blablabla, PTM blablabla, ADM blablabla, BS blablabla). But it can't delete the old row. Please help me with the script. my php script: INSERT INTO... (2 Replies)
Discussion started by: jazzyzha
2 Replies

LEARN ABOUT MOJAVE

locale::codes::langfam

Locale::Codes::LangFam(3pm)				 Perl Programmers Reference Guide			       Locale::Codes::LangFam(3pm)

NAME

       Locale::Codes::LangFam - standard codes for language extension identification

SYNOPSIS

	  use Locale::Codes::LangFam;

	  $lext = code2langfam('apa');		       # $lext gets 'Apache languages'
	  $code = langfam2code('Apache languages');    # $code gets 'apa'

	  @codes   = all_langfam_codes();
	  @names   = all_langfam_names();

DESCRIPTION

       The "Locale::Codes::LangFam" module provides access to standard codes used for identifying language families, such as those as defined in
       ISO 639-5.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 639-5
       language family codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying language families. A code set may be specified using either a name, or a
       constant that is automatically exported by this module.

       For example, the two are equivalent:

	  $lext = code2langfam('apa','alpha');
	  $lext = code2langfam('apa',LOCALE_LANGFAM_ALPHA);

       The codesets currently supported are:

       alpha
	   This is the set of three-letter (lowercase) codes from ISO 639-5 such as 'apa' for Apache languages.

	   This is the default code set.

ROUTINES

       code2langfam ( CODE [,CODESET] )
       langfam2code ( NAME [,CODESET] )
       langfam_code2code ( CODE ,CODESET ,CODESET2 )
       all_langfam_codes ( [CODESET] )
       all_langfam_names ( [CODESET] )
       Locale::Codes::LangFam::rename_langfam  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Codes::LangFam::add_langfam  ( CODE ,NAME [,CODESET] )
       Locale::Codes::LangFam::delete_langfam  ( CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_alias  ( NAME ,NEW_NAME )
       Locale::Codes::LangFam::delete_langfam_alias  ( NAME )
       Locale::Codes::LangFam::rename_langfam_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::delete_langfam_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.loc.gov/standards/iso639-5/id.php
	   ISO 639-5 .

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 2011-2013 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.18.2							    2013-11-04					       Locale::Codes::LangFam(3pm)
All times are GMT -4. The time now is 04:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy