Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Remove duplicates from comma separated list Post 303041350 by niki0211 on Thursday 21st of November 2019 04:13:14 PM
Old 11-21-2019
Remove duplicates from comma separated list

Hi,


I have following input file:


Code:
niki      niki
niki1    niki
niki2    niki,niki2
niki3    niki,niki3,niki
niki4    niki4,blabla
niki5    jkjkl
niki6    niki60,niki6

I would like to delete lines with identical matches completely and remove the selfmatches in the other lines.


Expected output:


Code:
niki1    niki
niki2    niki 
niki3    niki,niki
niki4    blabla
niki5    jkjkl
niki6    niki60


Is that somehow possible?


I tried several things with awk but I can not find a way to solve it properly.


I hope someone can help here.


Thanks!

Last edited by Scrutinizer; 11-21-2019 at 05:59 PM..
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove whitespaces between comma separated fields from file

Hello all, I am a unix dummy. I am trying to remove spaces between fields. I have the file in the following format 12332432, 2345 , asdfsdf ,100216 , 9999999 12332431, 2341 , asdfsd2 ,100213 , 9999999 &... (2 Replies)
Discussion started by: nitinbjoshi
2 Replies

2. Shell Programming and Scripting

Need comma separated processing

I have a file like this OUTLN OPEN Y SCOTT OPEN N USER4 OPEN Y DBSNMP EXPIRED & LOCKED N ... (4 Replies)
Discussion started by: ilugopal
4 Replies

3. Shell Programming and Scripting

Help parse comma separated list

I have a list of files with the same name, but they have a different date stamp in the name. I can find the first file, but I need to find the second file. I am using this information to create a variable I use later. Here is a example of how I find the first file. "ls -mr... (11 Replies)
Discussion started by: NoMadBanker
11 Replies

4. Shell Programming and Scripting

Output of command in comma separated list

Hi; I have an output of a particular command say $command fstl:r-x ajay:r-x how can i get this in comma separated list, eg: fstl:r-x,ajay:r-x Thnks; (4 Replies)
Discussion started by: ajaypadvi
4 Replies

5. Shell Programming and Scripting

Need Help - comma inside double quote in comma separated csv,

Hello there, I have a comma separated csv , and all the text field is wrapped by double quote. Issue is some text field contain comma as well inside double quote. so it is difficult to process. Input in the csv file is , 1,234,"abc,12,gh","GH234TY",34 I need output like below,... (8 Replies)
Discussion started by: Uttam Maji
8 Replies

6. UNIX for Dummies Questions & Answers

[solved] Comma separated values to space separated

Hi, I have a large number of files which are written as csv (comma-separated values). Does anyone know of simple sed/awk command do achieve this? Thanks! ---------- Post updated at 10:59 AM ---------- Previous update was at 10:54 AM ---------- Guess I asked this too soon. Found the... (0 Replies)
Discussion started by: lost.identity
0 Replies

7. Shell Programming and Scripting

awk to parse comma separated field and removing comma in between number and double quotes

Hi Experts, Please support I have below data in file in comma seperated, but 4th column is containing comma in between numbers, bcz of which when i tried to parse the file the column 6th value(5049641141) is being removed from the file and value(222.82) in column 5 becoming value of column6. ... (3 Replies)
Discussion started by: as7951
3 Replies

8. Shell Programming and Scripting

Remove duplicates separated by delimiter

First post, been browsing for 3 days and came out with nothing so far. M3 C2 V5 D5 HH:FF A1-A2,A5-A6,A1-A2,A1-4 B4-B6,B2-B4,B4-B6,B1-B2output should be M3 C2 V5 D5 HH:FF A1-A2,A5-A6,A1-A4 B2-B4,B4-B6,B1-B2On col 6 and 7 there are strings in form of Ax-Ax and Bx-Bx respectively. Each string are... (9 Replies)
Discussion started by: enrikS
9 Replies

9. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies
Locale::Codes::LangExt(3pm)				 Perl Programmers Reference Guide			       Locale::Codes::LangExt(3pm)

NAME
Locale::Codes::LangExt - standard codes for language extension identification SYNOPSIS
use Locale::Codes::LangExt; $lext = code2langext('acm'); # $lext gets 'Mesopotamian Arabic' $code = langext2code('Mesopotamian Arabic'); # $code gets 'acm' @codes = all_langext_codes(); @names = all_langext_names(); DESCRIPTION
The "Locale::Codes::LangExt" module provides access to standard codes used for identifying language extensions, such as those as defined in the IANA language registry. Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default IANA language registry codes will be used. SUPPORTED CODE SETS
There are several different code sets you can use for identifying language extensions. A code set may be specified using either a name, or a constant that is automatically exported by this module. For example, the two are equivalent: $lext = code2langext('acm','alpha'); $lext = code2langext('acm',LOCALE_LANGEXT_ALPHA); The codesets currently supported are: alpha This is the set of three-letter (lowercase) codes from the IANA language registry, such as 'acm' for Mesopotamian Arabic. This is the default code set. ROUTINES
code2langext ( CODE [,CODESET] ) langext2code ( NAME [,CODESET] ) langext_code2code ( CODE ,CODESET ,CODESET2 ) all_langext_codes ( [CODESET] ) all_langext_names ( [CODESET] ) Locale::Codes::LangExt::rename_langext ( CODE ,NEW_NAME [,CODESET] ) Locale::Codes::LangExt::add_langext ( CODE ,NAME [,CODESET] ) Locale::Codes::LangExt::delete_langext ( CODE [,CODESET] ) Locale::Codes::LangExt::add_langext_alias ( NAME ,NEW_NAME ) Locale::Codes::LangExt::delete_langext_alias ( NAME ) Locale::Codes::LangExt::rename_langext_code ( CODE ,NEW_CODE [,CODESET] ) Locale::Codes::LangExt::add_langext_code_alias ( CODE ,NEW_CODE [,CODESET] ) Locale::Codes::LangExt::delete_langext_code_alias ( CODE [,CODESET] ) These routines are all documented in the Locale::Codes::API man page. SEE ALSO
Locale::Codes The Locale-Codes distribution. Locale::Codes::API The list of functions supported by this module. http://www.iana.org/assignments/language-subtag-registry The IANA language subtag registry. AUTHOR
See Locale::Codes for full author history. Currently maintained by Sullivan Beck (sbeck@cpan.org). COPYRIGHT
Copyright (c) 2011-2013 Sullivan Beck This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.18.2 2013-11-04 Locale::Codes::LangExt(3pm)
All times are GMT -4. The time now is 03:58 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy