Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Remove duplicates from comma separated list Post 303041357 by Chubler_XL on Thursday 21st of November 2019 09:46:09 PM
Old 11-21-2019
You could use split() to put $2 in an array and then step thru it like this:

Code:
awk '
{ 
   a=""
   e=split($2, vals, ",")
   for(i=1;i<=e;i++) if(vals[i]!=$1) a=a","vals[i]
   if(a) print $1,substr(a,2)
} ' infile

 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove whitespaces between comma separated fields from file

Hello all, I am a unix dummy. I am trying to remove spaces between fields. I have the file in the following format 12332432, 2345 , asdfsdf ,100216 , 9999999 12332431, 2341 , asdfsd2 ,100213 , 9999999 &... (2 Replies)
Discussion started by: nitinbjoshi
2 Replies

2. Shell Programming and Scripting

Need comma separated processing

I have a file like this OUTLN OPEN Y SCOTT OPEN N USER4 OPEN Y DBSNMP EXPIRED & LOCKED N ... (4 Replies)
Discussion started by: ilugopal
4 Replies

3. Shell Programming and Scripting

Help parse comma separated list

I have a list of files with the same name, but they have a different date stamp in the name. I can find the first file, but I need to find the second file. I am using this information to create a variable I use later. Here is a example of how I find the first file. "ls -mr... (11 Replies)
Discussion started by: NoMadBanker
11 Replies

4. Shell Programming and Scripting

Output of command in comma separated list

Hi; I have an output of a particular command say $command fstl:r-x ajay:r-x how can i get this in comma separated list, eg: fstl:r-x,ajay:r-x Thnks; (4 Replies)
Discussion started by: ajaypadvi
4 Replies

5. Shell Programming and Scripting

Need Help - comma inside double quote in comma separated csv,

Hello there, I have a comma separated csv , and all the text field is wrapped by double quote. Issue is some text field contain comma as well inside double quote. so it is difficult to process. Input in the csv file is , 1,234,"abc,12,gh","GH234TY",34 I need output like below,... (8 Replies)
Discussion started by: Uttam Maji
8 Replies

6. UNIX for Dummies Questions & Answers

[solved] Comma separated values to space separated

Hi, I have a large number of files which are written as csv (comma-separated values). Does anyone know of simple sed/awk command do achieve this? Thanks! ---------- Post updated at 10:59 AM ---------- Previous update was at 10:54 AM ---------- Guess I asked this too soon. Found the... (0 Replies)
Discussion started by: lost.identity
0 Replies

7. Shell Programming and Scripting

awk to parse comma separated field and removing comma in between number and double quotes

Hi Experts, Please support I have below data in file in comma seperated, but 4th column is containing comma in between numbers, bcz of which when i tried to parse the file the column 6th value(5049641141) is being removed from the file and value(222.82) in column 5 becoming value of column6. ... (3 Replies)
Discussion started by: as7951
3 Replies

8. Shell Programming and Scripting

Remove duplicates separated by delimiter

First post, been browsing for 3 days and came out with nothing so far. M3 C2 V5 D5 HH:FF A1-A2,A5-A6,A1-A2,A1-4 B4-B6,B2-B4,B4-B6,B1-B2output should be M3 C2 V5 D5 HH:FF A1-A2,A5-A6,A1-A4 B2-B4,B4-B6,B1-B2On col 6 and 7 there are strings in form of Ax-Ax and Bx-Bx respectively. Each string are... (9 Replies)
Discussion started by: enrikS
9 Replies

9. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies
split(1)						      General Commands Manual							  split(1)

NAME
split - split a file into pieces SYNOPSIS
line_count] suffix_length] [file [name]] nsuffix_length] [file [name]] Obsolescent n] [file [name]] DESCRIPTION
reads file and writes it in pieces (default 1000 lines) onto a set of output files. The name of the first output file is name with appended, and so on lexicographically, up to (only ASCII letters are used, a maximum of 676 files). If no output name is given, is the default. If no input file is given, or if is given instead, the standard input file is used. Options recognizes the following command-line options and arguments: The input file is split into pieces line_count lines in size. suffix_length letters are used to form the suffix of the output filenames. This option allows creation of more than 676 output files. The output file names created cannot exceed the maximum file name length allowed in the directory containing the files. The input file is split into pieces n bytes in size. The input file is split into pieces n x 1024 bytes in size. No space separates the n from the The input file is split into pieces n x 1048576 bytes in size. No space separates the n from the The input file is split into pieces n lines in size. This option is obsolescent and is equivalent to using the option. EXTERNAL INFLUENCES
Environment Variables determines the locale for the interpretation of text as single- and/or multi-byte characters. determines the language in which messages are displayed. If or is not specified in the environment or is set to the empty string, the value of is used as a default for each unspecified or empty variable. If is not specified or is set to the empty string, a default of "C" (see lang(5)) is used instead of If any internationalization variable contains an invalid setting, behaves as if all internationalization variables are set to "C". See environ(5). International Code Set Support Single- and multi-byte character code sets are supported. SEE ALSO
csplit(1). STANDARDS CONFORMANCE
split(1)
All times are GMT -4. The time now is 11:27 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy