Home Man
Search
Today's Posts
Register

If you're not sure where to post a Unix or Linux question, post it here. All unix and Linux beginners welcome in this forum!

How can I remove partial duplicates and manipulate text?

Login to Reply

 
Thread Tools Search this Thread
# 1  
Old 12-30-2017
How can I remove partial duplicates and manipulate text?

Hello,

How can I remove partial duplicates and manipulate text in bash using either awk, grep or sed? Thanks.


Input:

Code:
ted,"foo,bar,zoo"
john-son,"foot,ben,zoo"
bob,"bar,foot"

Expected Output:

Code:
foo,ted
bar,ted
zoo,ted
foot,john-son
ben,john-son

# 2  
Old 12-30-2017
What have you tried so far?
# 3  
Old 12-30-2017
this did not work.

Code:
perl -lpe 's/\s\K\S+/join ",", grep {!$seen{$_}++} split ",", $&/e'

# 4  
Old 12-30-2017
It is interesting that you want code written in awk, grep, or sed but show us non-working perl code.

You might be able to use something like:
Code:
awk -F, -v OFS=, '
{	gsub(/"/, "")
	for(i = 2; i <= NF; i++)
		if(!($i in seen)) {
			seen[$i]
			print $i, $1
		}
}' file

which, if file contains your sample input, produces the output you said you wanted.

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
# 5  
Old 12-30-2017
Thank you very much. It worked.
Login to Reply

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
A better way to manipulate text da1 Shell Programming and Scripting 8 09-07-2018 07:31 AM
Script to compare partial filenames in two folders and delete duplicates shaky Shell Programming and Scripting 2 01-12-2017 10:33 AM
Match partial text cmccabe Shell Programming and Scripting 7 03-10-2015 12:49 PM
Manipulate the text file in UNIX siva.pitchai Shell Programming and Scripting 6 11-27-2014 02:40 AM
sed to remove partial text in one line only kumars1331@gmai Shell Programming and Scripting 3 10-12-2014 08:54 PM
Remove duplicates dtdt Shell Programming and Scripting 3 04-20-2014 02:26 PM
Awk: Remove Duplicates siramitsharma Shell Programming and Scripting 4 01-23-2014 04:27 AM
Remove the partial duplicates by checking the length of a field asyed Shell Programming and Scripting 3 09-23-2011 09:33 AM
Script to manipulate logfile text rosspaddock Shell Programming and Scripting 11 12-02-2008 09:10 PM
using sed to manipulate text in files Jefferson333 UNIX for Dummies Questions & Answers 3 08-09-2008 03:26 AM


All times are GMT -4. The time now is 05:14 AM.

Unix & Linux Forums Content Copyrightę1993-2018. All Rights Reserved.
UNIX.COM Login
Username:
Password:  
Show Password