05-05-2008
duplicates lines with one column different
Hi
I have the following lines in a file
SANDI108085FRANKLIN WRAP 7285
SANDI109514ZIPLOC STRETCH N SEAL 7285
SANDI110198CHOICE DM 0911
SANDI111144RANDOM WEIGHT BRAND 0704
SANDI111144RANDOM WEIGHT BRAND 0738
SANDI111144RANDOM WEIGHT BRAND 0739
SANDI113951NBL-NO COMPANY LISTED 7285
SANDI115203HOME BASICS 7285
I need the output like
SANDI108085FRANKLIN WRAP 7285
SANDI109514ZIPLOC STRETCH N SEAL 7285
SANDI110198CHOICE DM 0911
SANDI111144RANDOM WEIGHT BRAND 0704 0738 0739
SANDI113951NBL-NO COMPANY LISTED 7285
SANDI115203HOME BASICS 7285
Note:- SANDI111144RANDOMWEIGHT BRAND has same lines repeated but the last column is different, i am grouping those columns
Is there any way in sed or awk which can fit the logic very easily.
Regards
Dhana
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Input:
a
b
b
c
d
d
I need:
a
c
I know how to get this (the lines that have duplicates) :
b
d
sort file | uniq -d
But i need opossite of this. I have searched the forum and other places as well, but have found solution for everything except this variant of the problem. (3 Replies)
Discussion started by: necroman08
3 Replies
2. Shell Programming and Scripting
I have my data something like this
(08/03/2009 22:57:42.414)(:) king aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbb
(08/03/2009 22:57:42.416)(:) John cccccccccccc cccccvssssssssss baaaaa
(08/03/2009 22:57:42.417)(:) Michael ddddddd tststststtststts
(08/03/2009 22:57:42.425)(:) Ravi... (11 Replies)
Discussion started by: rdhanek
11 Replies
3. Shell Programming and Scripting
Hi All,
i need ti delete two duplicate processss which are running on the same device type (column 1) and port ID (column 2). here is the sample data
p1sc1m1 15517 11325 0 01:00:24 ? 0:00 scagntclsx25octtcp 2967 in3v mvmp01 0 8000 N S 969 750@751@752@
p1sc1m1 15519 11325 0 01:00:24 ? ... (5 Replies)
Discussion started by: neeraj617
5 Replies
4. Shell Programming and Scripting
Given a file such as this I need to remove the duplicates.
00060011 PAUL BOWSTEIN ad_waq3_921_20100826_010517.txt
00060011 PAUL BOWSTEIN ad_waq3_921_20100827_010528.txt
0624-01 RUT CORPORATION ad_sade3_10_20100827_010528.txt
0624-01 RUT CORPORATION ... (13 Replies)
Discussion started by: script_op2a
13 Replies
5. Shell Programming and Scripting
Hi all,
I have an input file like this
Now
I have to remove duplicates only in first column and nothing has to be changed in second and third column. so that output would be
Please let me know scripting regarding this (20 Replies)
Discussion started by: manigrover
20 Replies
6. Shell Programming and Scripting
Hi,
I have a file (sorted by sort) with 8 tab delimited columns. The first column contains duplicated fields and I need to merge all these identical lines.
My input file:
comp100002 aaa bbb ccc ddd eee fff ggg
comp100003 aba aba aba aba aba aba aba
comp100003 fff fff fff fff fff fff fff... (5 Replies)
Discussion started by: falcox
5 Replies
7. Shell Programming and Scripting
Hi all,
I have huge a tab-delimited file with the following format and I want to remove the duplicates according to their frequency based on Column2 and Column3.
Column1 Column2 Column3 Column4 Column5 Column6 Column7
1 user1 access1 word word 3 2
2 user2 access2 ... (10 Replies)
Discussion started by: corfuitl
10 Replies
8. Shell Programming and Scripting
Dear All,
I really enjoy your help or suggestion for resolving an issue.
Briefly, I have a file like this:
a b c
a d e
f g h
k g h
x y z
If the first column has the same ID, for example a, just remove it.
The output should be this:
f g h
k g h
x y z
I was thinking to do it... (11 Replies)
Discussion started by: giuliangiuseppe
11 Replies
9. Shell Programming and Scripting
Hi folks,
I've got a csv file called test.csv
Column A Column B
Apples 1900
Apples 1901
Pears 1902
Pears 1903I want to count and keep duplicates in the first column. Desired output
Column A Column B Column C
Apples 2 1900
Apples ... (5 Replies)
Discussion started by: pshields1984
5 Replies
10. Shell Programming and Scripting
Hi Experts,
Please bear with me, i need help
I am learning AWk and stuck up in one issue.
First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique.
Second point : For... (1 Reply)
Discussion started by: as7951
1 Replies
LEARN ABOUT FREEBSD
brand.4th
BRAND.4TH(8) BSD System Manager's Manual BRAND.4TH(8)
NAME
brand.4th -- FreeBSD ASCII art boot module
DESCRIPTION
The file that goes by the name of brand.4th is a set of commands designed to draw the ASCII art BSD brand above the boot loader menu. The
commands of brand.4th by themselves are not enough for most uses. Please refer to the examples below for the most common situations, and to
loader(8) for additional commands.
Before using any of the commands provided in brand.4th, it must be included through the command:
include brand.4th
This line is present in the default /boot/menu.rc file, so it is not needed (and should not be re-issued) in a normal setup.
The commands provided by it are:
draw-brand Draws the BSD brand.
The brand that is drawn is configured by setting the loader_brand variable in loader.conf(5) to one of ``fbsd''
(the default) or ``none''.
The position of the logo can be configured by setting the loader_brand_x and loader_brand_y variables in
loader.conf(5). The default values are 2 (x) and 1 (y).
The environment variables that effect its behavior are:
loader_brand
Selects the desired brand in the beastie boot menu. Possible values are: ``fbsd'' (default) or ``none''.
loader_brand_x
Sets the desired column position of the brand. Default is 2.
loader_brand_y
Sets the desired row position of the brand. Default is 1.
FILES
/boot/loader The loader(8).
/boot/brand.4th brand.4th itself.
/boot/loader.rc loader(8) bootstrapping script.
EXAMPLES
Set FreeBSD brand in loader.conf(5):
loader_brand="fbsd"
SEE ALSO
loader.conf(5), loader(8)
HISTORY
The brand.4th set of commands first appeared in FreeBSD 9.0.
AUTHORS
The brand.4th set of commands was written by Devin Teske <dteske@FreeBSD.org>.
BSD
May 18, 2011 BSD