Sponsored Content
Top Forums Shell Programming and Scripting sed working as uniq 1st word only Post 302629089 by motorcek on Tuesday 24th of April 2012 09:46:10 AM
Old 04-24-2012
sed working as uniq 1st word only

Hello, everyone.
I am having trouble figuring out sed command which emulates uniq. The task I want to do is that 2 consecutive lines in file should be considered the same using the first word only. Example:
Code:
cat tmp.txt
ddd eee
aaa bbb ccc
ddd eee fff
asd fdd
asd fdd bbb
aaa bbb
asd fgh yyy
asd fgh

followed by:
Code:
sort -n |sed '$!N; /^\(.*\)\n\1$/!P; D'

produces output:
Code:
aaa bbb
aaa bbb ccc
asd fdd
asd fdd bbb
asd fgh
asd fgh yyy
ddd eee
ddd eee fff

But I need only one of the "same lines" (preferably the 2nd)
I couldn't figure out the sed magic to get it working, any ideas anyone?
Thank you very much.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

whole word substitution in SED

I am trying to substitute something with sed and what I want is to substitute a whole word and not part of a word. ie sed 's/class/room/g' filename will substitute both class and classes into room and roomes which is not what i want Grep for instance can use the -w option or <> grep -w... (7 Replies)
Discussion started by: gikay01
7 Replies

2. Shell Programming and Scripting

Whole word with sed?

Hi experts, need a help in SED file=counter.c module=abcd i=http://svn.company.com/svn/${module}/trunk/counter/${file} When i do echo ${i}| sed "s|http://svn.company.com/svn/${module}/trunk/||g"| sed "s|${file}||g" it results in ounter.c (5 Replies)
Discussion started by: ganga.dharan
5 Replies

3. Shell Programming and Scripting

uniq not working, so any alternate?

Hi, I have these two files . file1 /home/prog/bug/perl /home/prog/bug/ant /home/prog/bug/make /home/prog/bug/gen /home/prog/bug/tiff file2 /home/prog/bug/make /home/prog/bug/gen i want a output file which should contain file1-file2 (2 Replies)
Discussion started by: debu182
2 Replies

4. UNIX for Dummies Questions & Answers

grep for word not working

Hi All..I need a help i am trying to find a word using below script whereas the word exists in my file nitin.txt as a directory but still i am getting "word not found" output..Your suggestions welcomed.: #to check for existence of nitin #!/bin/bash cd /apps/uat1/deploy/app ls -lrt >... (4 Replies)
Discussion started by: nattynitin
4 Replies

5. UNIX for Dummies Questions & Answers

How to print line starts with specific word and contains specific word using sed?

Hi, I have gone through may posts and dint find exact solution for my requirement. I have file which consists below data and same file have lot of other data. <MAPPING DESCRIPTION ='' ISVALID ='YES' NAME='m_TASK_UPDATE' OBJECTVERSION ='1'> <MAPPING DESCRIPTION ='' ISVALID ='NO'... (11 Replies)
Discussion started by: tmalik79
11 Replies

6. UNIX for Dummies Questions & Answers

Find EXACT word in files, just the word: no prefix, no suffix, no 'similar', just the word

I have a file that has the words I want to find in other files (but lets say I just want to find my words in a single file). Those words are IDs, so if my word is ZZZ4, outputs like aaZZZ4, ZZZ4bb, aaZZZ4bb, ZZ4, ZZZ, ZyZ4, ZZZ4.8 (or anything like that) WON'T BE USEFUL. I need the whole word... (6 Replies)
Discussion started by: chicchan
6 Replies

7. Shell Programming and Scripting

Filtering data using uniq and sed

Hello, Does anyone know an easy way to filter this type of file? I want to get everything that has score (column 2) 100.00 and get rid of duplicates (for example gi|332198263|gb|EGK18963.1| below), so I guess uniq can be used for this? gi|3379182634|gb|EGK18561.1| 100.00... (6 Replies)
Discussion started by: narachaid
6 Replies

8. Shell Programming and Scripting

Replacing a particular word with another word in all the xml's under a particular directory with sed

Hi Folks, Could you please advise what will be the SED command to replace a word in all xml's under a particular directory for example let say I rite now at the following below location $ cd /ter/rap/config now under config directory there will be lots of xml file , now my objective is to... (1 Reply)
Discussion started by: punpun66
1 Replies

9. Shell Programming and Scripting

awk - why this is not working? trying next word!

Hi Experts, Can you please advise , why I am not able to make it work, or why this is not working: I spent quite a lot of time on this figuring out , but not working, file : This is a test file thanks for your reply This is another file again Have a nice day this is a small file... (3 Replies)
Discussion started by: rveri
3 Replies

10. Shell Programming and Scripting

awk word boundaries not working

Hi, I am trying below code but the word boundaries not seem to be working. What am I doing incorrectly? echo " ECHO " | awk '{ q="ECHO" ; if ( $0 ~ /\bq\b/) print "HELLO" ; }' OR echo " ECHO " | awk '{ q="ECHO" ; if ( $0 ~ /\b'$q'\b/) print "HELLO" ; }' Or echo " ECHO " | awk... (6 Replies)
Discussion started by: ahmedwaseem2000
6 Replies
UNIQ(1) 						    BSD General Commands Manual 						   UNIQ(1)

NAME
uniq -- report or filter out repeated lines in a file SYNOPSIS
uniq [-cdu] [-f fields] [-s chars] [input_file [output_file]] DESCRIPTION
The uniq utility reads the standard input comparing adjacent lines, and writes a copy of each unique input line to the standard output. The second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are not adjacent, so it may be necessary to sort the files first. The following options are available: -c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space. -d Don't output lines that are not repeated in the input. -f fields Ignore the first fields in each input line when doing comparisons. A field is a string of non-blank characters separated from adja- cent fields by blanks. Field numbers are one based, i.e. the first field is field one. -s chars Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the first chars characters after the first fields fields will be ignored. Character numbers are one based, i.e. the first character is character one. -u Don't output lines that are repeated in the input. If additional arguments are specified on the command line, the first such argument is used as the name of an input file, the second is used as the name of an output file. The uniq utility exits 0 on success, and >0 if an error occurs. COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation. SEE ALSO
sort(1) STANDARDS
The uniq utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible. BSD
January 6, 2007 BSD
All times are GMT -4. The time now is 09:46 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy