Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Do Not Output Duplicates
Top Forums Shell Programming and Scripting Do Not Output Duplicates Post 302898759 by bartus11 on Wednesday 23rd of April 2014 05:29:19 PM
Old 04-23-2014
Try:
Code:
find $SRC -type f -name *.emlx | 	
  while read FILE 
  do 	   
    awk '/^From:/ && gsub(/.*<|>.*/,x)' $FILE 	
  done | sort | uniq > ~/Desktop/output.txt 
echo "complete"

This User Gave Thanks to bartus11 For This Post:
sudo
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Reading Input from File and Duplicates Output

Greetings to all, I would like to read input from a file and make duplications from it with Linux shell. For e.g. Input file ----------- ABC ABB ABA ------------------------------- Output file ------------ ABC ABC ABC ABB ABB (6 Replies)
Discussion started by: noelcantona
6 Replies

2. HP-UX

getting duplicates

how to get duplicates in a file containing data in columns using command or scripting? (4 Replies)
Discussion started by: megh
4 Replies

3. Shell Programming and Scripting

Non Duplicates

I have input file like below. I00789524 0213 5212 D00789524 0213 5212 I00778787 2154 5412 The first two records are same(Duplicates) except I & D in the first character. I want non duplicates(ie. 3rd line) to be output. How can we get this . Can you help. Is there any single AWK or SED... (3 Replies)
Discussion started by: awk_beginner
3 Replies

4. UNIX for Dummies Questions & Answers

Duplicates

Hi, How to eliminate the duplicate values in unix? I have a excel file which contains duplicate values. Need to use this in a script. Thanks in advance. (3 Replies)
Discussion started by: venkatesht
3 Replies

5. AIX

Duplicates in bootlist

Hello, I'm moving some disks from the rootvg on AIX 5.3. # replacepv hdiskOLD hdiskNEW I have for example hdisk12 and hdisk13 with hd5 (boot) LV and want to move hdisk13 So 1st I'm excluding it from the bootlist: # bootlist -om normal hdisk12 then # replacepv hdisk13... (7 Replies)
Discussion started by: emoubi
7 Replies

6. Shell Programming and Scripting

Help in removing duplicates

I have an input file abc.txt with info like: abcd rateuse inklite robet rateuse abcd I need to remove duplicates from the file (eg: abcd,rateuse) from the file and need to place the contents in same file abc.txt if needed can be placed in another file. can anyone help me in this :( (4 Replies)
Discussion started by: rkrish
4 Replies

7. Shell Programming and Scripting

Remove duplicates based on query and subject fields from blast output file

Hi all I have a blast outfile file like this : NZ_1540841_1561981 ICMP_1687819_1695946 92.59 27 2 0 12826 12852 3136 3162 0.28 38.2 NZ_1540841_1561981 ICMP_1687819_1695946 95.65 23 1 0 12268 12290 5815 5837 0.28 38.2 NZ_1540841_1561981 ICMP_3674888_3676546 82.70 185 32 0 9454 9638 11 195 6e-24 ... (2 Replies)
Discussion started by: pbioinfo
2 Replies

8. UNIX for Dummies Questions & Answers

Filtering the duplicates

Hello, I want to filter all the duplicates of a record to one place. Sample input and output will give you better idea. I am new to unix. Can some one help me on this? Input: 7488 7389 chr1.fa chr1.fa 3546 9887 chr5.fa chr9.fa 7387 7898 chrX.fa chr3.fa 7488 7389 chr1.fa chr1.fa... (2 Replies)
Discussion started by: koneru_18
2 Replies

9. Shell Programming and Scripting

sed & remove duplicates on output

sed -e '1d' -e 's/^\(]\{2\}\)-\(]\{3\}\)-\(]\{4\}\).*/"0000020\1\200\3"\,/g' abc.txt This script returns many duplicates due to the duplciates in the .txt file. i.e. ... "000002012149000060", "000002012149000064", "000002012149000064", "000002012149000064", "000002012149000064",... (9 Replies)
Discussion started by: Daniel Gate
9 Replies

10. Shell Programming and Scripting

Remove duplicates

I have a file with the following format: fields seperated by "|" title1|something class|long...content1|keys title2|somhing class|log...content1|kes title1|sothing class|lon...content1|kes title3|shing cls|log...content1|ks I want to remove all duplicates with the same "title field"(the... (3 Replies)
Discussion started by: dtdt
3 Replies

LEARN ABOUT DEBIAN

desktop-file-validate

DESKTOP-FILE-VALIDATE(1)				      General Commands Manual					  DESKTOP-FILE-VALIDATE(1)

NAME

       desktop-file-validate - Validate desktop entry files

SYNOPSIS

       desktop-file-validate [--no-warn-deprecated] [--warn-kde] FILE...

DESCRIPTION

       The desktop-file-validate program is a tool to validate desktop entry files according to the Desktop Entry specification 1.0.

       The  specification  describes  a  file format to provide information such as name, icon and description for an application. Such a file can
       then be used as an application launcher and to display the application in the applications menu.

       For information about the Desktop Entry specification, see http://freedesktop.org/wiki/Specifications/desktop-entry-spec.

       The desktop entry files are commonly called desktop files.

OPTIONS

       The following options are supported:

       --no-warn-deprecated
	      Do not warn about usage of deprecated items that were defined in previous versions of the specification.

       --warn-kde
	      Warn about usage of KDE extensions to the specification. This includes the use of the KDE Desktop Entry group, of the  ServiceTypes,
	      DocPath,	Keywords, InitialPreference, Dev, FSType, MountPoint, ReadOnly, UnmountIcon keys, or of the Service, ServiceType and FSDe-
	      vice types.

BUGS

       If you find bugs in the desktop-file-validate program, please report these on https://bugs.freedesktop.org.

SEE ALSO

       desktop-file-edit(1) desktop-file-install(1)

								  FREEDESKTOP.ORG					  DESKTOP-FILE-VALIDATE(1)
All times are GMT -4. The time now is 07:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy