want unique data


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting want unique data
# 1  
Old 08-14-2009
want unique data

Hi All,

I have a question again, In the code below in want to display data that is unique, there are many directories in which i am going and parsing a .su file and getting a string out, if i do ls $i | grep .su | $AWK -F"_" '{print $1} | uniq inside the for loop it will not work.

What do i do?

Thanks
Adi

Code:
#!/bin/sh

ECHO=/bin/echo
FIND=/bin/find
AWK=/bin/awk

ROOT_PATH=/net/icebox/vol/local_images/spins
STREAM_PATH=$ROOT_PATH/7.1

SHARE_ENTITY_PATH=`$FIND $STREAM_PATH . -depth -name ShareableEntities`
#$ECHO $SHARE_ENTITY_PATH

for i in $SHARE_ENTITY_PATH
do
ls $i | grep .su |  $AWK -F"_" '{print $1}'
done

# 2  
Old 08-14-2009
How about...

Code:
for i in $SHARE_ENTITY_PATH
do
  ls $i | grep .su |  $AWK -F"_" '{print $1}'
done | sort -u

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

2. UNIX for Dummies Questions & Answers

Print unique lines without sort or unique

I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies

3. Shell Programming and Scripting

Change unique file names into new unique filenames

I have 84 files with the following names splitseqs.1, spliseqs.2 etc. and I want to change the .number to a unique filename. E.g. change splitseqs.1 into splitseqs.7114_1#24 and change spliseqs.2 into splitseqs.7067_2#4 So all the current file names are unique, so are the new file names.... (1 Reply)
Discussion started by: avonm
1 Replies

4. Shell Programming and Scripting

AWK, Perl or Shell? Unique strings and their maximum values from 3 column data file

I have a file containing data like so: 2012-01-02 GREEN 4 2012-01-02 GREEN 6 2012-01-02 GREEN 7 2012-01-02 BLUE 4 2012-01-02 BLUE 3 2012-01-02 GREEN 4 2012-01-02 RED 4 2012-01-02 RED 8 2012-01-02 GREEN 4 2012-01-02 YELLOW 5 2012-01-02 YELLOW 2 I can't always predict what the... (4 Replies)
Discussion started by: rich@ardz
4 Replies

5. UNIX Desktop Questions & Answers

Extracting only unique data between two columns

:wall:Hi there, I am trying to extract/filter a unique data between specific columns from a tab deliminated file, that has a number of columns: input file as follow: 5 rs1 70 A C 7 1 1 Blue 5 rs9 66 A E ... (2 Replies)
Discussion started by: houkto
2 Replies

6. Shell Programming and Scripting

Generate unique user id with each addition of data

As I add new data to database.txt I want to generate a unqiue User Id. This is my current solution. echo $RANDOM:$lname:$fname >> database.txt But I seem to have problems when I try and use $RANDOM for formatting like so: grep "^4539" database.txt | awk -F":" '{print "User... (4 Replies)
Discussion started by: yonkers062986
4 Replies

7. Shell Programming and Scripting

Perl Data Structure - Non unique values

I have the perl data structure and what i need to do is find all values in @{$extractColumns{'2'}{'D'}} which are not there in @{$extractColumns{'2'}{'M'}} but seems like i need to put a flag somewhere and i messed up foreach my $order (keys %extractColumns) { foreach my $value... (2 Replies)
Discussion started by: dinjo_jo
2 Replies

8. Shell Programming and Scripting

get part of file with unique & non-unique string

I have an archive file that holds a batch of statements. I would like to be able to extract a certain statement based on the unique customer # (ie. 123456). The end for each statement is noted by "ENDSTM". I can find the line number for the beginning of the statement section with sed. ... (5 Replies)
Discussion started by: andrewsc
5 Replies

9. Shell Programming and Scripting

What some ideas with sort and get unique data

Hi all, I am writing a script where i can parse through the directory and get common string in two directories i get. The command below SUN_PLATFORM=`$FIND $STREAM_PATH . -depth -name ShareableEntities | $AWK -F"/" '{if($10 ~ /sun5/) print $0}'` gives the following output:- ... (1 Reply)
Discussion started by: asirohi
1 Replies

10. Shell Programming and Scripting

Shell script to check the unique numbers in huge data

Friends, I have to write a shell script,the description is---- i Have to check the uniqueness of the numbers in a file. A file is containing 200thousand tickets and a ticket have 15 numbers in asecending order.And there is a strip that is having 6 tickets that means 90 numbers.I... (7 Replies)
Discussion started by: namishtiwari
7 Replies
Login or Register to Ask a Question