sorting using substring


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sorting using substring
# 1  
Old 03-24-2011
sorting using substring

Dear all,

I want to list all duplicate files that are present in all subdirectories. I used the following the command and it worked fine.
Code:
find . -type f -print | sort -i

This is giving a sample output as follows:
============================
Code:
./out
./cas/catch.dat
./cas/File1.dat
./baab/bumber.dat
./baab/File1.dat
./uday/hahah.dat
./uday/samp/CAS/test.txt
./uday/samp/File1.dat
./uday/bhas/File1.dat
./uday/File1.dat
./aaaa/sample.dat
./aaaa/File1.dat
./aaaa/a12/File1.dat

============================
How ever i want to sort this out such that all the duplicate file are listed in a single line. I mean i want to sort based on substring in each line

I want to sort based on substring starting from last occurrence of "/" to the end of the line. is there a staright forward way to do this

thanks in advance
uday

Last edited by Franklin52; 03-24-2011 at 04:09 AM.. Reason: Please use code tags
# 2  
Old 03-24-2011
Try...
Code:
find . -type f -print | awk -F/ '{print $NF, $0}' file1|sort -i

# 3  
Old 03-24-2011
Hi tvsubhaskar,

If you only want to show in a single line all files from the sample output:
Code:
awk '{sub(".*/",""); sub("^","/");printf "%s ", $0}' inputfile
/out /catch.dat /File1.dat /bumber.dat /File1.dat /hahah.dat /test.txt /File1.dat /File1.dat /File1.dat /sample.dat /File1.dat /File1.dat

If you only want to show in a single only unique files
Code:
awk -F"/" '$NF~/\./{R[$NF]=$NF}END{ for(i in R) if(R[i]>1) printf "%s ", "/"i}' inputfile
/hahah.dat /test.txt /bumber.dat /sample.dat /File1.dat /catch.dat

Or only unique files, showing times they appear within ()
Code:
awk -F"/" '$NF~/\./{R[$NF]++}END{ for(i in R) printf "%s ", "/"i"("R[i]")"}' inputfile
/hahah.dat(1) /test.txt(1) /bumber.dat(1) /sample.dat(1) /File1.dat(7) /catch.dat(1)

Or only show duplicates files(appear more than once) with how many times appear within ()
Code:
awk -F"/" '$NF~/\./{R[$NF]++}END{ for(i in R) if(R[i]>1) printf "%s ", "/"i"("R[i]")"}' inputfile
/File1.dat(7)

Hope it helps,

Regards
# 4  
Old 03-24-2011
Code:
while read file; do echo ${file%/*} ${file##.*/} ; done < inputfile | sort -k2 | awk '{printf a==$2?$1"/"$2" ":"\n"$1"/"$2;}{a=$2}'

# 5  
Old 03-24-2011
I guess you want to show duplicate files listed in the same line.
Code:
$ awk -F [/] '{++a[$NF];c[$NF]=c[$NF] " " $0} END{for(i in c){ if (a[i]>1){print c[i],a[i]}}}' input
 ./aaaa/sample.dat ./sample.dat 2
 ./baab/bumber.dat ./test/tes/baab/bumber.dat ./baab/bumber.dat 3
 ./cas/File1.dat  ./baab/File1.dat ./uday/samp/File1.dat ./uday/bhas/File1.dat  ./uday/File1.dat ./aaaa/File1.dat ./aaaa/a12/File1.dat 7

Hope this helps.

Last edited by Franklin52; 03-24-2011 at 05:08 AM.. Reason: Please use code tags, thank you
# 6  
Old 04-07-2011
find . -type f | sort -t"/" +2
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Substring

Hi All, In ksh, am trying to get a substring stuff done. Not sure where the problem is.. can you guys guide me on this... for instance, var1=41, and var2=4175894567, then i want to know whether var2 starts with var1.. var1 and var2 can be of any length.. VAR1=41 VAR2=419068567777... (6 Replies)
Discussion started by: nram_krishna@ya
6 Replies

2. Shell Programming and Scripting

substring

I have a string '<Hi>abc</Hi>" How to print "abc" (6 Replies)
Discussion started by: sandy1028
6 Replies

3. UNIX for Dummies Questions & Answers

Substring

Hi I use the below cmd to get the list of files that are modified than <temp> file in the <path> diretory cmd:find <path> -name '*.zip' -type f -newer <temp> -print i am getting all the list of files that are new or modified, with abs path, i want to copy all of these files to a... (3 Replies)
Discussion started by: Naveen_5960
3 Replies

4. UNIX for Dummies Questions & Answers

substring

hi , I have complete path Sample: /pkgs/Incoming/Completed/abc123_xyz.zip /pkgs/Incoming/Completed/12_abcxyz.zip /pkgs/Incoming/Completed/qwabcxyz.zip i just need , so that i can copy these files to different directory, abc123_xyz.zip 12_abcxyz.zip qwabcxyz.zip (5 Replies)
Discussion started by: Naveen_5960
5 Replies

5. UNIX for Dummies Questions & Answers

Need help with substring

I need to check the occurrence of one string within another. code ******************** if ;then do something done ******************** Thanks (7 Replies)
Discussion started by: w020637
7 Replies

6. Shell Programming and Scripting

Substring HELP!

Hi, I am trying to do something which I thought was very simple but still being a beginner, has proved not to be. Input: val1 val2 val3 val4 val5 val6 . . . etc Desired Output: Every row in which value of val6 is a number starting with 0.0 or contains a capital E. The input... (2 Replies)
Discussion started by: awknerd
2 Replies

7. Shell Programming and Scripting

substring

Dear All, i have a file that contains, FROM_DATE: 06-08-2007 00:00:00 TO_DATE: 06-08-2007 23:59:59 Total number of lines: 6874154 in another file,the contain is, FROM_DATE: 06-08-2007 00:00:00 Total number of lines: 874154 alltime i want to find the particular string... (4 Replies)
Discussion started by: panknil
4 Replies

8. UNIX for Dummies Questions & Answers

substring

Hi, I have a value of a filepath in a variable DATAFILE with value as "customtop/gpsore37/gepspo/1.0/bin/ashoka.csv ". Now i want the value of last 4 charcters in to another variable. That is EXTENSION = .csv How can i do this in Shell scripting Thanks in advance Alla Kishore (8 Replies)
Discussion started by: alla.kishore
8 Replies

9. Shell Programming and Scripting

Getting a substring

This is probably pretty simple butI'm not sure how to best go about it. If I have FILE="myBigLongFileName_1.xls" FILE_PREFIX=`echo $FILE| cut -d"." -f1` # that gives "myBigLongFileName_1" All i want to do now is chop the "_1" from the end of $FILE_PREFIX Any ideas anyone? (3 Replies)
Discussion started by: djkane
3 Replies

10. Shell Programming and Scripting

getting a substring

Hi, I have several lines like this ones: 123456789abcde /aa/bb/123456_$data.log 123456789abcde /aa/bb/123456_not_a_data_log 987654321ab /aa/bb/xpto123456_$data.log ... How do I get into a variable the value "/aa/bb/123456_$data.log" , searching only for the beggining and ending... (3 Replies)
Discussion started by: Scarlos
3 Replies
Login or Register to Ask a Question