Shell Programming and Scripting

Hi All

I've googled around for this and can't see a way of doing it.

I have a file that contains a number of records that are layed out something like the following.

/path/to/directory/that/contains/a/file/I/need/filename.pdf

The path itself can vary both in terms of the names and the number of subdirectories before I get to the file.

I realise I can use basename to extract the filename and do a sort but what I want to do is to sort the list using the filename as the key and then find duplicate filenames and then identify the paths of the conflicting filename.

I also want to be able to do a quick and dirty compare from the command line something like

Code:

cat /the/input/file | sed -e 's\: delete up to the last forward slash in the path to leave just the filename ::' | sort | uniq

To allow me to do a fast visual confirmation on if there are duplicates in the file. I can't see a way to get sed to delete up to the last instance of a specific character on a line (basename won't work from the command line as there's multiple records in the input file so it needs a loop).

All ideas gratefully received.

This might help:

code:
echo "/path/to/directory/that/contains/a/file/I/need/filename.pdf" | awk -F"/" ' { print $NF } '

Use this awk syntax for the rest of the lines.

Thanks, what's the $NF variable? I don't think I've ever used it before.

edit
Ignore, I've just seen it's the number of fields and therefore by definition the last field, nice idea, thanks.

an awk one:

Code:

 
awk -F"/" '{ a[$NF]++;b[$NF]=$0"\n"b[$NF]} END{ for(i in a ) if (a[i]>1) print i"-->",a[i],"\n"b[i]}' input_file

Not sure of your requirement but I hope the above helps!

Code:

while read rec
do
  echo "$(basename $rec)  $rec"
done < listfile | sort | awk '{print $2}'

start with that

yet another [g]awker...

Code:

awk -F/ '{f[$NF]=sprintf("%s",f[$NF]?f[$NF]"\n"$0:$0)}END{for(i in f) print f[i]}' file

Jay

Just wanted a further follow up to thank you for this.

I was using basename a couple of other places in the script and replacing with this is going to strip about an hour off the script runtime.

Regards