UNIX for Advanced & Expert Users

Hi There I have a big problem, but maybe a simple question.

I'm attempting something that is turning out to be a huge job, but maybe it could be simpler if I knew some more advanced commands or techniques.

My problem is this: I work for a company that has a primary file server with 27T capacity. I have a database keeping track of the files on this server that reports that only ~10T total when I sum the file sizes.

I need to recover the wasted space by deleting files that are not referenced in the database, and then prevent this storage leak from occuring again. I know that my predecessor has "lost" the database before on at least on occasion and restored an out of date backup.

Some details:
The file ids are stored on the file server as the mysql autoincrement id like this:

id -> padded with leading 0's and broken into 2 digit folders, each file is stored in a unique folder.
1194649 -> /ifs/data/00/00/01/19/46/49/filename.ext

So far I have made a list of all directories on the file server with `find -type d` and outputted a list of all file id's. One id per line.

I tried to get unique ids that do not exist in the database and the file server directory tree by doing:

Code:

cat file_ids_from_db file_ids_from_filesystem | sort | uniq -u > list_of_unique_ids

This didn't work, I tested a sample of the ids on the database before trying to delete the files and I found matches.

My question is this: What is the best way to do a left join between these files, only getting ids that are in the filesystem and not the database?

Another question is why am I doing it this way, is there a better solution?

Thank you in advance for any help.

-Charles

Pls check the following command

comm -13 <(sort file1) <(sort file2)

Thanks,
Thangaraju.

Thanks, I'm trying the comm command out. I'll post tomorrow to let you know if I solved this problem.