08-03-2011
Unique files in a given directory
I keep all my files on a NAS device and copy files from it to usb or local storage when needed. The bad part about this is that I often have the same file on numerous places. I'd like to write a script to check if the files in a given directory exist in another.
An example:
say I have a directory called "Stuff" and another called "AllMyFiles" I want a script to check the directory "Stuff" to tell me which files already exist in "AllMyFiles". The way I currently do this is to use fdupes to create a list of all duplicate files in both directories, then use grep to spot which duplicate files are in "Stuff". The drawback to this is that it checks all files for any duplicates, including those in "AllMyFiles", so fdupes takes a long time. Is there a clever way of avoiding this and checking only the files in "Stuff" to see if a duplicate exists for it in "AllMyFiles"?
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi, I know that inode for each file is unique, but is it the for the directory? So far I found different directories has the same inode nubmer when you do ls -i, could some one explain why? Thanks a lot. (9 Replies)
Discussion started by: nj302
9 Replies
2. UNIX for Dummies Questions & Answers
here i have two files:
file 1
1
2
3
4
5
5
6
7
8
9
file 2
4
5
6
6
8
8 (6 Replies)
Discussion started by: i.scientist
6 Replies
3. Shell Programming and Scripting
Ok, so I just got charged with the task of deleting some 300 user folders in a FTP server to free up some space. I managed to grep and cut the list of user folders to delete into a list of one user folder per line.
Example:
bob00
jane01
sue03
In the home folder, there are folders a-z, and... (5 Replies)
Discussion started by: b4sher
5 Replies
4. Shell Programming and Scripting
Hi,
I have a directory with Multiple subdirectories and 1000s of pictures (jpg) in each directory. The problem is that each directory has a 001.jpg in them. I want to append a unique name (the directory_name)would be fine. and then move them to one main backup directory once they have been... (1 Reply)
Discussion started by: kmaq7621
1 Replies
5. UNIX for Dummies Questions & Answers
hi
i have used comm -13 <(sort 1.txt) <(sort 2.txt) option to get the unique lines that are present in file 2 but not in file 1. but some how i am getting the entire file 2. i would expect few but not all uncommon lines fro my dat. is there anything wrong with the way i used the command?
my... (1 Reply)
Discussion started by: anurupa777
1 Replies
6. Shell Programming and Scripting
Hello,
I`m a complete newbie to coding, please help with this problem.
I have multiple files in a directory, I have to loop through the contents of each file and extract number of unique isoforms in that file. Each file is tab delimited and only the line with the first parent (column 3)... (1 Reply)
Discussion started by: ritakadm
1 Replies
7. Shell Programming and Scripting
In a incoming folder i have list of files like below,i want to pick the unique files to process the job. if same file contain more than one then it should pick latest date modified file to process.
drwxrwsrwx 2 n308799 infagrp 256 May 20 17:42 Final_Working
drwxrwsrwx 2... (1 Reply)
Discussion started by: katakamvivek
1 Replies
8. Shell Programming and Scripting
I am trying to add a unique identifier to two file extensions .bam and .vcf in a directory located at /home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.
The identifier is in $2 of the input file. What the code below is attempting to do is strip off the last portion... (21 Replies)
Discussion started by: cmccabe
21 Replies
9. Shell Programming and Scripting
Given a directory containing say a few thousand files,
please output a list of all the names of the files in the directory that are exactly the same, i.e. have the same contents.
func(a_directory_name) output -> {“matches”: , ... ]}
e.g. func(“/home/my/files”) where the directory... (7 Replies)
Discussion started by: anuragpgtgerman
7 Replies
10. UNIX for Beginners Questions & Answers
I have a directory of files, I can show the number of lines in each file and order them from lowest to highest with:
wc -l *|sort
15263 Image.txt
16401 reference.txt
40459 richtexteditor.txt
How can I also print the number of unique lines in each file?
15263 1401 Image.txt
16401... (15 Replies)
Discussion started by: spacegoose
15 Replies
LEARN ABOUT NETBSD
makedev.local
MAKEDEV.LOCAL(8) BSD System Manager's Manual MAKEDEV.LOCAL(8)
NAME
MAKEDEV.local -- create site-specific device special files
SYNOPSIS
MAKEDEV.local [-fMsu] [-m mknod] [-p pax] [-t mtree] {all | site-specific-argument} [...]
DESCRIPTION
MAKEDEV.local is used to create site-specific device special files. Each argument may be the word all or a site-specific argument. By
default, there are no valid site-specific arguments, and the all argument has no effect; This may be changed by editing the script.
The script is in /dev/MAKEDEV.local. Devices are created in the current working directory; in normal use, MAKEDEV.local should be invoked
with /dev as the current working directory.
Supported options for MAKEDEV.local are the same as for MAKEDEV(8).
FILES
/dev special device files directory
/dev/MAKEDEV script that invokes MAKEDEV.local with the all argument.
/dev/MAKEDEV.local script described in this man page
SEE ALSO
config(1), intro(4), MAKEDEV(8), mknod(8)
HISTORY
The MAKEDEV.local command appeared in 4.2BSD. Handling of the same command line options as MAKEDEV(8), and the use of MAKEDEV(8) as a func-
tion library, was added in NetBSD 5.0.
NOTES
The relationship between MAKEDEV.local and MAKEDEV(8) is complex:
o If MAKEDEV(8) is invoked with the all or local argument, then it will invoke MAKEDEV.local as a child process, with options similar to
those that were originally passed to MAKEDEV(8), and with the all argument.
o MAKEDEV.local uses shell functions defined in MAKEDEV(8). This is done by loading MAKEDEV(8) using the shell ``.'' command, with the
MAKEDEV_AS_LIBRARY variable set (to inform MAKEDEV(8) that it should behave as a function library, not as an independent program).
BSD
August 6, 2011 BSD