Sponsored Content
Top Forums Shell Programming and Scripting Compare 2 folders to find several missing files among huge amounts of files. Post 302420083 by jiapei100 on Monday 10th of May 2010 01:51:15 PM
Old 05-10-2010
Compare 2 folders to find several missing files among huge amounts of files.

Hi, all:

I've got two folders, say, "folder1" and "folder2".
Under each, there are thousands of files.

It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command.

However, if I change the above question a bit:

I've got two folders, say, "folder1" and "folder2",
under each and its subfolders, and it's subsubfolders, etc.. recursively,
there are thousands of files.

Now, I would like to compare how many files missing in each of "folder1" and "folder2", and it's better that the shell command might be able to give me some indication about where the missing files are???

Best Regards
JIA
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare 2 huge files wrt to a key using awk

Hi Folks, I need to compare two very huge file ( i.e the files would contain a minimum of 70k records each) using awk or sed. The comparison needs to be done with respect to a 'key'. For example : File1 ********** 1234|TONY|Y75634|20/07/2008 1235|TINA|XCVB56|30/07/2009... (13 Replies)
Discussion started by: Ranjani
13 Replies

2. Shell Programming and Scripting

Compare the checksum of files in 2 different folders

Hi I have 2 different folders on different machines. they are supposed to be same but some time for unknown reason they are not. then we have to generate a report for files which are not matching. I was doing as below - cd folder1 find . -type f | sort | cksum >1.txt cd folder2 find .... (7 Replies)
Discussion started by: reldb
7 Replies

3. Shell Programming and Scripting

Compare files in two folders and delete missing ones

I do not know much about shell scripting so I am at a loss here. If someone can help me, that would be great! I have two directories /dir1 /dir2 I need to delete all files from /dir1 and that does not have a correspondent file in /dir2. It should NOT check file suffixes in /dir2 . Why?... (20 Replies)
Discussion started by: kaah
20 Replies

4. Shell Programming and Scripting

How to compare files in two folders using cmp?

i recently copied 400GB of data from a NTFS drive to a ext4 drive. I want to verify that the data is 100% identical to the original. I wanted to use cmp but it only does two files. The directory that was copied contains many subdirectories and all sorts of files (not just text). So I guess... (5 Replies)
Discussion started by: fuzzylogic25
5 Replies

5. Shell Programming and Scripting

Format & Compare two huge CSV files

I have two csv files having 90K records each & each row has around 50 columns.Lets say the file names are FILE1 and FILE2. I have to compare both the files and generate a new file that has rows from FILE2 if it differs. FILE1 ----- 2001,"John",25,19901130,21211.41,Unix Forum... (3 Replies)
Discussion started by: Sheel
3 Replies

6. Shell Programming and Scripting

Linux Script to compare two folders and copy missing files

Hi, I need help in shell scripting. If someone can help me, that would be great! Problem. I want Linux Script to compare two folders and copy missing files. Description. I have two directories /dir1 /dir2 I need to copy all distinct/new/unique/missing files from /dir1 and that... (1 Reply)
Discussion started by: S.Praveen Kumar
1 Replies

7. Shell Programming and Scripting

Compare two files and get only missing names

I need to compare two files (oldfile1 & newfile). Need to ignore the values which are present in both files. At the same time, i need to get only records in new file. Tried using Join -v1 -v2 oldfile1 newfile (suspect it has not worked as expected). could anyone of you please help me here. (5 Replies)
Discussion started by: Selva_2507
5 Replies

8. Shell Programming and Scripting

Split a folder with huge number of files in n folders

We have a folder XYZ with large number of files (>350,000). how can i split the folder and create say 10 of them XYZ1 to XYZ10 with 35,000 files each. (doesnt matter which files go where). (12 Replies)
Discussion started by: AlokKumbhare
12 Replies

9. Shell Programming and Scripting

Compare 2 files and find missing fields awk

Hello experts! I have 2 files. file1 is a list file containing uniquely names. e.g.: name1 number number name2 number number name5 number number name10 number number ... file2 is a data file arbitrary containing the names of file1 in paragraphs separated by "10" e.g. name4 ... (3 Replies)
Discussion started by: phaethon
3 Replies

10. Shell Programming and Scripting

Script to compare files in 2 folders and delete the large file

Hello, my first thread here. I've been searching and fiddling around for about a week and I cannot find a solution.:confused: I have been converting all of my home videos to HEVC and sometimes the files end up smaller and sometimes they don't. I am currently comparing all the video files... (5 Replies)
Discussion started by: Josh52180
5 Replies
FLIST(1)                                                             [nmh-1.5]                                                            FLIST(1)

NAME
flist, flists - list the number of messages in given sequence(s) SYNOPSIS
flist [+folder1 [+folder2 ...]] [-sequence name1 [-sequence name2 ...]] [-all | -noall] [-showzero | -noshowzero] [-recurse | -norecurse] [-fast | -nofast] [-alpha | -noalpha] [-version] [-help] flists is equivalent to flist -all DESCRIPTION
Flist is used to search a list of folders and display the number of messages in these folders that are in a given sequence or set of sequences (for example the "unseen" sequence). This is especially useful if you use some mechanism such as slocal or procmail (typically in conjunction with rcvstore) to pre-sort your mail into different folders before you view it. By default, the command flist will search the current folder for the given sequence or sequences (usually "unseen"). If (possibly multi- ple) folders are specified on the command line with +folder, then all these folders are searched for the given sequence(s). Flist will display for each folder searched, the number of messages in each of the specified sequences, and the total number of messages. The option -sequence is used to specify the name of a sequence in which to search for. This option may be used multiple times to specify multiple sequences. If this is not given, then the default is to search for all the sequences specified by the "Unseen-Sequence" profile component. For more details about sequences, read the mh-sequence(5) man page. Typically, flist will produce a line for each sequence, for every folder that is searched, even those which do not contain any messages in the given sequence. Specifying -noshowzero will cause flist to print only those folder/sequence combinations such the folder has a non- zero number of messages in the given specified sequence. If -recurse is given, then for each folder that is search, flist will also recursively descend into those folders to search subfolders for the given sequence. If -fast is given, only the names of the folders searched will be displayed, and flist will suppress all other output. If this option is used in conjunction with -noshowzero, then flist will only print the names of those folders searched that contain messages in in at least one of the specified sequences. Multiple Folders If the option -all is given (and no folders are specified with +folder), then flist will search all the folders in the top level of the users nmh directory. These folders are all preceded by the read-only folders, which occur as "atr-cur-" entries in the user's nmh context. An example of the output of flist -all is: /work/Mail has 5 in sequence unseen (private); out of 46 inbox+ has 10 in sequence unseen ; out of 153 junklist has 0 in sequence unseen ; out of 63 postmaster has 1 in sequence unseen ; out of 3 The "+" after inbox indicates that it is the current folder. The "private" flag indicates that the given sequence for that folder is private. See the mh-sequence(5) man page for details about private sequences. If the option -all and +folder are both specified, then flist will search this folder, and all its first level subfolders for the given sequence. You may specify multiple folders in this way. If flist is invoked by a name ending with "s" (e.g. flists), then the switch -all is assumed by default. The sorting order for the listing is alphabetical (with -alpha), or in a priority order defined by the "Flist-Order" profile entry (with -noalpha). Each item in the "Flist-Order" is a folder name or a folder name pattern that uses * to match zero or more characters. Longer matching patterns have precedence over shorter matching patterns. For example: Flist-Order: personal petproject mh* * admin *junk This order puts a few interesting folders first, such as those with mail addressed to you personally, those about a pet project, and those about mh-related things. It places uninteresting folders at the end, and it puts everything else in the middle in alphabetical order. FILES
$HOME/.mh_profile The user profile PROFILE COMPONENTS
Path: To determine the user's nmh directory mh-sequences: File that contains public sequences Unseen-Sequence: The name of the unseen message sequence Flist-Order: To sort folders by priority SEE ALSO
folder(1), rcvstore(1), slocal(1), mh-sequence(5) DEFAULTS
`-sequence' defaults to Unseen-Sequence profile entry `-showzero' `-noall' `-norecurse' `-noalpha' `-nofast' CONTEXT
If +folder is given, it will become the current folder. If multiple folders are given, the last one specified will become the current folder. MH.6.8 11 June 2012 FLIST(1)
All times are GMT -4. The time now is 03:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy