Sponsored Content
Top Forums Shell Programming and Scripting Compare 2 folders to find several missing files among huge amounts of files. Post 302420096 by quirkasaurus on Monday 10th of May 2010 03:00:37 PM
Old 05-10-2010
JIA -- the first question we'd need to know is..... is the directory and file structure between the 2 directories supposed to be identical?

Presuming that the answer is "yes".... Then the answer is simple:

Code:
date > frogs

find . -print | sort > /tmp/a

/bin/rm frogs

find . -print | sort > /tmp/b

diff /tmp/a /tmp/b

This just shows the general concept. Any questions, ask away.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare 2 huge files wrt to a key using awk

Hi Folks, I need to compare two very huge file ( i.e the files would contain a minimum of 70k records each) using awk or sed. The comparison needs to be done with respect to a 'key'. For example : File1 ********** 1234|TONY|Y75634|20/07/2008 1235|TINA|XCVB56|30/07/2009... (13 Replies)
Discussion started by: Ranjani
13 Replies

2. Shell Programming and Scripting

Compare the checksum of files in 2 different folders

Hi I have 2 different folders on different machines. they are supposed to be same but some time for unknown reason they are not. then we have to generate a report for files which are not matching. I was doing as below - cd folder1 find . -type f | sort | cksum >1.txt cd folder2 find .... (7 Replies)
Discussion started by: reldb
7 Replies

3. Shell Programming and Scripting

Compare files in two folders and delete missing ones

I do not know much about shell scripting so I am at a loss here. If someone can help me, that would be great! I have two directories /dir1 /dir2 I need to delete all files from /dir1 and that does not have a correspondent file in /dir2. It should NOT check file suffixes in /dir2 . Why?... (20 Replies)
Discussion started by: kaah
20 Replies

4. Shell Programming and Scripting

How to compare files in two folders using cmp?

i recently copied 400GB of data from a NTFS drive to a ext4 drive. I want to verify that the data is 100% identical to the original. I wanted to use cmp but it only does two files. The directory that was copied contains many subdirectories and all sorts of files (not just text). So I guess... (5 Replies)
Discussion started by: fuzzylogic25
5 Replies

5. Shell Programming and Scripting

Format & Compare two huge CSV files

I have two csv files having 90K records each & each row has around 50 columns.Lets say the file names are FILE1 and FILE2. I have to compare both the files and generate a new file that has rows from FILE2 if it differs. FILE1 ----- 2001,"John",25,19901130,21211.41,Unix Forum... (3 Replies)
Discussion started by: Sheel
3 Replies

6. Shell Programming and Scripting

Linux Script to compare two folders and copy missing files

Hi, I need help in shell scripting. If someone can help me, that would be great! Problem. I want Linux Script to compare two folders and copy missing files. Description. I have two directories /dir1 /dir2 I need to copy all distinct/new/unique/missing files from /dir1 and that... (1 Reply)
Discussion started by: S.Praveen Kumar
1 Replies

7. Shell Programming and Scripting

Compare two files and get only missing names

I need to compare two files (oldfile1 & newfile). Need to ignore the values which are present in both files. At the same time, i need to get only records in new file. Tried using Join -v1 -v2 oldfile1 newfile (suspect it has not worked as expected). could anyone of you please help me here. (5 Replies)
Discussion started by: Selva_2507
5 Replies

8. Shell Programming and Scripting

Split a folder with huge number of files in n folders

We have a folder XYZ with large number of files (>350,000). how can i split the folder and create say 10 of them XYZ1 to XYZ10 with 35,000 files each. (doesnt matter which files go where). (12 Replies)
Discussion started by: AlokKumbhare
12 Replies

9. Shell Programming and Scripting

Compare 2 files and find missing fields awk

Hello experts! I have 2 files. file1 is a list file containing uniquely names. e.g.: name1 number number name2 number number name5 number number name10 number number ... file2 is a data file arbitrary containing the names of file1 in paragraphs separated by "10" e.g. name4 ... (3 Replies)
Discussion started by: phaethon
3 Replies

10. Shell Programming and Scripting

Script to compare files in 2 folders and delete the large file

Hello, my first thread here. I've been searching and fiddling around for about a week and I cannot find a solution.:confused: I have been converting all of my home videos to HEVC and sometimes the files end up smaller and sometimes they don't. I am currently comparing all the video files... (5 Replies)
Discussion started by: Josh52180
5 Replies
cdbmake(1)						      General Commands Manual							cdbmake(1)

NAME
cdbmake - create a constant database SYNOPSIS
cdbmake cdb cdb.tmp DESCRIPTION
cdbmake reads a series of encoded records from its standard input and writes a constant database to cdb. cdbmake ensures that cdb is updated atomically, so programs reading cdb never have to wait for cdbmake to finish. It does this by first writing the database to cdb.tmp and then moving cdb.tmp on top of cdb. If cdb.tmp already exists, it is destroyed. The directories con- taining cdb.tmp and cdb must be writable to cdbmake; they must also be on the same filesystem. cdbmake always makes sure that cdb.tmp is safely written to disk before it replaces cdb. If the input is in a bad format or if cdbmake has any trouble writing cdb.tmp to disk, cdbmake complains and leaves cdb alone. RECORD FORMAT Records are indexed by keys. A key is a string. cdb is structured so that another program, starting from a key, can quickly find the rel- evant record. cdbmake allows several records with the same key, although most readers take only the first record, and cdbmake slows down somewhat if there are many records with the same key. cdbmake and cdbdump(1) preserve the order of records. A record is encoded for cdbmake as +klen,dlen:key->data followed by a newline. Here klen is the number of bytes in key and dlen is the number of bytes in data. The end of data is indicated by an extra newline. For example: +3,5:one->Hello +3,7:two->Goodbye key and data may contain any characters, including colons, dashes, newlines, and nulls. Keys and data do not have to fit into memory. A database cannot exceed 4 gigabytes. cdb is portable across machines. SEE ALSO
cdbdump(1), cdbget(1), cdbstats(1) cdbmake(1)
All times are GMT -4. The time now is 02:45 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy