Comparing list of files in parallel


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Comparing list of files in parallel
# 1  
Old 09-18-2015
Comparing list of files in parallel

Hi everyone.

I have a list of files like:

file001
file002
file003
.
.
.
.
file385
file386
file387

There are more files than above, but I hope you understand what I'm trying to do here.

Is there a way I can create a loop to compare:
file001 with file385
file002 with file386
file003 with file387


I tried a nested loop, but all it did was compare file001 with file385, file386, file387.
Then is compared file002 with file385, file386, file387,,,etc.

I just want two sets of ranges compared in parallel

Thanks in advance.
# 2  
Old 09-18-2015
There are lots of ways... What operating system and shell are you using?
# 3  
Old 09-18-2015
bash

I've tried the followin:

Code:
Code:
a=({$bfile..$efile})
b=({$bsfile..$esfile}

for (( c=0 ; c<= ${#a[@]} ; c++ ))
  do
    echo "${a[c]}" "${b[c]}"
  done
)

# 4  
Old 09-18-2015
Perhaps something more like:
Code:
#!/bin/ksh
base="file"
start1=1
start2=385
for ((i = start1; i < start2; i++))
do	f1="$(printf '%s%03d' "$base" $i)"
	f2="$(printf '%s%03d' "$base" $((i + start2 - start1)))"
	if [[ -f "$f1" ]] && [[ -f "$f2" ]]
	then	printf '%s\n' "*** comparing $f1 to $f2"
		cmp "$f1" "$f2"
	else	printf '%s\n' "*** $f1 or $f2 is not a regular file; exiting"
		break
	fi
done

will do what you want. This was written an tested using a 1993 version of the Korn shell, but also works with bash.
This User Gave Thanks to Don Cragun For This Post:
# 5  
Old 09-18-2015
Still working on it Don. Getting close.
Thanks much for your reply!
# 6  
Old 09-19-2015
Just for the fun of it - and the logics - try
Code:
ls -r file*| for j in file*; do  read i; if [ $i == $j ]; then break; fi; echo $i"  "$j; done
file387  file001
file386  file002
file385  file003
file384  file004

This User Gave Thanks to RudiC For This Post:
# 7  
Old 11-19-2015
Same Problem I think

I am having a problem I think is similar to this one. I recently started a new class and I am way behind and I do not have a clue. I got one class with powershell ISE which that one I have ok I can get the work but for this shell and scripting class I am lost. I found a script but it only opens one file so I don't know what to do with that. I DL Fedora virtual OS on USB and I am running that then i used the terminal to install kornshellSmilie I ran this code I found and changed the name to testfile 1 and test file2 but I can only get it to read one file.

I used this code that I found here
#!/bin/ksh
file="/path/to/file.txt"
# while loop
while IFS= read -r line
do
# display line or do somthing on $line
echo "$line"
done <"$file"

and I tried to change it by adding
echo "$line" twice and changing
(while IFS= read -r line)
to

while IFS= read -r line1
IFS= read -r line2

but it does not work anymore. I will add the assignment details as well
Comparing list of files in parallel-12244106_497622357083107_651219004_njpg
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Comparing two files and list the difference with common first line content of both files

I have two file as given below which shows the ACL permissions of each file. I need to compare the source file with target file and list down the difference as specified below in required output. Can someone help me on this ? Source File ************* # file: /local/test_1 # owner: own #... (4 Replies)
Discussion started by: sarathy_a35
4 Replies

2. Shell Programming and Scripting

Comparing two edge list

Hello, I have two network edgelists with first two columns as nodes and the last column pearson correlation coefficient (PCC). I want to remove the edges from net1 whose edges are common with net2 && (PCC)net2>=(PCC)net1 net1.txt A B 0.6 A C 0.7 B C 0.7 D C ... (1 Reply)
Discussion started by: Sanchari
1 Replies

3. Shell Programming and Scripting

Curl parallel download file list

Hello guys, first post sorry if I did some mess here =) Using Ubuntu 14.04lts 64bits server version. I have a list (url.list) with only URLs to download, one per line, that looks like this: http://domain.com/teste.php?a=2&b=3&name=1 http://domain.com/teste.php?a=2&b=3&name=2 ...... (6 Replies)
Discussion started by: tonispa
6 Replies

4. Shell Programming and Scripting

Comparing list of files

Hi All, I have a list of files and want to find the difference between each file if there is difference then i want the file name to be printed and log the duplicate files For example Size Filename 23 a1 23 a2 23 a3 45 a4 If i diff a1 and a2 and find... (3 Replies)
Discussion started by: wedng.bell
3 Replies

5. Shell Programming and Scripting

Need Help with Bash - comparing directory contents with list of files

Hey guys, need help with a script I'm trying to write. Basically I need to compare the contents of a folder called "profiles" with a list of files called "template". when the file matches the contents of the folder it needs to set a variable called "checked" to "1" Cookies to anyone... (4 Replies)
Discussion started by: Scriporium
4 Replies

6. Shell Programming and Scripting

Need to read two files in parallel

Following is the requirement In FileA I have the content as follows. 1,2,3 111,222,333 1000,2000,3000 In FileB I have the content as follows. 4,5,6 444,555,666 4000,5000,6000 I need to read FileA and FileB parallely and create the FileC as follows. 1,2,3,4,5,6... (1 Reply)
Discussion started by: kmanivan82
1 Replies

7. Shell Programming and Scripting

Comparing two files and list the differences

Hi * I have two text files which has the file size, timestamp and the file name. I need to compare these two files and get the differences in the output format. Can anyone help me out with this. * cat file1.txt *474742 Apr 18* 2010 sample.log *135098 Apr 18* 2010 Testfile 134282 Apr 18* 2010... (7 Replies)
Discussion started by: Sendhil.Kumaran
7 Replies

8. UNIX for Dummies Questions & Answers

Copy files in Parallel

Is there any out of box Command in Unix or is it possible through shell scripting to copy all the files in a directory in Parallel. Example. I am doing a COLD database backup. I have the data/dbx directory which has around 1000 data files. I was thinking is there a way to spawn the copy... (10 Replies)
Discussion started by: simonsimon
10 Replies

9. Shell Programming and Scripting

Comparing a distinct value in 1 list with another list

Hi all, I need to compare the contents of 2 directories where the file contents are similar and take out the filenames whose contents does not exist within the 2 directories. Directory1 1 2 3 4 Directory2 54 55 56 57 Does anyone has a script which can do this? At the end of... (6 Replies)
Discussion started by: manualvin
6 Replies

10. UNIX for Dummies Questions & Answers

Comparing data list...

I have a list of files that I want to compare to another list of files, how do I do that? The first list will be my known list and hard coded, for example: mylist="janfile.tar jarfile.jar jan.rpt.Z" etc. The second list will be found by doing an 'ls' piped to a file: ls > filelist.dat ... (4 Replies)
Discussion started by: giannicello
4 Replies
Login or Register to Ask a Question