Need to check 1 file against others in a folder


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need to check 1 file against others in a folder
# 1  
Old 03-12-2010
I'm trying to compare 1 file(the boiler plate file we'll call it) against others in the same folder. I need to see if the other files have extra keys in them. i.e. - the other files should have the same amount of variables as found in boiler plate file

File1.txt format:
----beginning of file----
key1=value1
key2=value2
---end of file-------

File2.txt(or any other file in dir)
----beginning of file----
key1=value1
key2=value2
keyx=valuex
---end of file-------

I want to find that variable keyx in the other file(s). thanks for any help!
------------------------------ 19.03.2010 post: ----------------------------------------

wonder if anyone has a pl script that does the following.

I have 3 or more files in a dir. File A is the base file I want to compare against the others, but need to know if lines are missing from File A that exist in the other files. Example of content in files

File A contains:
###file info###
line1=value1
line2=value2

File B contains:
###file info###
line1=value1
line2=value2
line3=valuex

FileC contains:
###file info###
line1=value1
line2=value2
line3=valuey
line4=valuez

Need a result like this:

File A is missing:
line3
line4

I don't care about the values to the right of the "=" just the variable to the left. Any help is appreciated!!!

Last edited by vbe; 03-19-2010 at 02:16 PM.. Reason: merge what seems to be bumping
# 2  
Old 03-12-2010
Not sure if this is what you want, but grep -v -f file1 file2 will filter out the contents of file1 from file2, displaying any extra stuff that is in file2.
Code:
# grep -v -f File1.txt File2.txt
keyx=valuex
#

I don't know what your other requirements are, but here are 4 possibilities for performing that on every file in a directory.
Code:
These all assume there is nothing in the files except "key" 
lines that you are interested in. If not, this can be added 
to the first two commands: | grep "^key"

ls | while read file
   do grep -v -f File1.txt $file 
done 

or...

ls | xargs -n1 grep -v -f File1.txt 

or...

grep -v -f File1.txt $(ls)

or...

grep -v -f File1.txt *

# 3  
Old 03-12-2010
Here's a perl approach:

data:
Code:
$ cat file1.txt
key1=value1
key2=value2

$ cat file2.txt
key1=value1
key2=value2
key21=value21
key22=value22
key23=value23
key24=value24

$ cat file3.txt
key1=value1
key2=value2
key31=value31
key32=value32
key33=value33
key34=value34

$ cat file4.txt
key1=value1
key2=value2
key41=value41
key42=value42
key43=value43
key44=value44

chk_keys.pl
Code:
#!/usr/bin/perl

my $datadir = "/home/mydir";
my $bfile = "file1.txt";
my %seen;
my $key;
my @flist;
my $keyfile;

open (BOILER_FILE, "<$bfile")
  or die "ERROR:  Unable to open $bfile";

while ($line = <BOILER_FILE>) {
   chomp $line;
   $key = ( split /=/, $line )[0];
   $seen{$key} = 1;

   print "BOILER KEY:  $key\n";
}

close (BOILER_FILE);

opendir DIRHANDLE, $datadir  or
   die "ERROR:  Unable to access $datadir" ;

@flist = (readdir(DIRHANDLE));
close DIRHANDLE;

for $keyfile (@flist) {

   next unless ($keyfile =~ /file.+\.txt/ && $keyfile ne $bfile);

   print "FILE:  $keyfile\n";

   open ( INFILE, "$datadir/$keyfile" )
      or die "ERROR:  Unable to open $keyfile";

   while ($line = <INFILE>) {
      chomp $line;
      $key = ( split /=/, $line )[0];

      next unless ! $seen{$key};

      print "   EXTRA KEY:  $key\n";
   }
}

exit 0

output:
Code:
$ ./chk_keys.pl
BOILER KEY:  key1
BOILER KEY:  key2
FILE:  file2.txt
   EXTRA KEY:  key21
   EXTRA KEY:  key22
   EXTRA KEY:  key23
   EXTRA KEY:  key24
FILE:  file3.txt
   EXTRA KEY:  key31
   EXTRA KEY:  key32
   EXTRA KEY:  key33
   EXTRA KEY:  key34
FILE:  file4.txt
   EXTRA KEY:  key41
   EXTRA KEY:  key42
   EXTRA KEY:  key43
   EXTRA KEY:  key44


Last edited by jsmithstl; 03-12-2010 at 07:37 PM.. Reason: added filename
# 4  
Old 03-12-2010
Quote:
Originally Posted by fubaya
Not sure if this is what you want, but grep -v -f file1 file2 will filter out the contents of file1 from file2, displaying any extra stuff that is in file2.
Code:
# grep -v -f File1.txt File2.txt
keyx=valuex
#

You've assumed that the values are identical. key1 in one file may not have the same value as key1 in the other. At least, that seems a distinct possibility since the original problem statement does not specify invariant values. If the values are different, your solution will not work.

Regards,
Alister

---------- Post updated at 06:52 PM ---------- Previous update was at 06:41 PM ----------

Perhaps this will do:
Code:
join -t= -v2 template data

It will print any lines in the "data" file with keys that are absent from the "template" file.

Regards,
Alister

Last edited by alister; 03-12-2010 at 07:50 PM..
# 5  
Old 03-15-2010
How about ...

Code:
[house@discovery] diff file1 file2
2a3
> keyx=valuex

# 6  
Old 03-15-2010
Thanks all. jsmithstl your pl script works perfect for my need. Thanks again!
# 7  
Old 03-19-2010
Bumping up posts or double posting is not permitted in these forums.

Please read the rules, which you agreed to when you registered, if you have not already done so.

You may receive an infraction for this. If so, don't worry, just try to follow the rules more carefully. The infraction will expire in the near future

Thank You.

The UNIX and Linux Forums.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to check directory and create missing folder from file

In the below bash I am trying to ensure that all folders (represented by $folders) in a given directory are created. In the file f1 the trimmed folder will be there somewhere (will be multiple trimmed folders). When that trimmed folder is found (represented by $S5) the the contents of $2 printed... (19 Replies)
Discussion started by: cmccabe
19 Replies

2. Shell Programming and Scripting

Shell script which will check the target file and folder exists and copy it

Hi All, I am a beginner in this and trying to write a shell script in linux which will : 1. Ask for a file name and check if its exists. 2. If file exists only then it will ask for the new target folder, after entering target folder name it will check if it exists. 3. If target folder... (3 Replies)
Discussion started by: ashish_neekhra
3 Replies

3. Shell Programming and Scripting

Check the file in folder

Hi friends, i have two folder let say app/f1/ app1/f2/ i need to send mail on EOD if no file found in app/f1/ and app/f2/ quick response will much appriciated. regards, PK (2 Replies)
Discussion started by: pkrabi78
2 Replies

4. Shell Programming and Scripting

check existence of files in a folder

Hi I am having a problem to verify existence of files. I need to know whether or not files in a folder that begins with a name. For example all files that start with The_File_ *. I was doing it this way, but gives me error. if text -f /work/The_File_* then ... else .. fi (5 Replies)
Discussion started by: Rodrih92
5 Replies

5. UNIX for Dummies Questions & Answers

Assistance with shell script to check file type and move to a folder.

Hi, Below is some code that I would like to implement however I am getting these errors: (what I am attempting to do is to check if a zip file has ascii files and if ascii and not binary then move the ascii files to a folder. some of the files are in xml format but are ascii and i will be moving... (0 Replies)
Discussion started by: bwcberb
0 Replies

6. Shell Programming and Scripting

check how many files in folder or total files in folder

Hi all, I have copied all my files to one folder.and i want to check how many files (count) in the folder recently moved or total files in my folder? please send me the query asap. (3 Replies)
Discussion started by: durgaprasad
3 Replies

7. Shell Programming and Scripting

File Management: How do I move all JPGS in a folder structure to a single folder?

This is the file structure: DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies

8. Shell Programming and Scripting

check if file exists in a mounted windows shared folder

hi, I posted a thread before on that subject, but with a wrong focus... here's my problem: I want to check if a file exists in a windows shared folder mounted using: sudo mount -t cifs -o username=xxx,password=xxx,uid=xxx,gid=xxx //192.168.0.92/public /media/92_shared I tried if ... (2 Replies)
Discussion started by: jul
2 Replies

9. Shell Programming and Scripting

simple check to see if a folder exists

Hi, I have cobbled together a simple script to create a Windows folder in a bunch of home folders on a mac server using the following code. for i in /Volumes/student_data/studenthomefolders/* do u=`echo $i | cut -d/ -f5` //if //then //echo "Folder already exists for "$u" Skipping" //else... (4 Replies)
Discussion started by: psyman
4 Replies

10. Shell Programming and Scripting

Parse the .txt file for folder name and FTP to the corrsponding folder.

Oracle procedure create files on UNIX folder on a regular basis. I need to FTP files onto windows server and place the files, based on their name, in the corresponding folders. File name is as follows: ccyymmddfoldernamefile.txt; Folder Name length could be of any size; however, the prefix and... (3 Replies)
Discussion started by: MeganP
3 Replies
Login or Register to Ask a Question