Compare two files and print output


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Compare two files and print output
# 1  
Old 11-19-2019
Compare two files and print output

Hi All,

i am trying to compare two files in Centos 6.

F1: /tmp/d21

Code:
NAME="xvda" TYPE="disk" SIZE="40G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT=""
NAME="xvda1" TYPE="part" SIZE="500M" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="/boot"
NAME="xvda2" TYPE="part" SIZE="39.5G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT=""
NAME="VolGroup-lv_root" TYPE="lvm" SIZE="37.6G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="/"
NAME="VolGroup-lv_swap" TYPE="lvm" SIZE="2G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="[SWAP]"
NAME="xvdq" TYPE="disk" SIZE="2G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="/var/log/audit"
NAME="xvdp" TYPE="disk" SIZE="7G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="/var/log"
NAME="xvdo" TYPE="disk" SIZE="7G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="/var"
NAME="xvdn" TYPE="disk" SIZE="2G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="/home"

File2 :

Code:
Filesystem Size Used Avail Use% Mounted
 /dev/mapper/VolGroup-lv_root
 37G 1.4G 34G 4% /
 tmpfs 1.9G 0 1.9G 0% /dev/shm
 /dev/xvda1 477M 126M 326M 28% /boot
 /dev/xvdn 2.0G 3.1M 1.9G 1% /home
 /dev/xvdo 6.8G 458M 6.0G 7% /var
 /dev/xvdp 6.8G 27M 6.4G 1% /var/log
 /dev/xvdq 2.0G 24M 1.8G 2% /var/log/audit


i want to compare the mountpoint variable value in file1 with Mounted column in file2
if both are same then it need to print both file1 and file2 data.

Desired output :

Code:
{"'file_system'":"'/dev/xvda1'","'total'":"'477M'","'disk'":"'xvda1'", "'used'":"'126M'", "'available'":"'326M'", "'used_percent'":"'28%'", "'type'":"part''", "'owner'":"'root'", "'group'":"'disk'", "'permissions'":"'brw-rw----'", "'mounted'":"'/boot'"}

like wise i need to print all the file systems info.

I used below script

Code:
#!/bin/bash



lsblk -o NAME,TYPE,SIZE,OWNER,GROUP,MODE,MOUNTPOINT  -P > /tmp/d21



df -h > /tmp/d1

awk '{print " "$1" "$2" "$3" "$4" "$5" "$6" "}' /tmp/d1 > /tmp/d3


awk 'FNR==NR{f1[$14]=$0;next} $6 in f1 {print $0, f1[$1]}' FS='"' /tmp/d21 FS=' ' /tmp/d3   | {


while IFS= read -r line

  do

  file_system=$(echo "$line"|awk '{print $1}')

  total=$(echo "$line"|awk '{print $2}')

  used=$(echo "$line"|awk '{print $3}')

  available=$(echo "$line"|awk '{print $4}')

  used_percent=$(echo "$line"|awk '{print $5}')

  type=$(echo "$line"|awk '{print $8}'|awk -F "=" '{print $2}')

  owner=$(echo "$line"|awk '{print $10}'|awk -F "=" '{print $2}')

  group=$(echo "$line"|awk '{print $11}'|awk -F "=" '{print $2}')

  permissions=$(echo "$line"|awk '{print $12}'|awk -F "=" '{print $2}')

disk=$(echo $partition|tr -d '[0-9+\-]')
mounted=$(echo "$line"|awk '{print $6}')





echo -e "{\"'file_system'\":\"'"$file_system"'\",\"'total'\":\"'"$total"'\",\"'disk'\":\"'"$disk"'\", \"'used'\":\"'"$used"'\", \"'available'\":\"'"$available"'\", \"'used_percent'\":\"'"$used_percent"'\", \"'type'\":\"'"$type"'\", \"'owner'\":\"'"$owner"'\", \"'group'\":\"'"$group"'\", \"'permissions'\":\"'"$permissions"'\", \"'mounted'\":\"'"$mounted"'\"}"



  done

}

but i am getting below output :


Code:
{"'file_system'":"'/dev/xvda1'","'total'":"'477M'","'disk'":"''", "'used'":"'126M'", "'available'":"'326M'", "'used_percent'":"'28%'", "'type'":"''", "'owner'":"''", "'group'":"''", "'permissions'":"''",  "'mounted'":"'/boot'"}

{"'file_system'":"'/dev/mapper/VolGroup-lv_root'","'total'":"''","'disk'":"''", "'used'":"''", "'available'":"''", "'used_percent'":"''", "'type'":"''", "'owner'":"''", "'group'":"''", "'permissions'":"''","'mounted'":"''"}
{"'file_system'":"'37G'","'total'":"'1.4G'","'disk'":"''", "'used'":"'34G'", "'available'":"'4%'", "'used_percent'":"'/'", "'type'":"''", "'owner'":"''", "'group'":"''", "'permissions'":"''","'mounted'":"''"}

Please correct me in this issue.

Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!

Last edited by vgersh99; 11-19-2019 at 12:36 PM.. Reason: code tags, please!
# 2  
Old 11-19-2019
So, if you are certain that the MOUNTPOINT=" will always be the last field, you could:-
  • List the filesystems
  • For each filesystem, get the data from file 1
  • For each filesystem, get the data from file 2
  • Display the output required or nothing if no match
It's a bit clunky, but it's for a very small volume of data. Something like this might do:-
Code:
#!/bin/bash

f1_file=/tmp/d21
f2_file=/tmp/f2                   # Or whatever


while read filesystem
do
   filesystem_list="${filesystem_list} ${filesystem}"
done < <(cut -f14 -d"\"" ${f1_file})

for filesystem in $filesystem_list
do
   f1_line=$(grep "MOUNTPOINT=\"${filesystems}\" "${f1_file}"
   f2_line=$(grep -E "${filesystems}$" "${f2_file}"
   if [ ! -z "${f2_line} ]
   then
      # Work on the two data lines you have to extract the data you want
   else
      echo "You have incomplete data for filesystem ${filesystem}"
   fi
done

Like I say, pretty clunky, but maybe that's all you need.



Robin
# 3  
Old 11-20-2019
How about - as a zeroth approximation - this:
Code:
awk '
NR == 1         {split ($0, HD)
                 next
                }
NR == FNR       {while (NF < 6) {getline X
                                 $0 = $0 FS X
                                }
                 for (i=1; i<=NF; i++) $i = HD[i] ":\"" $i "\""
                 TMP = $NF
                 sub (OFS $NF,_)
                 REF[TMP] = $0
                 next
                }
                {gsub (/=/, ":")
                 sub (/MOUNTPOINT/, "Mounted")
                }
$NF in REF      {sub ("^" $1 FS, _)
                 $1 = $1
                 print REF[$NF], $0
                }

' OFS=, file2 file1
Filesystem:"/dev/xvda1",Size:"477M",Used:"126M",Avail:"326M",Use%:"28%",TYPE:"part",SIZE:"500M",OWNER:"root",GROUP:"disk",MODE:"brw-rw----",Mounted:"/boot"
Filesystem:"/dev/mapper/VolGroup-lv_root",Size:"37G",Used:"1.4G",Avail:"34G",Use%:"4%",TYPE:"lvm",SIZE:"37.6G",OWNER:"root",GROUP:"disk",MODE:"brw-rw----",Mounted:"/"
Filesystem:"/dev/xvdq",Size:"2.0G",Used:"24M",Avail:"1.8G",Use%:"2%",TYPE:"disk",SIZE:"2G",OWNER:"root",GROUP:"disk",MODE:"brw-rw----",Mounted:"/var/log/audit"
Filesystem:"/dev/xvdp",Size:"6.8G",Used:"27M",Avail:"6.4G",Use%:"1%",TYPE:"disk",SIZE:"7G",OWNER:"root",GROUP:"disk",MODE:"brw-rw----",Mounted:"/var/log"
Filesystem:"/dev/xvdo",Size:"6.8G",Used:"458M",Avail:"6.0G",Use%:"7%",TYPE:"disk",SIZE:"7G",OWNER:"root",GROUP:"disk",MODE:"brw-rw----",Mounted:"/var"
Filesystem:"/dev/xvdn",Size:"2.0G",Used:"3.1M",Avail:"1.9G",Use%:"1%",TYPE:"disk",SIZE:"2G",OWNER:"root",GROUP:"disk",MODE:"brw-rw----",Mounted:"/home"

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare columns of multiple files and print those unique string from File1 in an output file.

Hi, I have multiple files that each contain one column of strings: File1: 123abc 456def 789ghi File2: 123abc 456def 891jkl File3: 234mno 123abc 456def In total I have 25 of these type of file. (5 Replies)
Discussion started by: owwow14
5 Replies

2. Shell Programming and Scripting

Compare two files and print list

Hi Gents, I have 2 files as seen below. File 1: 9 1020 10 1001 11 1001 12 1002 13 1003 14 1004 15 1004 File 2: 9 1000 11 1001 12 1002 13 1003 15 1004 (5 Replies)
Discussion started by: jiam912
5 Replies

3. Shell Programming and Scripting

Compare two files and print using awk

I have 2 files: email_1.out 1 abc@yahoo.com 2 abc_1@yahoo.com 3 abc_2@yahoo.com data_1.out <tr> 1 MAIL # 1 TO src_1 </tr> <tr><td class="hcol">col_id</td> <td class="hcol">test_dt</td> <td class="hcol">user_type</td> <td class="hcol">ct</td></tr> <tr><td... (1 Reply)
Discussion started by: sol_nov
1 Replies

4. Shell Programming and Scripting

Compare columns 2 files and print

File 1 has 16 columns so does File 2 I want to remove all records from File 2 that column 1 and column 16 match between file 1 and file 2 delimter of files is ~ (10 Replies)
Discussion started by: sigh2010
10 Replies

5. UNIX for Advanced & Expert Users

Shell Script to compare xml files and print output to a file

All, PLease can you help me with a shell script which can compare two xml files and print the difference to a output file. I have attached one such file for you reference. <Group> <Member ID=":Year_Quad:41501" childCount="4" fullPath="PEPSICO Year-Quad-Wk : FOLDER.52 Weeks Ending Dec... (2 Replies)
Discussion started by: kanthrajgowda
2 Replies

6. Shell Programming and Scripting

compare two files and search keyword and print output

You have two files to compare by searching keyword from one file into another file File A 23 >pp_ANSWER 24 >aa hello 25 >jau head wear 66 >jss oops 872 >aqq olps ploww oww sss 722 >GG_KILLER ..... large files File B Beta done KILLER John Mayor calix meyers ... (5 Replies)
Discussion started by: cdfd123
5 Replies

7. Shell Programming and Scripting

awk to compare flat files and print output to another file

Hello, I am strugling from quite a some time to compare flat files with over 1 million records could anyone please help me. I want to compare two pipe delimited flat files, file1 with file2 and output the unmatched rows from file2 in file3 Sample File1: ... (9 Replies)
Discussion started by: suhaeb
9 Replies

8. Shell Programming and Scripting

compare columns from seven files and print the output

Hi guys, I need some help to come out with a solution . I have seven such files but I am showing only three for convenience. filea a5 20 a8 16 fileb a3 42 a7 14 filec a5 23 a3 07 The output file shoud contain the data in table form showing first field of... (7 Replies)
Discussion started by: smriti_shridhar
7 Replies

9. Shell Programming and Scripting

compare two files and print the last row into first

suppose fileA vis vis gyh gye gyh fileB vis 23 gyh 21 gye 32 output shud be like in fileA ... vis 23 vis 23 gyh 21 gye 32 gyh 21 (1 Reply)
Discussion started by: cdfd123
1 Replies

10. Shell Programming and Scripting

awk to compare lines of two files and print output on screen

hey guys, I have two files both with two columns, I have already created an awk code to ignore certain lines (e.g lines that start with 963) as they wou ld begin with a certain string, however, the rest I have added together and calculated the average. At the moment the code also displays... (3 Replies)
Discussion started by: chlfc
3 Replies
Login or Register to Ask a Question