Sponsored Content
Full Discussion: Comparing 2 UNIX files
Top Forums UNIX for Dummies Questions & Answers Comparing 2 UNIX files Post 302958657 by Rahul Raj on Sunday 25th of October 2015 12:58:15 AM
Old 10-25-2015
Hi Don,

Thanks for your reply.

I don't see how you get rid of all duplicates by selecting on a few samples???
- The duplicates will be eliminated first and then sample records will be selected.

Are you trying to compare databases? Or, are you trying to compare text files that have been extracted from those databases?
- text files extracted from databases(around 10 tables)

You say the sql server file is larger. How many records are in each database? How many fields are in each database?
- each of the 10 tables have different structure. On an average around 10-15 records and I would select a sample of around 1000 records from each file.

If you extracted each entire database into a pipe delimited text file, how many records (lines) would appear in each file and how many characters would be in the longest line in each file?
- the records in each file would vary from around 10K - 10L records.Sample would be around 1000 records and each file would have somewhere around 1000 characters

Please show us a sanitized sample of the records you want to compare and show us the output you want to produce from those input samples? (Please use CODE tags when you post the sample input and output files.)
- I can't provide the print of records due to the company policy but it would be something as below :

Code:
 File 1
 value1|unique_key|||value2(decimal)|value3(decimal)|||
 value1|unique_key|||value2|value3|||
  
 File 2
 value1|unique_key|||value2(decimal)|value3(decimal)|||
 value1|unique_key|||value2|value3|||
  
 result:
 want the result of the mismatched records in another file.

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

comparing shadow files with real files

Hi I need to compare shadow file sizes with their real file counterparts. If the shadow file size differs form the realfile size then it must send a mail. My problem is that our system has over 1600 shadowfiles in different directories, with different names. the only consistancy is the .sh file... (4 Replies)
Discussion started by: terrym
4 Replies

2. Shell Programming and Scripting

Need help comparing two files and deleting some things in those files!

So I have two files: File1 pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2 pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2 ref4948 1.1 treehouse.txt 1.6 ref8573 1.5 ref3284 1.4 ref5838... (24 Replies)
Discussion started by: linuxkid
24 Replies

3. Shell Programming and Scripting

Comparing the matches in two files using awk when both files have their own field separators

I've two files with data like below: file1.txt: AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 file2.txt: Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 So,the field separator in file1.txt is a comma and in file2.txt,it is | Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies

4. UNIX for Advanced & Expert Users

How to find duplicates contents in a files by comparing other files?

Hi Guys , we have one directory ...in that directory all files will be set on each day.. files must have header ,contents ,footer.. i wants to compare the header,contents,footer ..if its same means display an error message as 'files contents same' (7 Replies)
Discussion started by: Venkatesh1
7 Replies

5. Shell Programming and Scripting

Comparing files in a directory against an array of files

I hope I can explain this correctly. I am using Bash-4.2 for my shell. I have a group of file names held in an array. I want to compare the names in this array against the names of files currently present in a directory. If the file does not exist in the directory, that is not a problem.... (5 Replies)
Discussion started by: BudMan
5 Replies

6. Shell Programming and Scripting

Comparing Select Columns from two CSV files in UNIX and create a third file based on comparision

Hi , I want to compare first 3 columns of File A and File B and create a new file File C which will have all rows from File B and will include rows that are present in File A and not in File B based on First 3 column comparison. Thanks in advance for your help. File A A,B,C,45,46... (2 Replies)
Discussion started by: ady_koolz
2 Replies

7. Shell Programming and Scripting

Need help in comparing two files in UNIX with a mismatch

Hi Everyone, I am comparing results of two environments using unix files. I am writing two different csv file using spool and merging both the files using paste command paste -d, file1.csv file2.csv > prod_uat_result.csv and then finding the difference and attaching the same in a mail... (8 Replies)
Discussion started by: karthik adiga
8 Replies

8. Shell Programming and Scripting

Comparing two files in UNIX and create a new file similar to equi join

I have 2 files namely branch.txt file & RXD.txt file as below Ex:Branch.txt ========================= B1,Branchname1,city,country B2,Branchname2,city,country B3,Branchname3,city,country B4,Branchname4,city,country B5,Branchname5,city,country RXD file : will... (11 Replies)
Discussion started by: satece
11 Replies

9. UNIX for Beginners Questions & Answers

Comparing 2 variables in UNIX

Hi, I have 2 variables as given below. How can i compare them and say its matching ? Appreciate your help VAR1=describe/read/write VAR2=read/write/describeThanks, Please use CODE tags as required by forum rules! (4 Replies)
Discussion started by: prince1987
4 Replies

10. UNIX for Beginners Questions & Answers

Comparing two files and list the difference with common first line content of both files

I have two file as given below which shows the ACL permissions of each file. I need to compare the source file with target file and list down the difference as specified below in required output. Can someone help me on this ? Source File ************* # file: /local/test_1 # owner: own #... (4 Replies)
Discussion started by: sarathy_a35
4 Replies
MYSQLDIFF(1)							  MySQL Utilities						      MYSQLDIFF(1)

NAME
mysqldiff - Identify differences among database objects SYNOPSIS
mysqldiff [options] {db1[:db1] | db1.obj1[:db2.obj2]} ... DESCRIPTION
This utility reads the definitions of objects and compares them using a diff-like method to determine whether they are the same. The util- ity displays the differences for objects that are not the same. Use the notation db1:db2 to name two databases to compare, or, alternatively just db1 to compare two databases with the same name. The latter case is a convenience notation for comparing same-named databases on different servers. The comparison may be run against two databases of different names on a single server by specifying only the --server1 option. The user can also connect to another server by specifying the --server2 option. In this case, db1 is taken from server1 and db2 from server2. When a database pair is specified, all objects in one database are compared to the corresponding objects in the other. Any objects not appearing in either database produce an error. To compare a specific pair of objects, add an object name to each database name in db.obj format. For example, use db1.obj1:db2.obj2 to compare two named objects, or db1.obj1 to compare an object with the same name in databases with the same name. It is not legal to mix a database name with an object name. For example, db1.obj1:db2 and db1:db2.obj2 are illegal. The comparison may be run against a single server for comparing two databases of different names on the same server by specifying only the --server1 option. Alternatively, you can also connect to another server by specifying the --server2 option. In this case, the first object to compare is taken from server1 and the second from server2. By default, the utilty generates object differences as a difference report. However, you can generate a transformation report containing SQL statements for transforming the objects for conformity instead. Use the 'sql' value for the --difftype option to produce a listing that contains the appropriate ALTER commands to conform the object definitions for the object pairs specified. If a transformation cannot be formed, the utility reports the diff of the object along with a warning statement. See important limitations in the NOTES section. To specify how to display diff-style output, use one of the following values with the --difftype option: unified (default) Display unified format output. context Display context format output. differ Display differ-style format output. sql Display SQL transformation statement output. The --changes-for option controls the direction of the difference (by specifying the object to be transformed) in either the difference report (default) or the transformation report (designated with the --difftype=sql option). Consider the following command: mysqldiff --server1=root@host1 --server2=root@host2 --difftype=sql db1.table1:dbx.table3 The leftmost database (db1) exists on the server designated by the --server1 option (host1). The rightmost database (dbx) exists on the server designated by the --server2 option (host2). o --changes-for=server1: Produce output that shows how to make the definitions of objects on server1 like the definitions of the corre- sponding objects on server2. o --changes-for=server2: Produce output that shows how to make the definitions of objects on server2 like the definitions of the corre- sponding objects on server1. The default direction is server1. For sql difference format, you can also see the reverse transformation by specifying the --show-reverse option. The utility stops on the first occurrence of missing objects or when an object does not match. To override this behavior, specify the --force option to cause the utility to attempt to compare all objects listed as arguments. OPTIONS
mysqldiff accepts the following command-line options: --help Display a help message and exit. --changes-for=<direction> Specify the server to show transformations to match the other server. For example, to see the transformation for transforming object definitions on server1 to match the corresponding definitions on server2, use --changes-for=server1. Permitted values are server1 and server2. The default is server1. --difftype=<difftype>, -d<difftype> Specify the difference display format. Permitted format values are unified, context, differ, and sql. The default is unified. --force Do not halt at the first difference found. Process all objects to find all differences. --quiet, -q Do not print anything. Return only an exit code of success or failure. --server1=<source> Connection information for the first server in <user>[:<passwd>]@<host>[:<port>][:<socket>] format. --server2=<source> Connection information for the second server in <user>[:<passwd>]@<host>[:<port>][:<socket>] format. --show-reverse Produce a transformation report containing the SQL statements to conform the object definitions specified in reverse. For example, if --changes-for is set to server1, also generate the transformation for server2. Note: The reverse changes are annotated and marked as comments. --verbose, -v Specify how much information to display. Use this option multiple times to increase the amount of information. For example, -v = verbose, -vv = more verbose, -vvv = debug. --version Display version information and exit. --width=<number> Change the display width of the test report. The default is 75 characters. NOTES
You must provide connection parameters (user, host, password, and so forth) for an account that has the appropriate privileges to access all objects to be compared. The SQL transformation feature has these known limitations: o When tables with partition differences are encountered, the utility generates the ALTER TABLE statement for all other changes but prints a warning and omits the partition differences. o If the transformation detects table options in the source table (specified with the --changes-for option) that are not changed or do not exist in the target table, the utility generates the ALTER TABLE statement for all other changes but prints a warning and omits the table option differences. o Rename for events is not supported. This is because mysqldiff compares objects by name. In this case, depending on the direction of the diff, the event is identified as needing to be added or a DROP EVENT statement is generated. o Changes in the definer clause for events are not supported. o SQL extensions specific to MySQL Cluster are not supported. For the --difftype option, the permitted values are not case sensitive. In addition, values may be specified as any unambiguous prefix of a valid value. For example, --difftype=d specifies the differ type. An error occurs if a prefix matches more than one valid value. EXAMPLES
To compare the employees and emp databases on the local server, use this command: $ mysqldiff --server1=root@localhost employees:emp1 # server1 on localhost: ... connected. WARNING: Objects in server1:employees but not in server2:emp1: EVENT: e1 Compare failed. One or more differences found. $ mysqldiff --server1=root@localhost employees.t1:emp1.t1 employees.t3:emp1.t3 # server1 on localhost: ... connected. # Comparing employees.t1 to emp1.t1 [PASS] # server1 on localhost: ... connected. # Comparing employees.t3 to emp1.t3 [PASS] Success. All objects are the same. $ mysqldiff --server1=root@localhost employees.salaries:emp1.salaries --differ # server1 on localhost: ... connected. # Comparing employees.salaries to emp1.salaries [FAIL] # Object definitions are not the same: CREATE TABLE `salaries` ( `emp_no` int(11) NOT NULL, `salary` int(11) NOT NULL, `from_date` date NOT NULL, `to_date` date NOT NULL, PRIMARY KEY (`emp_no`,`from_date`), KEY `emp_no` (`emp_no`) - ) ENGINE=InnoDB DEFAULT CHARSET=latin1 ? ^^^^^ + ) ENGINE=MyISAM DEFAULT CHARSET=latin1 ? ++ ^^^ Compare failed. One or more differences found. The following examples show how to generate a transformation report. Assume the following object definitions: Host1: CREATE TABLE db1.table1 (num int, misc char(30)); Host2: CREATE TABLE dbx.table3 (num int, notes char(30), misc char(55)); To generate a set of SQL statements that transform the definition of db1.table1 to dbx.table3, use this command: $ mysqldiff --server1=root@host1 --server2=root@host2 --changes-for=server1 --difftype=sql db1.table1:dbx.table3 # server1 on host1: ... connected. # server2 on host2: ... connected. # Comparing db1.table1 to dbx.table3 [FAIL] # Transformation statments: ALTER TABLE db1.table1 ADD COLUMN notes char(30) AFTER a, CHANGE COLUMN misc misc char(55); Compare failed. One or more differences found. To generate a set of SQL statements that transform the definition of dbx.table3 to db1.table1, use this command: $ mysqldiff --server1=root@host1 --server2=root@host2 --changes-for=server2 --difftype=sql db1.table1:dbx.table3 # server1 on host1: ... connected. # server2 on host2: ... connected. # Comparing db1.table1 to dbx.table3 [FAIL] # Transformation statments: ALTER TABLE dbx.table3 DROP COLUMN notes, CHANGE COLUMN misc misc char(30); Compare failed. One or more differences found. To generate a set of SQL statements that transform the definitions of dbx.table3 and db1.table1 in both directions, use this command: $ mysqldiff --server1=root@host1 --server2=root@host2 --show-reverse --difftype=sql db1.table1:dbx.table3 # server1 on host1: ... connected. # server2 on host2: ... connected. # Comparing db1.table1 to dbx.table3 [FAIL] # Transformation statments: # --destination=server1: ALTER TABLE db1.table1 ADD COLUMN notes char(30) AFTER a, CHANGE COLUMN misc misc char(55); # --destination=server2: # ALTER TABLE dbx.table3 # DROP COLUMN notes, # CHANGE COLUMN misc misc char(30); Compare failed. One or more differences found. COPYRIGHT
Copyright (c) 2011, 2012, Oracle and/or its affiliates. All rights reserved. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER- CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA AUTHOR
MySQL Utilities Team COPYRIGHT
2010, Oracle and/or its affiliates. All rights reserved. 1.0.3 May 09, 2012 MYSQLDIFF(1)
All times are GMT -4. The time now is 10:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy