Sponsored Content
Full Discussion: Comparing 2 UNIX files
Top Forums UNIX for Dummies Questions & Answers Comparing 2 UNIX files Post 302958657 by Rahul Raj on Sunday 25th of October 2015 12:58:15 AM
Old 10-25-2015
Hi Don,

Thanks for your reply.

I don't see how you get rid of all duplicates by selecting on a few samples???
- The duplicates will be eliminated first and then sample records will be selected.

Are you trying to compare databases? Or, are you trying to compare text files that have been extracted from those databases?
- text files extracted from databases(around 10 tables)

You say the sql server file is larger. How many records are in each database? How many fields are in each database?
- each of the 10 tables have different structure. On an average around 10-15 records and I would select a sample of around 1000 records from each file.

If you extracted each entire database into a pipe delimited text file, how many records (lines) would appear in each file and how many characters would be in the longest line in each file?
- the records in each file would vary from around 10K - 10L records.Sample would be around 1000 records and each file would have somewhere around 1000 characters

Please show us a sanitized sample of the records you want to compare and show us the output you want to produce from those input samples? (Please use CODE tags when you post the sample input and output files.)
- I can't provide the print of records due to the company policy but it would be something as below :

Code:
 File 1
 value1|unique_key|||value2(decimal)|value3(decimal)|||
 value1|unique_key|||value2|value3|||
  
 File 2
 value1|unique_key|||value2(decimal)|value3(decimal)|||
 value1|unique_key|||value2|value3|||
  
 result:
 want the result of the mismatched records in another file.

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

comparing shadow files with real files

Hi I need to compare shadow file sizes with their real file counterparts. If the shadow file size differs form the realfile size then it must send a mail. My problem is that our system has over 1600 shadowfiles in different directories, with different names. the only consistancy is the .sh file... (4 Replies)
Discussion started by: terrym
4 Replies

2. Shell Programming and Scripting

Need help comparing two files and deleting some things in those files!

So I have two files: File1 pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2 pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2 ref4948 1.1 treehouse.txt 1.6 ref8573 1.5 ref3284 1.4 ref5838... (24 Replies)
Discussion started by: linuxkid
24 Replies

3. Shell Programming and Scripting

Comparing the matches in two files using awk when both files have their own field separators

I've two files with data like below: file1.txt: AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 file2.txt: Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 So,the field separator in file1.txt is a comma and in file2.txt,it is | Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies

4. UNIX for Advanced & Expert Users

How to find duplicates contents in a files by comparing other files?

Hi Guys , we have one directory ...in that directory all files will be set on each day.. files must have header ,contents ,footer.. i wants to compare the header,contents,footer ..if its same means display an error message as 'files contents same' (7 Replies)
Discussion started by: Venkatesh1
7 Replies

5. Shell Programming and Scripting

Comparing files in a directory against an array of files

I hope I can explain this correctly. I am using Bash-4.2 for my shell. I have a group of file names held in an array. I want to compare the names in this array against the names of files currently present in a directory. If the file does not exist in the directory, that is not a problem.... (5 Replies)
Discussion started by: BudMan
5 Replies

6. Shell Programming and Scripting

Comparing Select Columns from two CSV files in UNIX and create a third file based on comparision

Hi , I want to compare first 3 columns of File A and File B and create a new file File C which will have all rows from File B and will include rows that are present in File A and not in File B based on First 3 column comparison. Thanks in advance for your help. File A A,B,C,45,46... (2 Replies)
Discussion started by: ady_koolz
2 Replies

7. Shell Programming and Scripting

Need help in comparing two files in UNIX with a mismatch

Hi Everyone, I am comparing results of two environments using unix files. I am writing two different csv file using spool and merging both the files using paste command paste -d, file1.csv file2.csv > prod_uat_result.csv and then finding the difference and attaching the same in a mail... (8 Replies)
Discussion started by: karthik adiga
8 Replies

8. Shell Programming and Scripting

Comparing two files in UNIX and create a new file similar to equi join

I have 2 files namely branch.txt file & RXD.txt file as below Ex:Branch.txt ========================= B1,Branchname1,city,country B2,Branchname2,city,country B3,Branchname3,city,country B4,Branchname4,city,country B5,Branchname5,city,country RXD file : will... (11 Replies)
Discussion started by: satece
11 Replies

9. UNIX for Beginners Questions & Answers

Comparing 2 variables in UNIX

Hi, I have 2 variables as given below. How can i compare them and say its matching ? Appreciate your help VAR1=describe/read/write VAR2=read/write/describeThanks, Please use CODE tags as required by forum rules! (4 Replies)
Discussion started by: prince1987
4 Replies

10. UNIX for Beginners Questions & Answers

Comparing two files and list the difference with common first line content of both files

I have two file as given below which shows the ACL permissions of each file. I need to compare the source file with target file and list down the difference as specified below in required output. Can someone help me on this ? Source File ************* # file: /local/test_1 # owner: own #... (4 Replies)
Discussion started by: sarathy_a35
4 Replies
LDIF(5) 							File Formats Manual							   LDIF(5)

NAME
ldif - LDAP Data Interchange Format DESCRIPTION
The LDAP Data Interchange Format (LDIF) is used to represent LDAP entries and change records in text form. LDAP tools, such as ldapadd(1) and ldapsearch(1), read and write LDIF entry records. ldapmodify(1) reads LDIF change records. This manual page provides a basic description of LDIF. A formal specification of LDIF is published in RFC 2849. ENTRY RECORDS
LDIF entry records are used to represent directory entries. The basic form of an entry record is: dn: <distinguished name> <attrdesc>: <attrvalue> <attrdesc>: <attrvalue> <attrdesc>:: <base64-encoded-value> <attrdesc>:< <URL> ... The value may be specified as UTF-8 text or as base64 encoded data, or a URI may be provided to the location of the attribute value. A line may be continued by starting the next line with a single space or tab, e.g., dn: cn=Barbara J Jensen,dc=exam ple,dc=com Lines beginning with a sharp sign ('#') are ignored. Multiple attribute values are specified on separate lines, e.g., cn: Barbara J Jensen cn: Babs Jensen If an value contains a non-printing character, or begins with a space or a colon ':', the <attrtype> is followed by a double colon and the value is encoded in base 64 notation. e.g., the value " begins with a space" would be encoded like this: cn:: IGJlZ2lucyB3aXRoIGEgc3BhY2U= If the attribute value is located in a file, the <attrtype> is followed by a ':<' and a file: URI. e.g., the value contained in the file /tmp/value would be listed like this: cn:< file:///tmp/value Other URI schemes (ftp,http) may be supported as well. Multiple entries within the same LDIF file are separated by blank lines. ENTRY RECORD EXAMPLE
Here is an example of an LDIF file containing three entries. dn: cn=Barbara J Jensen,dc=example,dc=com cn: Barbara J Jensen cn: Babs Jensen objectclass: person description:< file:///tmp/babs sn: Jensen dn: cn=Bjorn J Jensen,dc=example,dc=com cn: Bjorn J Jensen cn: Bjorn Jensen objectclass: person sn: Jensen dn: cn=Jennifer J Jensen,dc=example,dc=com cn: Jennifer J Jensen cn: Jennifer Jensen objectclass: person sn: Jensen jpegPhoto:: /9j/4AAQSkZJRgABAAAAAQABAAD/2wBDABALD A4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQ ERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVG ... Note that the description in Barbara Jensen's entry is read from file:///tmp/babs and the jpegPhoto in Jennifer Jensen's entry is encoded using base 64. CHANGE RECORDS
LDIF change records are used to represent directory change requests. Each change record starts with line indicating the distinguished name of the entry being changed: dn: <distinguishedname> changetype: <[modify|add|delete|modrdn]> Finally, the change information itself is given, the format of which depends on what kind of change was specified above. For a changetype of modify, the format is one or more of the following: add: <attributetype> <attrdesc>: <value1> <attrdesc>: <value2> ... - Or, for a replace modification: replace: <attributetype> <attrdesc>: <value1> <attrdesc>: <value2> ... - If no attributetype lines are given to replace, the entire attribute is to be deleted (if present). Or, for a delete modification: delete: <attributetype> <attrdesc>: <value1> <attrdesc>: <value2> ... - If no attributetype lines are given to delete, the entire attribute is to be deleted. For a changetype of add, the format is: <attrdesc1>: <value1> <attrdesc1>: <value2> ... <attrdescN>: <value1> <attrdescN>: <value2> For a changetype of modrdn or moddn, the format is: newrdn: <newrdn> deleteoldrdn: 0 | 1 newsuperior: <DN> where a value of 1 for deleteoldrdn means to delete the values forming the old rdn from the entry, and a value of 0 means to leave the val- ues as non-distinguished attributes in the entry. The newsuperior line is optional and, if present, specifies the new superior to move the entry to. For a changetype of delete, no additional information is needed in the record. Note that attribute values may be presented using base64 or in files as described for entry records. Lines in change records may be con- tinued in the manner described for entry records as well. CHANGE RECORD EXAMPLE
The following sample LDIF file contains a change record of each type of change. dn: cn=Babs Jensen,dc=example,dc=com changetype: add objectclass: person objectclass: extensibleObject cn: babs cn: babs jensen sn: jensen dn: cn=Babs Jensen,dc=example,dc=com changetype: modify add: givenName givenName: Barbara givenName: babs - replace: description description: the fabulous babs - delete: sn sn: jensen - dn: cn=Babs Jensen,dc=example,dc=com changetype: modrdn newrdn: cn=Barbara J Jensen deleteoldrdn: 0 newsuperior: ou=People,dc=example,dc=com dn: cn=Barbara J Jensen,ou=People,dc=example,dc=com changetype: delete INCLUDE STATEMENT
The LDIF parser has been extended to support an include statement for referencing other LDIF files. The include statement must be sepa- rated from other records by a blank line. The referenced file is specified using a file: URI and all of its contents are incorporated as if they were part of the original LDIF file. As above, other URI schemes may be supported. For example: dn: dc=example,dc=com objectclass: domain dc: example include: file:///tmp/example.com.ldif dn: dc=example,dc=org objectclass: domain dc: example This feature is not part of the LDIF specification in RFC 2849 but is expected to appear in a future revision of this spec. It is supported by the ldapadd(1), ldapmodify(1), and slapadd(8) commands. SEE ALSO
ldap(3), ldapsearch(1), ldapadd(1), ldapmodify(1), slapadd(8), slapcat(8), slapd-ldif(5), slapd.replog(5). "LDAP Data Interchange Format," Good, G., RFC 2849. ACKNOWLEDGEMENTS
OpenLDAP Software is developed and maintained by The OpenLDAP Project <http://www.openldap.org/>. OpenLDAP Software is derived from Uni- versity of Michigan LDAP 3.3 Release. OpenLDAP 2.4.28 2011/11/24 LDIF(5)
All times are GMT -4. The time now is 08:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy