Sponsored Content
Full Discussion: Comparing 2 huge text files
Top Forums Shell Programming and Scripting Comparing 2 huge text files Post 302523838 by linuxgeek on Friday 20th of May 2011 03:51:58 AM
Old 05-20-2011
hi ctsgnb,

i have this additional requirement and here goes:

Code:
awk 'NR==FNR{if (/^uid:/) a[$2];next}{x=$0;sub("@.*","",x);if(!(x in a)) print $1}' access.ldif k5login

from the above script...i need to save the script output to temporarily file let say k5login-temp and then parse it or compare it again to another ldif file let say bny.ldif. I'm having a hard time for a modified script .

---------- Post updated at 03:51 PM ---------- Previous update was at 03:00 PM ----------

ah... i think i got it..testing now...
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

comparing text files

I am comparing text files where there are number of rows of numbers from window to unix box Is there any way of checking lets say 4 document of text file and seeing the difference only (or missing rows of numbers) with simple commands with lets say a batch file FROM ABSOULTE... (2 Replies)
Discussion started by: sjumma
2 Replies

2. Solaris

Huge (repeated Entry) text files

Somebody HELP! I have a huge log file (TEXT) 76298035 bytes. It's a logfile of IMEIs and IMSIS that I get from my EIR node. Here is how the contents of the file look like: 000000, 1 33016382000913 652020100423994 1 33016382002353 652020100430743 1 33017035101003 652020100441736... (4 Replies)
Discussion started by: axl
4 Replies

3. AIX

comparing within text files

hi! some looping problem here... i have a 2-column text file 4835021 20060903FAL0132006 4835021 20060904FAL0132006 4835021 20060905FAL0132006 4835023 20060903FAL0132006 4835023 20061001HAL0132006 4835023 ... (3 Replies)
Discussion started by: d3ck_tm
3 Replies

4. UNIX for Dummies Questions & Answers

comparing Huge Files - Performance is very bad

Hi All, Can you please help me in resolving the following problem? My requirement is like this: 1) I have two files YESTERDAY_FILE and TODAY_FILE. Each one is having nearly two million data. 2) I need to check each record of TODAY_FILE in YESTERDAY_FILE. If exists we can skip that by... (5 Replies)
Discussion started by: madhukalyan
5 Replies

5. Shell Programming and Scripting

Comparing two huge files

Hi, I have two files file A and File B. File A is a error file and File B is source file. In the error file. First line is the actual error and second line gives the information about the record (client ID) that throws error. I need to compare the first field (which doesnt start with '//') of... (11 Replies)
Discussion started by: kmkbuddy_1983
11 Replies

6. UNIX for Advanced & Expert Users

Best way to search for patterns in huge text files

I have the following situation: a text file with 50000 string patterns: abc2344536 gvk6575556 klo6575556 .... and 3 text files each with more than 1 million lines: ... 000000 abc2344536 46575 0000 000000 abc2344536 46575 4444 000000 abc2344555 46575 1234 ... I... (8 Replies)
Discussion started by: andy2000
8 Replies

7. Shell Programming and Scripting

comparing to text files

Hi All, I have two files of the following formats file 1 - this is a big file >AB_1 gi|229194403|ref|ZP_04321208.1| group II intron reverse transcriptase/maturase gdfjafhlkhlnlklaklskckcfhhahgfahajfkkallalfafafa >AB_2 gi|229194404|ref|ZP_04321209.1| gfksjgfkjsfjslfslfslhf >AB_3... (1 Reply)
Discussion started by: Lucky Ali
1 Replies

8. Shell Programming and Scripting

Comparing two huge files on field basis.

Hi all, I have two large files and i want a field by field comparison for each record in it. All fields are tab seperated. file1: Email SELVAKUMAR RAMACHANDRAN Email SHILPA SAHU Web NIYATI SONI Web NIYATI SONI Email VIINII DOSHI Web RAJNISH KUMAR Web ... (4 Replies)
Discussion started by: Suman Singh
4 Replies

9. Shell Programming and Scripting

How to fix line breaks format text for huge files?

Hi, I need to correct line breaks for huge files (more than 1MM records in a file) and then format it properly. Except the header and trailer, each record starts with 'D'. Requirement:Scan the whole file except the header and trailer records and see if any of the records start with... (19 Replies)
Discussion started by: kikionline
19 Replies

10. Shell Programming and Scripting

Perl: Need help comparing huge files

What do i need to do have the below perl program load 205 million record files into the hash. It currently works on smaller files, but not working on huge files. Any idea what i need to do to modify to make it work with huge files: #!/usr/bin/perl $ot1=$ARGV; $ot2=$ARGV; open(mfileot1,... (12 Replies)
Discussion started by: mrn6430
12 Replies
DB2LDIF.PL(8)						      System Manager's Manual						     DB2LDIF.PL(8)

NAME
db2ldif.pl - Directory Server perl script for exporting a database to a LDIF file SYNOPSIS
db2ldif.pl [-Z serverID] [-D rootdn] { -w password | -w - | -j pwfilename } [-P protocol] {-n backendname}* | {-s includeSuffix}* [{-x excludeSuffix}*] [-m] [-M] [-r] [-u] [-C] [-N] [-E] [-1] [-U] [-a filename] [-v] [-h] DESCRIPTION
Exports the contents of the database to LDIF. This script creates an entry in the directory that launches this dynamic task. OPTIONS
A summary of options is included below: -Z Server Identifier The server ID of the Directory Server instance. If there is only one instance on the system, this option can be skipped. -D Root DN The Directory Manager DN, or root DN. If not specified, the script will search the server instance configuration for the value. -w password The rootdn password. -w - Prompt for the rootdn password. -j password filename The name of the file that contains the root DN password. -n Backend Name The name of the LDBM database to restore. Example: userRoot -s includeSuffix The suffix DN to be included in the exported LDIF file. -x excludeSuffix The suffix DN to be excluded from the exported LDIF file. -a filename Name for the exported LDIF file. -P Protocol The connection protocol to connect to the Directory Server. Protocols are STARTTLS, LDAPS, LDAPI, and LDAP. If this option is skipped, the most secure protocol that is available is used. For LDAPI, AUTOBIND is also available for the root user. -m Minimize the base64 encodings in the exported LDIF file. -M Exported ldif is stored in multiple files. These files are named : <instance>_<filename> By default, all instances are stored in the filename specified by the -a option. -r Export replication data(information required to initialize a replica when the LDIF is imported). -u Do not export the unique-id attribute. -U Requests that the output LDIF is not folded. -C Uses only the main database file. -N Suppress printing sequential number -E Decrypts any encrypted data during export. -1 Deletes, for reasons of backward compatibility, the first line of the LDIF file that gives the version of the LDIF standard. -v Display verbose ouput -h Display usage EXAMPLE
db2ldif.pl -Z instance3 -n userRoot -a /LDAP/ldif/export.ldif -P STARTTLS Note: security must be enabled to use protocol STARTTLS. If STARTTLS is not available it will default to next strongest/available protocol automatically. db2ldif.pl -s 'dc=example,dc=com' -a /LDAP/ldif/export.ldif -m -E -q DIAGNOSTICS
Exit status is zero if no errors occur. Errors result in a non-zero exit status and a diagnostic message being written to standard error. AUTHOR
db2ldif.pl was written by the 389 Project. REPORTING BUGS
Report bugs to http://bugzilla.redhat.com. COPYRIGHT
Copyright (C) 2013 Red Hat, Inc. Mar 5, 2013 DB2LDIF.PL(8)
All times are GMT -4. The time now is 05:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy