10-25-2015
Comparing 2 UNIX files
Hi,
I am planning to automate the comparison of data between few tables in 2 different databases ( Teradata and sql server).
Below is the approach which I think of. Please suggest any improvements/Modification :
1) The sql server file is having more records and I have to eliminate the duplicates based on the unique key and select only few sample records for comparison. The file is pipe delimited. I am thinking of using the sort -u command to eliminate the duplicates and then will select few sample records based on the unique key.
2) Fetch the unique key from the ssql file into a new input file and then using teradata BTEQ utility will export the records corresponding to the unique key in ssql file.
3) use comp command to do column wise comparison for the table. Is there any way I can highlight the columns not matching, like in excel we can compare the columns and hightlight the values having "FALSE" result.
4) This needs to be done for around 10 tables and the structure of the tables is similar in both the databases.
Thanks in Advance!
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hi
I need to compare shadow file sizes with their real file counterparts. If the shadow file size differs form the realfile size then it must send a mail. My problem is that our system has over 1600 shadowfiles in different directories, with different names. the only consistancy is the .sh file... (4 Replies)
Discussion started by: terrym
4 Replies
2. Shell Programming and Scripting
So I have two files:
File1
pictures.txt 1.1 1.3
dance.txt 1.2 1.4
treehouse.txt 1.3 1.5
File2
pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244
dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2 ref4948 1.1
treehouse.txt 1.6 ref8573 1.5 ref3284 1.4 ref5838... (24 Replies)
Discussion started by: linuxkid
24 Replies
3. Shell Programming and Scripting
I've two files with data like below:
file1.txt:
AAA,Apples,123
BBB,Bananas,124
CCC,Carrot,125
file2.txt:
Store1|AAA|123|11
Store2|BBB|124|23
Store3|CCC|125|57
Store4|DDD|126|38
So,the field separator in file1.txt is a comma and in file2.txt,it is |
Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies
4. UNIX for Advanced & Expert Users
Hi Guys ,
we have one directory ...in that directory all files will be set on each day..
files must have header ,contents ,footer..
i wants to compare the header,contents,footer ..if its same means display an error message as 'files contents same' (7 Replies)
Discussion started by: Venkatesh1
7 Replies
5. Shell Programming and Scripting
I hope I can explain this correctly. I am using Bash-4.2 for my shell.
I have a group of file names held in an array. I want to compare the names in this array against the names of files currently present in a directory. If the file does not exist in the directory, that is not a problem.... (5 Replies)
Discussion started by: BudMan
5 Replies
6. Shell Programming and Scripting
Hi , I want to compare first 3 columns of File A and File B and create a new file File C which will have all rows from File B and will include rows that are present in File A and not in File B based on First 3 column comparison.
Thanks in advance for your help.
File A
A,B,C,45,46... (2 Replies)
Discussion started by: ady_koolz
2 Replies
7. Shell Programming and Scripting
Hi Everyone,
I am comparing results of two environments using unix files.
I am writing two different csv file using spool and merging both the files using paste command
paste -d, file1.csv file2.csv > prod_uat_result.csv
and then finding the difference and attaching the same in a mail... (8 Replies)
Discussion started by: karthik adiga
8 Replies
8. Shell Programming and Scripting
I have 2 files namely branch.txt file & RXD.txt file as below
Ex:Branch.txt
=========================
B1,Branchname1,city,country
B2,Branchname2,city,country
B3,Branchname3,city,country
B4,Branchname4,city,country
B5,Branchname5,city,country
RXD file : will... (11 Replies)
Discussion started by: satece
11 Replies
9. UNIX for Beginners Questions & Answers
Hi,
I have 2 variables as given below. How can i compare them and say its matching ? Appreciate your help
VAR1=describe/read/write
VAR2=read/write/describeThanks,
Please use CODE tags as required by forum rules! (4 Replies)
Discussion started by: prince1987
4 Replies
10. UNIX for Beginners Questions & Answers
I have two file as given below which shows the ACL permissions of each file. I need to compare the source file with target file and list down the difference as specified below in required output. Can someone help me on this ?
Source File
*************
# file: /local/test_1
# owner: own
#... (4 Replies)
Discussion started by: sarathy_a35
4 Replies
LEARN ABOUT OPENDARWIN
lsort
lsort(n) Tcl Built-In Commands lsort(n)
__________________________________________________________________________________________________________________________________________________
NAME
lsort - Sort the elements of a list
SYNOPSIS
lsort ?options? list
_________________________________________________________________
DESCRIPTION
This command sorts the elements of list, returning a new list in sorted order. The implementation of the lsort command uses the merge-sort
algorithm which is a stable sort that has O(n log n) performance characteristics.
By default ASCII sorting is used with the result returned in increasing order. However, any of the following options may be specified
before list to control the sorting process (unique abbreviations are accepted):
-ascii Use string comparison with Unicode code-point collation order (the name is for backward-compatability reasons.) This
is the default.
-dictionary Use dictionary-style comparison. This is the same as -ascii except (a) case is ignored except as a tie-breaker and (b)
if two strings contain embedded numbers, the numbers compare as integers, not characters. For example, in -dictionary
mode, bigBoy sorts between bigbang and bigboy, and x10y sorts between x9y and x11y.
-integer Convert list elements to integers and use integer comparison.
-real Convert list elements to floating-point values and use floating comparison.
-command command Use command as a comparison command. To compare two elements, evaluate a Tcl script consisting of command with the two
elements appended as additional arguments. The script should return an integer less than, equal to, or greater than
zero if the first element is to be considered less than, equal to, or greater than the second, respectively.
-increasing Sort the list in increasing order (``smallest'' items first). This is the default.
-decreasing Sort the list in decreasing order (``largest'' items first).
-index index If this option is specified, each of the elements of list must itself be a proper Tcl sublist. Instead of sorting
based on whole sublists, lsort will extract the index'th element from each sublist and sort based on the given element.
The keyword end is allowed for the index to sort on the last sublist element, and end-index sorts on a sublist element |
offset from the end. For example,
lsort -integer -index 1 {{First 24} {Second 18} {Third 30}}
returns {Second 18} {First 24} {Third 30}, and |
lsort -index end-1 {{a 1 e i} {b 2 3 f g} {c 4 5 6 d h}} |
returns {c 4 5 6 d h} {a 1 e i} {b 2 3 f g}. This option is much more efficient than using -command to achieve the
same effect.
-unique If this option is specified, then only the last set of duplicate elements found in the list will be retained. Note
that duplicates are determined relative to the comparison used in the sort. Thus if -index 0 is used, {1 a} and {1 b}
would be considered duplicates and only the second element, {1 b}, would be retained.
NOTES
The options to lsort only control what sort of comparison is used, and do not necessarily constrain what the values themselves actually
are. This distinction is only noticeable when the list to be sorted has fewer than two elements.
The lsort command is reentrant, meaning it is safe to use as part of the implementation of a command used in the -command option.
EXAMPLES
Sorting a list using ASCII sorting:
% lsort {a10 B2 b1 a1 a2}
B2 a1 a10 a2 b1
Sorting a list using Dictionary sorting:
% lsort -dictionary {a10 B2 b1 a1 a2}
a1 a2 a10 b1 B2
Sorting lists of integers:
% lsort -integer {5 3 1 2 11 4}
1 2 3 4 5 11
% lsort -integer {1 2 0x5 7 0 4 -1}
-1 0 1 2 4 0x5 7
Sorting lists of floating-point numbers:
% lsort -real {5 3 1 2 11 4}
1 2 3 4 5 11
% lsort -real {.5 0.07e1 0.4 6e-1}
0.4 .5 6e-1 0.07e1
Sorting using indices:
% # Note the space character before the c
% lsort {{a 5} { c 3} {b 4} {e 1} {d 2}}
{ c 3} {a 5} {b 4} {d 2} {e 1}
% lsort -index 0 {{a 5} { c 3} {b 4} {e 1} {d 2}}
{a 5} {b 4} { c 3} {d 2} {e 1}
% lsort -index 1 {{a 5} { c 3} {b 4} {e 1} {d 2}}
{e 1} {d 2} { c 3} {b 4} {a 5}
Stripping duplicate values using sorting:
% lsort -unique {a b c a b c a b c}
a b c
More complex sorting using a comparison function:
% proc compare {a b} {
set a0 [lindex $a 0]
set b0 [lindex $b 0]
if {$a0 < $b0} {
return -1
} elseif {$a0 > $b0} {
return 1
}
return [string compare [lindex $a 1] [lindex $b 1]]
}
% lsort -command compare
{{3 apple} {0x2 carrot} {1 dingo} {2 banana}}
{1 dingo} {2 banana} {0x2 carrot} {3 apple}
SEE ALSO
list(n), lappend(n), lindex(n), linsert(n), llength(n), lsearch(n), lset(n), lrange(n), lreplace(n) |
KEYWORDS
element, list, order, sort
Tcl 8.3 lsort(n)