Need help regarding comparison between two files through UNIX script
Hi All ,
I am aware of unix command ,but not comforable in putting together in script level.I came to situation where I need to compare between two .txt files fieldwise and need a mismatch report. As I am new to unix script arena ,if anyone can help in the below scenario that will be really helpful.
We have one source mainframe .txt file (readble pipe dililimited format) and also have one target hdfs .txt file (pipe dilimited format).I need to compare two files field by field and not by whole line.Need to compare like
f1.first field with f2.first field
f1.second field with f2.second field
and so on .Please find below sample source & target file.
f1 :
f2 :
Once the comparison between two files are complete fieldwise ,we need a mismatch report which will contain source/target count validation ,field level src & target mismatches and their corresponding mismatch details .
It will be helpful if mismatch report like below :
Source data might contain leading spaces/zero ,precision and target data might not have those.We can ignore these cases in the mismatch report.
If anyone can help me in the above scenario that will really beneficial for me.Thanks !
I am very new to Unix. What are the similiarities and differences between ScoUnix and AIX5 if any? Where might i find the information? Which is better? (1 Reply)
Hi, There are two files in UNIX system with some lines are exactly the same, some lines are not.
I want to compare these two files.The 2 files (both the files have data in Column format )should be compared row wise and any difference in data for a particular row should lead to storage of data of... (32 Replies)
Kindly help on follows.
I have 2 files. One file contains only one column of mobile numbers. And total records in a file 12 million. Second file contains 2 columns mobile numbers and balance. and total records 30 million. I want to find out balance of each data in file 1 corresponding to file 2.... (2 Replies)
Hi,
I have a problem with comparison of two files
file1
20100101
20090101
20080101
20071001
20121229
file2
19990112 12 456 7
20011131 19
20100101 2 567 1 987
17890709 123 555
and, sh script needs to compare of these two files and give out to me result:
20100101 2 567 1 987
it... (5 Replies)
Hello,
I have 2 files and I want them to be compared in a specific fashion
file1:
A_1200_1250
A_1251_1300
B_1301_1350
B_1351_1400
B_1401_1450
C_1451_1500 and so on...
file2:
1210 1305 1260 1295
1400 1500 1450 1495
Now The script should look for "1200" from A_1200_1250 of... (8 Replies)
Hi All,
For past some days iam trying, which not able to get to..so please help me on this..
My exact requirement is...
Step1: Find how many files/sub files exist in /some/path (maybe in multiple path)
Step2: Count the no. of files/sub files with their respective size.
Step3: Then a file... (1 Reply)
Hi All,
For past some days iam trying, which not able to get to..so please help me on this..
My exact requirement is...
Step1: Find how many files/sub files exist in /some/path (maybe in multiple path)
Step2: Count the no. of files/sub files with their respective size.
Step3: Then a file... (0 Replies)
Hi All ,
As I am new to unix scripting ,I need a help regarding unix scripting .I have two .txt files .One is source file and another is target file.I need a script through which I can compare those two files.I need a automated comparison report in a directory after comparing between source &... (2 Replies)
I have two files which has component name and version number separated by a space
cat file1
com.acc.invm:FNS_PROD 94.0.5
com.acc.invm:FNS_TEST_DCCC_Mangment 94.1.6
com.acc.invm:FNS_APIPlat_BDMap 100.0.9
com.acc.invm:SendEmail 29.6.113
com.acc.invm:SendSms 12.23.65
cat file2 ... (8 Replies)
I have the requirement
I have two files
cat fileA
something
anythg
nothing
everythg
cat fileB
everythg
anythg
Now i shld use fileB and compare every line at fileA and get the output as
something
nothing (3 Replies)
Discussion started by: Priya Amaresh
3 Replies
LEARN ABOUT OPENSOLARIS
join
join(1) User Commands join(1)NAME
join - relational database operator
SYNOPSIS
join [-a filenumber | -v filenumber] [-1 fieldnumber]
[-2 fieldnumber] [-o list] [-e string] [-t char] file1 file2
join [-a filenumber] [-j fieldnumber] [-j1 fieldnumber]
[-j2 fieldnumber] [-o list] [-e string] [-t char] file1 file2
DESCRIPTION
The join command forms, on the standard output, a join of the two relations specified by the lines of file1 and file2.
There is one line in the output for each pair of lines in file1 and file2 that have identical join fields. The output line normally con-
sists of the common field, then the rest of the line from file1, then the rest of the line from file2. This format can be changed by using
the -o option (see below). The -a option can be used to add unmatched lines to the output. The -v option can be used to output only
unmatched lines.
The default input field separators are blank, tab, or new-line. In this case, multiple separators count as one field separator, and leading
separators are ignored. The default output field separator is a blank.
If the input files are not in the appropriate collating sequence, the results are unspecified.
OPTIONS
Some of the options below use the argument filenumber. This argument should be a 1 or a 2 referring to either file1 or file2, respectively.
-a filenumber In addition to the normal output, produce a line for each unpairable line in file filenumber, where filenumber is 1 or
2. If both -a 1 and -a 2 are specified, all unpairable lines will be output.
-e string Replace empty output fields in the list selected by option -o with the string string.
-j fieldnumber Equivalent to -1fieldnumber -2fieldnumber.
-j1 fieldnumber Equivalent to -1fieldnumber.
-j2 fieldnumber Equivalent to -2fieldnumber. Fields are numbered starting with 1.
-o list Each output line includes the fields specified in list. Fields selected by list that do not appear in the input will be
treated as empty output fields. (See the -e option.) Each element of which has the either the form filenumber.fieldnum-
ber, or 0, which represents the join field. The common field is not printed unless specifically requested.
-t char Use character char as a separator. Every appearance of char in a line is significant. The character char is used as the
field separator for both input and output. With this option specified, the collating term should be the same as sort
without the -b option.
-v filenumber Instead of the default output, produce a line only for each unpairable line in filenumber, where filenumber is 1 or 2.
If both -v 1 and -v 2 are specified, all unpairable lines will be output.
-1 fieldnumber Join on the fieldnumberth field of file 1. Fields are decimal integers starting with 1.
-2fieldnumber Join on the fieldnumberth field of file 2. Fields are decimal integers starting with 1.
OPERANDS
The following operands are supported:
file1
file2 A path name of a file to be joined. If either of the file1 or file2 operands is -, the standard input is used in its place.
file1 and file2 must be sorted in increasing collating sequence as determined by LC_COLLATE on the fields on which they are to be joined,
normally the first in each line (see sort(1)).
USAGE
See largefile(5) for the description of the behavior of join when encountering files greater than or equal to 2 Gbyte (2^31 bytes).
EXAMPLES
Example 1 Joining the password file and group file
The following command line will join the password file and the group file, matching on the numeric group ID, and outputting the login name,
the group name and the login directory. It is assumed that the files have been sorted in ASCII collating sequence on the group ID fields.
example% join -j1 4-j2 3 -o 1.1 2.1 1.6 -t:/etc/passwd /etc/group
Example 2 Using the -o option
The -o 0 field essentially selects the union of the join fields. For example, given file phone:
!Name Phone Number
Don +1 123-456-7890
Hal +1 234-567-8901
Yasushi +2 345-678-9012
and file fax:
!Name Fax Number
Don +1 123-456-7899
Keith +1 456-789-0122
Yasushi +2 345-678-9011
where the large expanses of white space are meant to each represent a single tab character), the command:
example% join -t"tab" -a 1 -a 2 -e '(unknown)' -o 0,1.2,2.2 phone fax
would produce
!Name Phone Number Fax Number
Don +1 123-456-7890 +1 123-456-7899
Hal +1 234-567-8901 (unknown
Keith (unknown) +1 456-789-012
Yasushi +2 345-678-9012 +2 345-678-9011
ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of join: LANG, LC_ALL, LC_CTYPE, LC_MES-
SAGES, LC_COLLATE, and NLSPATH.
EXIT STATUS
The following exit values are returned:
0 All input files were output successfully.
>0 An error occurred.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWcsu |
+-----------------------------+-----------------------------+
|CSI |Enabled |
+-----------------------------+-----------------------------+
|Interface Stability |Standard |
+-----------------------------+-----------------------------+
SEE ALSO awk(1), comm(1), sort(1), uniq(1), attributes(5), environ(5), largefile(5), standards(5)NOTES
With default field separation, the collating sequence is that of sort -b; with -t, the sequence is that of a plain sort.
The conventions of the join, sort, comm, uniq, and awk commands are wildly incongruous.
SunOS 5.11 8 Feb 2000 join(1)