Urgent - Comma delimited file comparision. Post: 302442994

Sponsored Content

Top Forums Shell Programming and Scripting Urgent - Comma delimited file comparision. Post 302442994 by ravigupta2u on Friday 6th of August 2010 07:17:33 AM

08-06-2010

Registered User

Urgent - Comma delimited file comparision.

Query:
There are two files as below –
/home/rgupta/input/file.txt.arch (source file)
/home/rgupta/output/file.txt (destination file)

Files details are attached for the reference.

Scenario:
File /home/rgupta/input/file.txt.arch picked up by an application xyz. Application xyz does some validation/checks on this file then finally kept at /home/rgupta/output/file.txt.
While copying the file at /home/rgupta/output, Following activities can happen by application xyz –
1) some records can be eliminated
2) There can be some records modified

There are many files with huge number of records with comma/semicolon separated fields.
I need to write a shell script which will compare source file with destination file line by line. And this script should create another file with following data –
1. If record is missing in destination file -
Source file name, destination file name, missing record, missing record number in source
file
2. If record is modified while transferring the file from source to destination –
- Source file name, Data record from source file [first line in new file]
- Destination file name, Data record from Destination file [second line in new file]
- modified record number, exact field which is modified [second line in new file]

I tried following command –
sdiff –s /home/rgupta/input/file.txt.arch /home/rgupta/input/file.txt
Output is –
6015a6016
> 29429293,1,,,01387262543,N,01387262543,N,0,01387262543,N
10000c10001
COUNT=9999 | COUNT=10000;

Output of this command is similar what I expect but the missing record is coming incomplete.

Could someone please provide me the solution?

file_details.doc (26.0 KB)

ravigupta2u

View Public Profile for ravigupta2u

Find all posts by ravigupta2u

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Comma Delimited file

I have a comma delimited file that sometimes has addresses details in. The problem is that the address detail can be seen as: "Sample House, Sample Road". When I run a script specifying the file is comma delimited I would like it to ignore comma's that are in between speech marks. Is this...

2. Shell Programming and Scripting

Converting Tab delimited file to Comma delimited file in Unix

Hi, Can anyone let me know on how to convert a Tab delimited file to Comma delimited file in Unix Thanks!!

3. Shell Programming and Scripting

how to convert this file into comma delimited format

Hi experts, I need urget help! I have the a text file with this format: Types of fruits Name of fruits 1,1 Farm_no,1 apple,1 pineapple,1 grapes,1 orange,1 banana,1 2,2--->this is the record seperator Farm_no,2 apple,1 pineapple,1 grapes,3 orange,2 banana,1 3,3--->this is the...

4. UNIX for Advanced & Expert Users

Urgent! need help! how to convert this file into comma delimited format

5. UNIX for Dummies Questions & Answers

Comma delimited file

Hi All, I have output of sql saved in comma separated file. Now i need to read line by line this file and assign word to a unix variable for further processing Eg: Test file world, 1, 3, 4 earth,2,3,4,5 moon,1,2,3,4 Output should be word1= world word2=1 echo " first word...

6. UNIX for Dummies Questions & Answers

Flat File - Comma Delimited

I have a flat file whose contents are comma delimited and there are 84 columns in total, so everytime I try to view the contents, things get over lapped it becomes diffcult to read through the result set. Is there a command / what would be the best way...if I want to view the results alligned...

7. Shell Programming and Scripting

Comma delimited file manipulation

Question about how to change the first & last name in column one & two so that the names have a capital letter for just the first letter. Example: asdf@asdf.com,asdf,asdfasdf,176.23.22.345,4/12/2012 changed to: asdf@asdf.com,Asdf,Asdfasdf,176.23.22.345,4/12/2012 Thank you kindly, Nick

8. Shell Programming and Scripting

Need a script to convert comma delimited files to semi colon delimited

Hi All, I need a unix script to convert .csv files to .skv files (changing a comma delimited file to a semi colon delimited file). I am a unix newbie and so don't know where to start. The script will be scheduled using cron and needs to convert each .csv file in a particular folder to a .skv...

9. Shell Programming and Scripting

Help/Advise please for converting space delimited string variable to comma delimited with quote

Hi, I am wanting to create a script that will construct a SQL statement based on a a space delimited string that it read from a config file. Example of the SQL will be For example, it will read a string like "AAA BBB CCC" and assign to a variable named IN_STRING. I then concatenate...

10. Shell Programming and Scripting

Modify comma delimited file columns.

Please help me to update a file which contains date values as below:- From:- "1912108",20161130,"2016-12-01-00.00.00.000000","2016-12-01-08.37.12.000000" "1912108",20161201,"2016-12-02-00.00.00.000000","2016-12-02-08.28.22.000000" To:- "1912108",2016-11-30,"2016-12-01...

LEARN ABOUT REDHAT

rasort

RASORT(1)						      General Commands Manual							 RASORT(1)

NAME

       rasort - sort argus(8) data file.

COPYRIGHT

       Copyright (c) 2000-2003 QoSient. All rights reserved.

SYNOPSIS

       rasort [[-M sortmode] [sortmode] ...]  [raoptions]

DESCRIPTION

       Rasort  reads  argus  data  from an argus-data source, sorts the records based on the criteria specified on the command line, and outputs a
       valid argus-stream.

OPTIONS

       Rasort, like all ra based clients, supports a number of ra options including filtering of input argus records through a terminating  filter
       expression.  See ra(1) for a complete description of ra options.  rasort(1) specific options are:

       -M sortmode    Supported sortmodes are:
	      time	     record start time <default>
	      startime	     record start time <default>
	      lasttime	     record last time.
	      trans	     aggregation record count.
	      dur	     record total duration.
	      avgdur	     record average duration.
	      saddr	     source IP addr.
	      daddr	     destination IP addr.
	      proto	     transaction protocol.
	      sport	     source port number.
	      dport	     destination port number.
	      stos	     source TOS byte value.
	      dtos	     destination TOS byte value.
	      sttl	     src -> dst TTL value.
	      dttl	     dst -> src TTL value.
	      bytes	     total transaction bytes.
	      sbytes	     src -> dst transaction bytes.
	      dbytes	     dst -> src transaction bytes.
	      pkts	     total transaction packet count.
	      spkts	     src -> dst packet count.
	      dpkts	     dst -> src packet count.
	      load	     bits per second.
	      loss	     pkts retransmitted or dropped.
	      rate	     pkts per second.
	      tranref	     argus transaction reference number.
	      seq	     argus sequence number.
	      srcid	     argus source identifier.

INVOCATION

       A sample invocation of rasort(1).  This call reads argus(8) data from inputfile and sorts the IP protocol based argus(8) data, first by the
       destination IP address, then by the service (destination) port number and then by the source IP address, and writes the results to  stdout.
       For most services, this arranges argus(8) formatted data by server, service, and then by client.

       rasort -r inputfile -M daddr dport saddr - ip

SEE ALSO

       ra(1), rarc(5), argus(8), tcpdump(1)

FILES

AUTHORS

       Carter Bullard (carter@qosient.com).

BUGS

								 07 November 2000							 RASORT(1)