Shell Programming and Scripting

Hi,

The problem here is if suppose for example this is one line in file1

Row Count:96 SUM(F1): 3739 MAX(F1):77 MIN(F1): 0 AVG(F1): 38.9479167 LENGTH(LINE): 2260.

This is a single line in file1.

But in the other file this one line is broken into two.
Row Count:96 SUM(F1): 3739 MAX(F1):77 MIN(F1): 0 AVG(F1): 38.9479167
LENGTH(LINE): 2260
Hence the script is showing this as a mismatch.

Can this be ignored and compared?

I think this is one of the problem.

Please help.

Can anyone help me please?

This is a single line in the file2.txt you posted. However, there are other examples where a long line has been folded (e.g. AQ$_FT_Q_BECMD_I in file1.txt)

I would propose that you normaize file1.txt and file2.txt so that these differences are removed. Change runs of more than one space to a single space, and remove a newline just before tilde (optionally with spaces before it). Then compare the resulting files instead.

Code:

perl -0777 -pi~ -e 's/ *\n *~/ ~/g; s/  */ /g' file1.txt file2.txt

The original files will be left as file1.txt~ and file2.txt~ in case you need to get them back.

(This will also modify the double spaces in the Start Time: and End Time: lines; hope that's not a problem.)

Hi era,

When i ran that code before running my comparison code
perl -0777 -pi~ -e 's/ *\n *~/ ~/g; s/ */ /g' file1.txt file2.txt

All lines starting with that tilde "~~" symbol were removed off for tables having huge number of columns.I dont want this to happen. They were not getting added to a single line.I think it is because of the line limit of the file.

Few lines are getting deleted.

They are not removed, they are merged with the previous line. However, seems that you have DOS carriage returns in there too, so you need to change \n to \r?\n in the script -- sorry for missing that.

The file format is kind of messy, so you might need additional normalizations still. Possibly add something like*s/:\s*(\d+)\s*(~|$)/: $1 $2/g to make all numbers between a colon and a tilde (or end of line) separated by whitespace on both sides.

perl -0777 -pi~ -e 's/ *\r?\n *~/ ~/g; s/ */ /g'; *s/:\s*(\d+)\s*(~|$)/: $1 $2/g file1.txt file2.txt.

I ran this code.Its giving syntax error
bash: syntax error near unexpected token `*s/:\s*(\d+)'

will lines Starting with the field values or field other than tilde i.e starting with number or name get merged to the previous line?Lines starting with field value or field name are not getting merged to the previous line.Only lines starting with tilde are getting merged.

How to solve this?