Shell Programming and Scripting

Hi,

I have a ksh which peocess and get me data from 3 days...
ie if i process it on jan 28.. it gets data for 25, 26 and 27.... the process run every day and get previous 3 days data...all this data is appened to a file lets call time.out
Now time.out cannot have deplicate data so what i want is when ever the appeneding is done i want look for the date in the file if it matches with the date the data is holding then i want it to be replace with new data

the time.out is like this

Code:

.
.
20090124,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090125,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090126,00:02:33,00:00:35,8187,00:01:42,00:01:32,02:02:08,321055,

while in the script we have the current data for which the process is running save in CUR_MAINT_DATE..
the way the output file is written is

Code:

 
Read.ksh `basename ${0}` $1 $2 ${CUR_MAINT_DATE} > runtime.out
echo >> time.out; tr -d '\n' < runtime.out >> time.out

The the script runs it initially get data of 25 ... now i want the script to cross check with the time.out file if the data is already present for that date then it have to replace that line with the data in runtime.out.... and so on till it comes to current maintaince date 27 as no data is there for 27 in time.out the script have put it in the time.out

can anyone say me how i can replace the data in output file with data form input

I've taken your input:

and created two files. The first test1.dat is as above. The second, test2.dat I took the first line above starting with "20090124" and changed that to "20090127" to spoof a date on the 27th, and moved it to the end so it looks like this:

Code:

20090125,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090126,00:02:33,00:00:35,8187,00:01:42,00:01:32,02:02:08,321055, 
20090127,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938,

Then I ran the following 'comm' command:

Code:

comm test1.dat test2.dat | tr -d '\t'

and got the following output:

Code:

20090124,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090125,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090126,00:02:33,00:00:35,8187,00:01:42,00:01:32,02:02:08,321055,
20090127,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938,

Which could be put into it's own file.

comm prints output in three columns. Col 1 is data only in the first file,
col 2 is data only in the second file and Col 3 is data in both files. The "columns" are created using tabs, and there is not more than one column's worth of data on a line. So by removing the tabs, I have output that lines up neatly, and gives me all the unique and non duplicated data from the previous two files.

I am sorry to say but i dint understand anything....

I dont want to compare file and remove duplicates....i want to cheack if that pattern is already present in the file then remove it and replace it with the input....if not just place the input...
The partten here CUR_MAINT_DATE... i want something with grep, sid and awk...

Thanks for the help

Seemed overkill to me to replace a line of data with an exact duplicate, when you can use a command that will get you only the newest data. 'comm' has options on the output.

What if that made things more complicated than they need to be?

Granted I could be missing something that you either haven't mentioned, or I just don't undestand your explanation very well. But if I understand you correctly, the file that should have all the data added to it is 'time.out' and that data is coming from 'runtime.out'

If the structure of both files are the same as you posted in your first post, then the comm command will work for you.

Code:

comm -23 runtime.out time.out >> time.out

You don't even need to run it through tr to get rid of tabs, since only the data in runtime.out not appearing in time.out will be printed in the first column w/o any added tabs.

Yes you are missing something that... time.out file is very old file and it have data form past 5 years... so it have to be the final file and the reason i have used this code is
echo >> time.out; tr -d '\n' < runtime.out >> time.out
because the data in runtime.out is like this

Code:

20090127,
00:02:33,
00:03:04,
62153,
00:01:39,
00:02:25,
01:39:55,
266355,
 ,
00:09:33,
00:00:22,
00:03:17,
00:00:02,
00:05:01,
 ,
 ,
 ,
00:03:21,
00:07:40,
 ,
00:03:38,
00:36:42,
 ,
 ,

Okay, suspected the second part, but you hadn't mentioned the format of runtime.out before so there was no way to know.

As I used comm with the -23 option, it wouldn't care about the old data of time.out at all, so that data would be safe, it would simply get the newest data from runtime.out, but the different file structure is a deal breaker unless you want to create a temporary file to hold data that can be comm'd to time.out.

I realize now that the reason you append a new line before inserting the data from runtime.out piped through tr to time.out is because after the last insertion, there is no newline since tr removes all newlines. I assume this is done on each and every record, so that you do wind up with newlines after each record, except for the last record.

I'd advisde changing this order if possible so the last record has a newline appended to it also, or that would be a deal breaker if you try to find a solution involving 'sed', because sed ignores lines w/o a newline at the end.

So if you make the line look as follows:

Code:

tr -d '\n' < runtime.out >> time.out;echo >> time.out

And then at the command prompt, one time only, before you run the collection script.

Code:

$> echo >> time.out

Then your good to go if you want to find a sed solution. That, however will take more time than I have today for me to look into it.

That really nice of you... thanks a lot...
I dont mind adding another tem file in between runtime.out and time.out if that will slove the problem... I just want some solution this as i have to forward the script by tomm morning to someone else...if not i will be a dealbreaker... I would prefer a sed... but i dont mind any other solution... till it works... please help me...