Add missing times


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Add missing times
# 1  
Old 01-15-2010
Add missing times

Not sure about the title if someone has a better name for it please lemme know and I will edit the title.

I have several (10+ files) which look something like:

Code:
File 1:
12/28/2009 04:0 8
12/28/2009 04:4 4
12/28/2009 05:0 4
.
.
.

File 2:
12/28/2009 04:1 7
12/28/2009 04:2 3
12/28/2009 05:0 2
.
.
.

10+ files all like this

The fields are obviously date, time (hh:m) and lastly occurrences of an event I want to check on a per 10 minute basis

Now as you can tell the event does not necessarily happen in a given 10 minute sample so I have gaps.

Eventually I want to graph this in a spreadsheet which has all the 10 minutes increments from startdate/starttime to enddate/endtime.
So I need to end up with something which changes this into.

Code:
File 1:
12/28/2009 04:0 8
12/28/2009 04:1 0
12/28/2009 04:2 0
12/28/2009 04:3 0
12/28/2009 04:4 4
12/28/2009 04:5 0
12/28/2009 05:0 4
.
.
.

File 2:
12/28/2009 04:1 7
12/28/2009 04:2 3
12/28/2009 04:3 0
12/28/2009 04:4 0
12/28/2009 04:5 0
12/28/2009 05:0 2

I was contemplating having a file "time" which contains every day/10 minute line and merging it somehow but have problems conceptualizing how to make it work.

This is what I tried unsuccessfully (some echo's in there for debugging purpose):
Code:
#!/bin/sh
TIME=`cat /tmp/js/time`
TARGET=`cat $1`
for i in "$TIME"
do
        for j in "$TARGET"
        do
                tmp=`echo "$j" |awk '{print $1,$2}'`
                echo "$tmp" |egrep "$i" |egrep "[a-z]"  2>&1 > /dev/null
                if [ $? = 0 ]
                then
                        echo "here"
                        echo "J is"
                        echo "$j"
                        echo "$i"
                else
                        echo "there"
                        echo "i is"
                        echo "$i"
                        echo "$j"
                fi
        done
done

Hope that makes sense.... any input welcome.
# 2  
Old 01-15-2010
You could go with your original thought - generate a file that has all the 10 minute intervals for the day. As you generate the file, grep your other existing files to pull out the lines that match the Date/Timestamp you are working on. Add the value found in the existing file to your total for that time slot.

Here's how it might look. This generate the every-10-minute data (instead of having it already in a file). Also I am going to use standard hh:mm (2-digit minutes) for the time format. You may need to adjust this to match the data in file1 file2

Code:
#!/bin/ksh 
## first - lets put today's date in a variable
TODAY=`date +"%m/%d/%Y"`

## Setup a loop for all the HOURs in a day
for HOUR in 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
do
   ## Setup a Loop for each 10-minute interval
   for MIN in 00 10 20 30 40 50
   do
 
      ## Set up a counter
      typeset -i COUNTER=0

      ## Print out the beginning of the line
      echo "${TODAY} ${HOUR}:${MIN} \c"
      
      ## look in the file to find any entries for the DAY HOUR:MIN we are working on 
	  ## save the values into a file for processing.  
	  ## If you have more than 1 file, add file2 file3 after file1 in the grep
      grep "${TODAY} ${HOUR}:${MIN}" file1 | cut -d" " -f3 > /tmp/$$_values

      ## go through all the values found and add them to the counter
      for VALUE in $(cat /tmp/$$_values)
      do 
         (( COUNTER=${COUNTER}+${VALUE} ))
      done

      ## Print out the counter and end the line
      echo "${COUNTER}"

   done
done

##clean up the tmp file if it exists. suppress the error if no such file exists
find /tmp -name $$_values -exec rm {} \; 2>/dev/null


I'm sure others will suggest sexier ways to accomplish this...but hey - it works and can be understood in 2 years when you look at this code again and wonder "what was I doing???".

Let me know if this helps you...
# 3  
Old 01-16-2010
Quote:
Originally Posted by jstrangfeld
Code:
File 1:
12/28/2009 04:0 8
12/28/2009 04:4 4
12/28/2009 05:0 4
.
.
.

File 2:
12/28/2009 04:1 7
12/28/2009 04:2 3
12/28/2009 05:0 2

Eventually I want to graph this in a spreadsheet which has all the 10 minutes increments from startdate/starttime to enddate/endtime.
The problem here is that your input don't match your output and your problem is unclear.

Do you want to merge all 10+ file or process one by one ?

My first try will be:
Code:
# cat file
12/28/2009 04:0 8
12/28/2009 04:4 4
12/28/2009 05:0 4
12/28/2009 04:1 7
12/28/2009 04:2 3
12/28/2009 05:0 2

# sort -k2 file | awk -F[\ :] '{sub($3 OFS,a[$2]++ OFS,$0)}1'

12/28/2009 04:0 8
12/28/2009 04:1 7
12/28/2009 04:2 3
12/28/2009 04:3 4
12/28/2009 05:0 2
12/28/2009 05:1 4

Please provide more details, data sample and matching required output.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Add word for a missing column

Hi team... I need some help/advise on adding a word for a missing column. I have a file as below. Rep N1 Forever Limited 2015 Rep N2 Limited 2016 since 2nd content is 3 lines missing one line so i need to make this like Rep N1 Forever (3 Replies)
Discussion started by: newbee5
3 Replies

2. Shell Programming and Scripting

How to add missing value?

Hi Gurus, I have a txt file. some record missed 2nd colum value, I need put default value into the file. ex: below file, first 4 records, missed one column (only 4 columns. it should be 5) $cat missedfield aaa,bbb,ccc,fff aaa,bbb,ccc,ddd 111,222,333,444 111,222,333,444... (3 Replies)
Discussion started by: ken6503
3 Replies

3. Shell Programming and Scripting

sed add space X times

Pattern: Mary walks at the park every day with her children sed 's/$/ /' will make it add 1 space at the end (trailing) I want to add X ($VARIABLE) number of spaces (which comes from RANDOM) i.e. VARIABLE='14' then it will do: sed 's/$/ /' = 14 spaces added at the... (10 Replies)
Discussion started by: holyearth
10 Replies

4. Solaris

[solved]Config/enable_mapping missing, how to add?

Issue resolved by upgrading from solaris 11 to solaris 11.1 I would like to enable network mapping. While using instructions from: https://blogs.oracle.com/VDIpier/entry/solaris_11_changing_the_hostname To change my hostname I noticed I am missing the enable mapping bool. What it should... (0 Replies)
Discussion started by: taltamir
0 Replies

5. UNIX Desktop Questions & Answers

merging files and add missing rows

hello all, I have files that have a specific way for naming the first column they are make of five names in Pattern of 3 Y = (no case sensitive) so the files are names $Y-$Y-$Y or $X-$Y-$Z depending how we look they only exist of the pattern exist now I want to create a file from them that... (9 Replies)
Discussion started by: A-V
9 Replies

6. Shell Programming and Scripting

Add missing linefeeds between formfeeds in reports

I need to take a report text file that is output from vendor software and there are some pages which have less then the normal amount of lines. I need to add these missing line feeds as there is a merge program that reads this file in fixed character and line mode template to output the final PDF... (6 Replies)
Discussion started by: rnygren
6 Replies

7. Programming

Problem with implementing the times() function in C (struct tms times return zero/negative values)

Hello, i'm trying to implement the times() function and i'm programming in C. I'm using the "struct tms" structure which consists of the fields: The tms_utime structure member is the CPU time charged for the execution of user instructions of the calling process. The tms_stime structure... (1 Reply)
Discussion started by: g_p
1 Replies

8. Shell Programming and Scripting

i need to add missing delimiters...

ladies, gents.. say i have a file that should have 10 fields... (9 delimiters) some records have 10 fields, some have 5 some have 8, etc.. nothing consistent, but i need my file to have 9 delimiters on each line, even if its null fields.. how can i go line by line and add the correct... (2 Replies)
Discussion started by: obarrett
2 Replies

9. Shell Programming and Scripting

Awk script to add times.

Hey, Im trying to format the last command to tell me just the user names, logins, and the time that they were logged in. So far I got the users logins using a loop that counts the amount of times a user logged in but im not sure how to start the time array. The time im trying to use is the last... (2 Replies)
Discussion started by: Trilogie
2 Replies
Login or Register to Ask a Question