Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need to validate that all 24 hr are in a table

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need to validate that all 24 hr are in a table
# 1  
Old 05-02-2011
Need to validate that all 24 hr are in a table - Is AWK right tool?
I have a table that looks like this, but for a whole year:

Code:
Hourly weather history for XXX 7 Jun
0:00	Clear weather    28
1:00	Clear weather    23
2:00	Clear weather    21
3:00	Clear weather    22
4:00	Clear weather    22
5:00	Clear weather    22
6:00	Clear weather    23
7:00	Clear weather    23
9:00	Clear weather    28
10:00	Clear weather    30
11:00	Clear weather    32
12:00	Clear weather    33
14:00	Clear weather    35
15:00	Clear weather    34
16:00	Clear weather    34
17:00	Clear weather    34
18:00	Clear weather    32
19:00	Clear weather    33
20:00	Clear weather    31
21:00	Clear weather    30
22:00	Clear weather    30
23:00	Clear weather    28
Hourly weather history for XXX 8 Jun
0:00	Clear weather    29
1:00	Clear weather    28
2:00	Clear weather    27
6:00	Clear weather    25
8:00	Clear weather    29
9:00	Clear weather    29
10:00	Clear weather    32
11:00	Clear weather    34
12:00	Clear weather    35
13:00	Clear weather    35
15:00	Clear weather    37
17:00	Clear weather    36
18:00	Clear weather    34
19:00	Clear weather    32
20:00	Clear weather    31
21:00	Clear weather    30
22:00	Clear weather    30
23:00	Clear weather    30

If you look carefully, you'll see the first day has all 24 hours accounted for, but the 2nd day is missing 0300-0500, 0700, 1400 and 1600

I'm trying to loop through the list and check if the time on line N+1 is equal to TIME +1. If not, insert that line (just the time)

So far I have this:
Code:
#! /bin/awk -f

{
if ($1 ~ /Hourly/)
	{
	print ""  #print a blank line for ease of finding new day
	tmp[1]="" #reset variables.  Necessary?
	lineONE[1]=""	  #
	}
	else
	{
	split($1,lineOne,":")   #split the time so that you're just using the hour portion
	
	if (lineOne[1] <= 23)					#Loop through until all 24 hours are tested
		{
		getline line2
		split(line2, tmp, ":")
		if  (tmp[1]== lineOne[1]+1)
			{
			print $0
			print line2
			}
		else
			{
			print $0
			print lineOne[1]+1 ":00\t!!!!inserted by program!!!!"
			}
		}
	}
}

Which works if there is only one missing hour, and there are 2 consecutive hours for the next set of hours. I can't figure out how to reset so that (getline line2) becomes lineOne. Any help appreciated.

Currently the output is:
Code:
paulyester$ awk -f awktest testhrfile.txt

kevin$ awk -f awktest testhrfile.txt

0:00	Clear weather    28
1:00	Clear weather    23
2:00	Clear weather    21
3:00	Clear weather    22
4:00	Clear weather    22
5:00	Clear weather    22
6:00	Clear weather    23
7:00	Clear weather    23
9:00	Clear weather    28
10:00	Clear weather    30
11:00	Clear weather    32
12:00	Clear weather    33
14:00	Clear weather    35
15:00	Clear weather    34
16:00	Clear weather    34
17:00	Clear weather    34
18:00	Clear weather    32
19:00	Clear weather    33
20:00	Clear weather    31
21:00	Clear weather    30
22:00	Clear weather    30
23:00	Clear weather    28

0:00	Clear weather    29
1:00	Clear weather    28
2:00	Clear weather    27
3:00	!!!!inserted by program!!!!    <-- still missing 04000-0500, but even worse, dropped 0600 too
8:00	Clear weather    29
9:00	Clear weather    29
10:00	Clear weather    32
11:00	Clear weather    34
12:00	Clear weather    35
13:00	Clear weather    35
15:00	Clear weather    37
16:00	!!!!inserted by program!!!!  <--dropped 1700
18:00	Clear weather    34
19:00	Clear weather    32
20:00	Clear weather    31
21:00	Clear weather    30
22:00	Clear weather    30
23:00	Clear weather    30

Last edited by pludi; 05-02-2011 at 06:29 PM.. Reason: Added AWK to title
# 2  
Old 05-02-2011
how about this?
Code:
 echo '0:00 Clear weather 29
1:00 Clear weather 28
2:00 Clear weather 27
6:00 Clear weather 25
8:00 Clear weather 29
9:00 Clear weather 29
10:00 Clear weather 32
11:00 Clear weather 34
12:00 Clear weather 35
13:00 Clear weather 35
15:00 Clear weather 37
17:00 Clear weather 36
18:00 Clear weather 34
19:00 Clear weather 32
20:00 Clear weather 31
21:00 Clear weather 30
22:00 Clear weather 30
23:00 Clear weather 30' |awk -F: 'NR>1{if($1-var1==1){var2=$1}else{while (var2<$1-1) print ++var2":00"}}{var1=$1}1'
0:00 Clear weather 29
1:00 Clear weather 28
2:00 Clear weather 27
3:00
4:00
5:00
6:00 Clear weather 25
6:00
7:00
8:00 Clear weather 29
9:00 Clear weather 29
10:00 Clear weather 32
11:00 Clear weather 34
12:00 Clear weather 35
13:00 Clear weather 35
14:00
15:00 Clear weather 37
15:00
16:00
17:00 Clear weather 36
18:00 Clear weather 34
19:00 Clear weather 32
20:00 Clear weather 31
21:00 Clear weather 30
22:00 Clear weather 30
23:00 Clear weather 30

# 3  
Old 05-02-2011
OK, that works brilliantly from the command line - even if I'm not quite sure why at this point.

How would I run that from within a script. The complete file has 8K+ rows that need to be checked, so echo isn't an option.

Sorry for being dense. This is new to me.

the filename is testhrfile.txt

the script I tried to make is

Code:
#! /bin/awk -f-F:

{

NR>1
	{
	if($1-var1==1)
		{
		var2=$1
		}
	else
	{while (var2<$1) 
		print ++var2":00"}
	}
	{var1=$1}1
}

but it spews out:
Code:
kevin$ awk -f awktest3 testhrfile.txt
1:00
2:00
3:00
4:00
5:00
6:00
7:00
8:00
9:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
26:00
27:00
28:00
29:00
30:00
31:00
32:00
33:00
34:00
35:00
36:00

etc until CTRL-C'd

Last edited by Franklin52; 05-03-2011 at 03:01 AM.. Reason: Please use code tags
# 4  
Old 05-02-2011
try this way:

Code:
#! /bin/awk
{

NR>1
{
if($1-var1==1)
{
var2=$1
}
else
{while (var2<$1) 
print ++var2":00"}
}
{var1=$1}
}1

awk -F: -f awk.script urfile
# 5  
Old 05-02-2011
Thanks. It worked once I took the headers at the top of each day out. doh!
# 6  
Old 05-02-2011
Quote:
Originally Posted by paulyester
Thanks. It worked once I took the headers at the top of each day out. doh!

if you want keep the headers in the file just make a little change replacing the old code
Code:
 {var1=$1}

replaced by this code
Code:
/^[0-9]/{var1=$1}

# 7  
Old 05-02-2011
Code:
$ awk -F: -f awktest3 testhrfile.txtawk: syntax error at source line 15 source file awktest3
 context is
	 >>> /^[0-9]/{ <<< 
awk: illegal statement at source line 15 source file awktest3
awk: illegal statement at source line 15 source file awktest3
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Web Development

Getting Rid of Annoying Bootstrap Table Borders and Wayward Table Lines

Bootstrap is great; but we have had some issues with Bootstrapped <tables> (and legacy <fieldset> elements) showing annoying, wayward lines. I solved that problem today with this simple jQuery in the footer: <script> $(function(){ $('tr, td, fieldset,... (0 Replies)
Discussion started by: Neo
0 Replies

2. Shell Programming and Scripting

awk to convert table-by-row to matrix table

Hello, I need some help to reformat this table-by-row to matrix? infile: site1 A:o,p,q,r,s,t site1 C:y,u site1 T:v,w site1 -:x,z site2 A:p,r,t,v,w,z site2 C:u,y site2 G:q,s site2 -:o,x site3 A:o,q,s,t,u,z site3 C:y site3 T:v,w,x site3 -:p,routfile: SITE o p q r s t v u w x y... (7 Replies)
Discussion started by: yifangt
7 Replies

3. UNIX and Linux Applications

Help in copying table structure to another table with constraints in Oracle

hi, i need to copy one table with data into another table, right now am using create table table1 as select * from table2 i want the constraints of table1 to be copied to table2 also , can anyone give me some solution to copy the constraints also, now am using oracle 10.2.0.3.0... (1 Reply)
Discussion started by: senkerth
1 Replies

4. UNIX and Linux Applications

Single table or split the table

I have an application that collect data from 10 server every minutes and stored to mysql db,. there are about 10K record from each server on every minutes. that data will need to stored in 1 month. Which ones is better, create single table for all servers and put field to identified data... (3 Replies)
Discussion started by: before4
3 Replies

5. Shell Programming and Scripting

Build a table from a list by comparing existing table entries

I am new to this shell scripting.... I have a file which contains list of users. This files get updated when new user comes into the system. I want to create script which will give a table containing unique list of users. When I say unique, it means script should match table while parsing... (3 Replies)
Discussion started by: dchavan1901
3 Replies

6. UNIX for Dummies Questions & Answers

Creating a condensed table from a pre-existing table in putty

Hello, I'm working with putty on Windows 7 professional and I'd like to know if there's a way to gather specific lines from a pre-existing table and make a new table with that information. More specifically, I'd like the program to look at a specific column, say column N, and see if any of the... (5 Replies)
Discussion started by: Deedee393
5 Replies

7. UNIX and Linux Applications

create table via stored procedure (passing the table name to it)

hi there, I am trying to create a stored procedure that i can pass the table name to and it will create a table with that name. but for some reason it creates with what i have defined as the variable name . In the case of the example below it creates a table called 'tname' for example ... (6 Replies)
Discussion started by: rethink
6 Replies

8. Shell Programming and Scripting

select values from db1 table and insert into table of DB2

Hi I am having three oracle databases running in three different machine. their ip address is different. from one of the DB am able to access both the databases.(means am able to select values and insert values in to tables individually.) I need to fetch some data from DB1 table(say DB1 ip is... (2 Replies)
Discussion started by: aemunathan
2 Replies

9. Shell Programming and Scripting

Check the record count in table (table in oracle)

I have requirement: 1) Check the record count in table (table in oracle) 2) If records exists generate the file for existing records and wait for some time then Go to sleep mode and Again check the record count after 10 min.......... (Loop this process if record count >0). 3) Generate touch... (1 Reply)
Discussion started by: kamineni
1 Replies

10. Shell Programming and Scripting

Check the record count in table (table in oracle)

I have requirement: 1) Check the record count in table (table in oracle) 2) If records exists generate the file for existing records and wait for some time (Go to sleep mode) and Again check the record count after 10 min.......... (Loop this process if record count >0). 3) Generate touch... (1 Reply)
Discussion started by: kamineni
1 Replies
Login or Register to Ask a Question