Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Report Generation with Grep

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Report Generation with Grep
# 1  
Old 04-01-2010
Report Generation with Grep
All,

I am pretty new to Unix Environment. I am not sure if my requirement can be accomplished in Unix. I did try searching this forum and others but could not get an answer. Requirement is explained below:

I have a set of files in a folder.
Code:
file1_unload
file2_unload
file3_unload
file1_load
file2_load
file3_load
file1_dat
file2_dat
file3_dat

And each file will have similar type of input. I will just give a files output for example:
file1_unload
Code:
Started at 22:24:32
593456 records unloaded
Ended at 23:08:40

file1_load
Code:
Started at 23:09:01
Loaded records 345298
Ended at 00:12:25

You can similarly imagine values for other unload and load files. And dat files will have the data in ASCII format. Now I want a report be created which looks like the following:

Code:
file1 <file1_dat's size> 593456 22:24:32 23:08:40 00:44:08 345298 23:09:01 00:12:25 01:03:24
file2 <file2_dat's size> <Unld> <UnldSt> <UnldEd> <timeDF> <Load> <LoadSt> <LoadEd> <timeDF>
file3 <file3_dat's size> <Unld> <UnldSt> <UnldEd> <timeDF> <Load> <LoadSt> <LoadEd> <timeDF>



For better understanding, the fields in the report are:
1. Part of the filename (file1, file2 etc.), NOT the entire filename.
2. Respective dat file's size.
3. Unloaded records from the respective Unload file.
4. Start Time from the respective Unload file.
5. End Time from the respective Unload file.
6. Time Difference between 4 and 5th columns.
7. Loaded records from the respective Load file.
8. Start Time from the respective Load file.
9. End Time from the respective Load file.
10. Time Difference between 8 and 9th columns.

Let me know if this can be done in UNIX. If so, please help me do it. Let me know for any questions on the inputs.

Thanks in advance,
Bharath
# 2  
Old 04-01-2010
Yes you can do it, with the lot of utilities available.

But before that, what have you tried so far ?
# 3  
Old 04-01-2010
Hi thegeek,

thanks for replying quickly. Here is what I am doing now. I have tried to do this, 1 column at a time, with Grep. Following are the Greps I used.

For columns 1 and 2:
ls -lrt

For Unload records: (Col 3)
grep -h " records " *_unload | awk '{print $1}'

For Unload Start and End Times: (Col 4, 5)
grep -h Started `ls -rt -1 *_unload`
grep -h Ended `ls -rt -1 *_unload`

No idea of how to get the Time Difference. (Col 6)

For Load records: (Col 7)
grep -h " records " *_load | awk '{print $3}'

For Load Start and End Times: (Col 8, 9)
grep -h Started `ls -rt -1 *_load`
grep -h Ended `ls -rt -1 *_load`

Again, No idea of how to get the Time Difference. (Col 10)

As I said in my previous post I am new to Unix and not sure of how to use the utilities, that you say are there! Pardon my ignorance.

Let me know if you need any other inputs.

Thanks
Bharath
# 4  
Old 04-02-2010
a solution using a scripting language would be a better one.. do you know any one ? then try that..
# 5  
Old 04-02-2010
The only idea I have is to write a loop getting the filenames and using the above grep statements for each file and echo the results.

Is there any better and effecient way of doing this?

Thanks
# 6  
Old 04-02-2010
If the xxx_load and xxx_unload files have always the same structure, i think it would be better to read them sequentially.
I believe the best way is to write a script with a for loop like
Code:
for FILE in $(ls file*_dat)
do
  {
    read X X UnldSt
    read Unld X
    read X X UnldEd
  } < ${FILE%_dat}_unload
  {
    read X X LoadSt
    read X X Load
    read X X LoadEd
  } < ${FILE%_dat}_load
  # Calculate the time differences given in seconds
  UnldDT=$(( $(date -d UnldEd +%s) - $(date -d UnldSt +%s) ))   # Works if you have GNU date
  ((UnldDT<0)) && ((UnldDT+=86400))
  LoadDT=$(( $(date -d LoadEd +%s) - $(date -d LoadSt +%s) ))
  ((LoadDT<0)) && ((LoadDT+=86400))
  echo "$(ls -lrt $FILE) $Unld $UnldSt $UnldEd $UnldDT $Load $LoadSt $LoadEd $LoadDT"
done

Last edited by frans; 04-02-2010 at 05:45 PM..
# 7  
Old 04-02-2010
Thanks Frans for the quick response.

I think I dont have GNU date. I am working on KSH on an IBM AIX OS. Below is the error that I got.

Code:
date: Not a recognized flag: d
Usage: date [-u] [+"Field Descriptors"]
date: Not a recognized flag: d
Usage: date [-u] [+"Field Descriptors"]

Let me know if I am missing something.

Thanks,
Bharath

---------- Post updated at 09:20 PM ---------- Previous update was at 05:34 PM ----------

Ok finally found a way!! Wrote a simple awk function to do that (only time difference within a day) Atleast thats what I needed:

Function:
Code:
TimeDiff ()
{
	echo `echo "$1" | awk 'BEGIN {
   	split("3600 60 1", sec_calc)
   	FS=":|[ ][ ]*"
	}	
	{
	  time1 = 0
	  for (i=1; i < 4; i++)
	    time1 += $i * sec_calc[i]
	  
	  time2 = 0
	    for (i=4; i < 7; i++)
	    time2 += $i * sec_calc[i-3]
	
	  time_diff = time2 - time1
	  hour_diff = time_diff / 3600 
	  calc_diff = time_diff % 3600 
	  min_diff = calc_diff / 60
	  sec_diff = calc_diff % 60
	  
	  printf("%.2d:%.2d:%.2d\n",int(hour_diff),int(min_diff),sec_diff) 
	  
	}'`
}

And called the function in my script as:

Code:
UnldDT=`TimeDiff "$UnldSt $UnldEd"`
LoadDT=`TimeDiff "$LoadSt $LoadEd"`

Thanks all for your time.
Bharath
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Report generation using script

Hi all I have a unix script that generates a report with the following information: uptime, mounted file systems, disk usage (> 90% --> critical, <75%-90%> --> warning, < 75% healthy), Mem usage, CPU usage and load average. But I would like to create one single report containing all this... (5 Replies)
Discussion started by: fretagi
5 Replies

2. Shell Programming and Scripting

Using awk and grep for sql generation

Hi, I have a file pk.txt which has table related data in following format TableName | PK Employee | id Contact|name,phone,country I have another file desc.txt which lists datatype of each field like this: Table|Field|Type Employee|id|int Contact|name|string Contact|country|string... (7 Replies)
Discussion started by: wahi80
7 Replies

3. Shell Programming and Scripting

Report generation using perl script

Hi, I have a perl script to read the log file and create a report from it. I have the script file and log file in a different directories. Now i have pipe the log file data to the perl script to create the report (HMTL file). I am using the below command this isn't working tail -f... (4 Replies)
Discussion started by: vel4ever
4 Replies

4. Shell Programming and Scripting

Report generation based on certain conditions

Hi I recently joined a project where I have been asked to generate a report using shell script accessing UNIX box. I have no idea on how to do it as I am a beginner and learning shell scripts. Suppose I have a XML: Code: ... (3 Replies)
Discussion started by: vat1kor
3 Replies

5. Shell Programming and Scripting

File Report Generation

hi all i need to generate a report file that contains the following details of files present in a directory. 1. File name 2.Complete path for each files and directory 3.File size 4.Days older example i have a directory testing that contains sub-directories and some files. i need to make a... (5 Replies)
Discussion started by: yashwantkumar
5 Replies

6. Shell Programming and Scripting

Disk report generation problem

Hello everyone, I have a list of inputs as below. My logic is to get the particular powerdisk which matches for ASM disk which means take the major & minor number of each asm disk and matches with powerdisk info then get the particular powerdisk $ ls -l /dev/asm_* ---> ASM disk info... (11 Replies)
Discussion started by: kannan84
11 Replies

7. Shell Programming and Scripting

Report generation

Hello, I got a requirement in writing a KSH script in unix, please help me out the requirement is there are two folders Folder1 and Folder2 and there are same files in the different folders. like file1,file2 in folder1 and file1 and file2 in folder2. I would like to compare all the similar... (3 Replies)
Discussion started by: gmahesh2k
3 Replies

8. UNIX for Dummies Questions & Answers

report generation

Hello, I got a requirement in writing a sheel script in unix, please help me out the requirement is there are two folders Folder1 and Folder2 and there are same files in the different folders. like file1,file2 in folder1 and file1 and file2 in folder2. I would like to compare all the... (2 Replies)
Discussion started by: gmahesh2k
2 Replies

9. Shell Programming and Scripting

awk- report generation from input file

I have input file with below content: Person: Name: Firstname1 lastname1 Address: 111, Straat City : Hilversum Person: Name : Fistname2 lastname2 Address: 222, street Cit: Bussum Person: Name : Firstname2 lastname3 Address: 333, station straat City: Amsterdam I need... (6 Replies)
Discussion started by: McLan
6 Replies

10. Shell Programming and Scripting

Oracle Report generation

Hi, I am beginner in shell programming.In a shell script i found a call to a script 'runrep25m',which i think is to generate oracle reports?Could anyone help me by providing some details about its usage With Thanks & Regards Dileep (7 Replies)
Discussion started by: DILEEP410
7 Replies
Login or Register to Ask a Question