creating delimiter file & append with cron


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting creating delimiter file & append with cron
# 1  
Old 07-21-2011
creating delimiter file & append with cron

I have the following script working fine, and need to generate a file delimiter (with tab or special character) for Excel data import. The script will run every hour in crontab to append the new rows to the delimiter, so that I can collect the data for i.e. a week, which will give me a lot of lines in the file.

Code:
#/bin/sh

tmpFile=/tmp/sartemp.out
sndFile=/tmp/sendfil.out

sysName=`uname -n`

#setup output file
date > $sndFile

# starting sar collection for cpu usage
sar -P ALL -o $tmpFile  1 3

#starting paging space information
echo "\n\n######################" >>$sndFile
echo "\n$sysName Paging Space information\n" >>$sndFile
lsps -s >>$sndFile

#starting memory collection
mmsize=`svmon -G|grep memory|awk '{print $2}'`
mminuse=`vmstat|awk '{print $3}'|tail -1`
#mminuse=`echo $memSvmon | awk '{print $3}'`
#mmfree=`echo $memSvmon | awk '{print $4}'`

#convert memory to MB
let "mmsize_a= ($mmsize)/256"
let "mminuse_a= ($mminuse)/256"
let "mmfree_a= ($mmsize_a -$mminuse_a)"
let "mmfreepercent=$((mmfree_a*100/mmsize_a))"

#
#format output
echo "\n\n######################" >>$sndFile
echo "\n$sysName Total Memory usage" >>$sndFile
echo "\nTotal Memory size: $mmsize_a MB" >>$sndFile
echo "\nTotal Memory inuse: $mminuse_a MB" >>$sndFile
echo "\nTotal Memory Free: $mmfree_a MB" >>$sndFile
echo "\nTotal Memory Free: $mmfreepercent%" >>$sndFile

#CPU usage information
cpusage=`sar -f $tmpFile | grep -p Average | awk '{print $2,$3,$4,$5}'`
cpu_usr=`echo $cpusage | awk '{print $1}'`
cpu_sys=`echo $cpusage | awk '{print $2}'`
cpu_wio=`echo $cpusage | awk '{print $3}'`
cpu_idle=`echo $cpusage | awk '{print $4}'`
#
#format output
echo "\n\n#########################" >>$sndFile
echo "\n$sysName CPU usage" >>$sndFile
echo "\nAverage CPU usage for User process: $cpu_usr%" >>$sndFile
echo "\nAverage CPU usage for System process: $cpu_sys%" >>$sndFile
echo "\nAverage CPU usage for I/O process: $cpu_wio%" >>$sndFile
echo "\nAverage CPU usage being idle: $cpu_idle%" >>$sndFile
echo "\n##################################" >>$sndFile

sysdate=`date +"%m/%d/%Y"`
systime=`date +%r`
cpucnt=`vmstat |grep lcpu |awk '{print $3}'`
pstotal=`lsps -s | awk '{print $1}'|tail -1`
psusedpercent=`lsps -s | awk '{print $2}'|tail -1`

This scripts gives me info like:
Code:
Total Paging Space   Percent Used
      71680MB               1%

Prod1 Total Memory usage

Total Memory size: 75776 MB
Total Memory inuse: 66925 MB
Total Memory Free: 8851 MB
Total Memory Free: 11%

Average CPU usage for User process: 56%
Average CPU usage for System process: 6%
Average CPU usage for I/O process: 9%
Average CPU usage being idle: 28%

The format I like to is:
Code:
Node           Date             Time                    Total_Paging_Space (MB)   Percent Used(%)   etc (in horizontal)
$sysName   $sysdate       $systime
Prod1          07/21/2011  01:00:00 AM  71680                              1                   so on so forth for Memory & CPU
Prod1          07/21/2011  02:00:00 AM  71680                              3                  so on so forth for Memory & CPU
Prod1          07/21/2011  03:00:00 AM  71680                              2                   so on so forth for Memory & CPU
Prod1          07/21/2011  04:00:00 AM  71680                              4                   so on so forth for Memory & CPU
Etc
Etc

Before appending the rows to the file, I need to do some tweaks.

The paging space data is from lsps –s, so I need to awk:
Code:
pstotal=`lsps -s | awk '{print $1}'|tail -1`
psusedpercent=`lsps -s | awk '{print $2}'|tail -1`

They return with MB & %, so I need to remove MB & % for the smooth import to Excel.
Code:
root@:/>>lsps -s | awk '{print $1}'|tail -1    
71680 MB
root@:/>>lsps -s | awk '{print $2}'|tail -1
2%

Also, CPU count is from:
Code:
cpucnt=`vmstat |grep lcpu |awk '{print $3}'`
root@:/>>vmstat |grep lcpu |awk '{print $3}'
lcpu=8

so, I need to remove “lcpu=” as well.

Please advise.

Thank you so much for your help.

Last edited by radoulov; 07-21-2011 at 04:38 PM.. Reason: Tags fixed.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

2. Shell Programming and Scripting

Help with ksh-to read ip file & append lines to another file based on pattern match

Hi, I need help with this- input.txt : L B white X Y white A B brown M Y black Read this input file and if 3rd column is "white", then add specific lines to another file insert.txt. If 3rd column is brown, add different set of lines to insert.txt, and so on. For example, the given... (6 Replies)
Discussion started by: prashob123
6 Replies

3. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

4. Shell Programming and Scripting

Creating delimiter file

#/bin/sh sysdate=`date +"%m/%d/%Y"` systime=`date +%r` ps_per=`lsps -s | nawk '{print $2+0}'|tail -1` ps_tot=`lsps -s | nawk '{print $1+0}'|tail -1` lcpu=`vmstat | nawk -F= '/lcpu/ {print $2+0}'` mem_tot=`vmstat | nawk -F= '/mem=/ {print $3+0}'` avm=`vmstat|awk '{print... (7 Replies)
Discussion started by: Daniel Gate
7 Replies

5. Shell Programming and Scripting

Split into columns based on the parameter and use & as delimiter

Here is my source, i have million lines like this on a file. disp0201.php?poc=4060&roc=1&ps=R&ooc=13&mjv=6&mov=5&rel=5&bod=155&oxi=2&omj=5&ozn=1&dav=20&cd=&daz=& drc=&mo=&sid=&lang=EN&loc=JPN I want to split this into columns in order to load in database, anything starts with"&mjv=6" as first... (13 Replies)
Discussion started by: elamurugu
13 Replies

6. Shell Programming and Scripting

Creating Header & Trailer for bulk volume data file

Hi all, I have a requirement to create a Header &Trailer for a file which is having 20 millions of records. If I use the following method, i think it will take more time. cat "Header"> file1.txt cat Data_File>>file1.txt cat "Trailer">>file1.txt since second CAT command has to read all... (4 Replies)
Discussion started by: Raamc
4 Replies

7. Shell Programming and Scripting

creating & sending formatted (with bolds & colors) CSV

Hi , I have a situation. Need is to create & send a formatted file with header in BOLD & colored & some sequel results as a content. I know echo -e \033 command, but its scope is limited in PUTTY. How to retain the formatting out of Putty; say after someone opens a email attachment... (2 Replies)
Discussion started by: infaWorld
2 Replies

8. Shell Programming and Scripting

append data in a file by using tab delimiter

Hi, I need to append the data in to a file by using tab delimiter. eg: echo "Data1" >> filename.txt echo "\t" >> filename.txt (its not working) echo "Data2" >> filename.txt. the result sould be like this. Data1 Data2 (6 Replies)
Discussion started by: Sharmila_P
6 Replies

9. Shell Programming and Scripting

creating a delimiter string

hi i have a function printValues() { var=$# count=0 qName="" while do if then echo QManager Name $1 fi if then echo Cluster Name$2 fi if (( $count != 0 && $count != 1 )) then ... (0 Replies)
Discussion started by: satish@123
0 Replies

10. Shell Programming and Scripting

Append date to a file from CRON

Hi, I am trying to append date in DAY_Mon_dd_yyyy at the end of a filename from cron. Cron entry looks as below. (script to execute) > test_file_`date +"a_%b_%d_%Y"` File name created after executing the job is test_file_ I am expecting the filename to be something like ... (8 Replies)
Discussion started by: dncs
8 Replies
Login or Register to Ask a Question