Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sar -u generates multiple column headers in csv file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sar -u generates multiple column headers in csv file
# 1  
Old 04-09-2013
Sar -u generates multiple column headers in csv file
Hi All,

The below sar -u command generates multiple column headers in csv file

Expected output should print column headers only once in the csv file

Code:
shell script:

$cat sar_cpu_EBS.sh

#!/bin/bash


while [ 1 -eq 1 ]; do
sar -u 15 1 | awk '/^[0-9]/ {print $1,$2,$4,$6,$7}' | tr -s ' ' ',' >> /sar_cpu_EBS_`date +%m%d%y`.csv
sleep 5
done


Code:
Output the shell script provides:

$cat sar_cpu_EBS.csv

10:23:03,AM,%user,%system,%iowait
10:23:18,AM,0.69,0.60,0.05
10:23:23,AM,%user,%system,%iowait
10:23:38,AM,0.74,0.68,0.10
10:23:43,AM,%user,%system,%iowait
10:23:58,AM,0.68,0.54,0.05
10:24:03,AM,%user,%system,%iowait
10:24:18,AM,0.45,0.56,0.03
10:24:23,AM,%user,%system,%iowait
10:24:38,AM,0.46,0.49,0.06
10:24:43,AM,%user,%system,%iowait
10:24:58,AM,0.52,0.60,0.33
10:25:03,AM,%user,%system,%iowait
10:25:18,AM,0.42,0.50,0.02
10:25:23,AM,%user,%system,%iowait
10:25:38,AM,0.48,0.49,0.03
10:25:43,AM,%user,%system,%iowait
10:25:58,AM,0.41,0.45,0.03
10:26:03,AM,%user,%system,%iowait
10:26:18,AM,0.47,0.58,0.04
10:26:23,AM,%user,%system,%iowait
10:26:38,AM,0.60,0.59,0.08
$

Code:
Expected output should be:

$cat sar_cpu_EBS.csv

10:23:03,AM,%user,%system,%iowait
10:23:18,AM,0.69,0.60,0.05
10:23:38,AM,0.74,0.68,0.10
10:23:58,AM,0.68,0.54,0.05
10:24:18,AM,0.45,0.56,0.03
10:24:38,AM,0.46,0.49,0.06
10:24:58,AM,0.52,0.60,0.33
$

Could anyone please let me know how to achieve this!

Thanks for your time!

Regards,
a1_win
# 2  
Old 04-09-2013
Try this... not tested...
Code:
#!/bin/bash

hdr=1
while true; do
  sar -u 15 1 |  awk '/user/{if(hdr){print $1,$2,$4,$6,$7};next} /^[0-9]/{print $1,$2,$4,$6,$7}' OFS=, hdr=$hdr >> /sar_cpu_EBS_`date +%m%d%y`.csv
  sleep 5
  hdr=0
done

--ahamed
Last edited by ahamed101; 04-09-2013 at 02:42 PM..
# 3  
Old 04-09-2013
Thanks Ahamed! This is working as per the expected output!

Please let me know what is meant by "OFS=," in the shell script you updated as above

Thanks for your time!

Regards,
a1_win
# 4  
Old 04-09-2013
OFS is the Output Field Separator. awk's default field separator is space which is why you were using tr to translate it to comma. So, here we define the OFS to be comma.
Try the statement with different and you will understand.

--ahamed
# 5  
Old 04-09-2013
Sar -u generates multiple column headers in csv file
Hi Ahamed,

Thanks but when I try using the same logic in the shell script to find out the page statistics, the headers are being repeated


Code:
-bash-3.2$ sar -B 5 1
Linux 2.6.32-300.41.1.el5uek   04/09/2013

02:13:08 PM  pgpgin/s pgpgout/s   fault/s  majflt/s
02:13:13 PM      0.00      2.41   1674.10      0.00
Average:         0.00      2.41   1674.10      0.00
-bash-3.2$


Code:
shell script:

 cat sar_page_EBS.sh

#!/bin/bash

hdr=1
while true; do
sar -B 5 1 | awk '/pgpgin\/s/{if(hdr){print $1,$2,$3,$4};next} /^[0-9]/{print $1,$2,$3,$4}'  OFS=, hdr=$hdr >> /sar_page_EBS_`date +%m%d%y`.csv
sleep 1
done


Code:
The output provided by the script:

-bash-3.2$ cat sar_page_EBS.csv

02:05:36,PM,pgpgin/s,pgpgout/s
02:05:41,PM,0.00,0.00
02:05:42,PM,pgpgin/s,pgpgout/s
02:05:47,PM,0.00,2.38
02:05:48,PM,pgpgin/s,pgpgout/s
02:05:53,PM,0.00,6.37
02:05:54,PM,pgpgin/s,pgpgout/s
02:05:59,PM,0.00,0.00
02:06:00,PM,pgpgin/s,pgpgout/s
02:06:05,PM,0.00,3.98
02:06:06,PM,pgpgin/s,pgpgout/s
02:06:11,PM,0.00,11.22
02:06:12,PM,pgpgin/s,pgpgout/s
02:06:17,PM,0.00,3.96
02:06:18,PM,pgpgin/s,pgpgout/s
02:06:23,PM,0.00,43.91
02:06:24,PM,pgpgin/s,pgpgout/s
02:06:29,PM,0.00,9.54
02:06:30,PM,pgpgin/s,pgpgout/s
02:06:35,PM,0.00,0.00
02:06:36,PM,pgpgin/s,pgpgout/s
02:06:41,PM,0.00,13.60
02:06:42,PM,pgpgin/s,pgpgout/s
02:06:47,PM,0.00,0.00


Code:
Expected output what should be(The columns should be printed once):


-bash-3.2$ cat sar_page_EBS.csv

02:05:36,PM,pgpgin/s,pgpgout/s
02:05:41,PM,0.00,0.00
02:05:47,PM,0.00,2.38
02:05:53,PM,0.00,6.37
02:05:59,PM,0.00,0.00
02:06:05,PM,0.00,3.98
02:06:11,PM,0.00,11.22
02:06:17,PM,0.00,3.96
02:06:23,PM,0.00,43.91
02:06:29,PM,0.00,9.54
02:06:35,PM,0.00,0.00
02:06:41,PM,0.00,13.60
02:06:47,PM,0.00,0.00

Please let me know what is missing here?

Thanks for your time!

Regards,
a1_win
# 6  
Old 04-09-2013
You missed hdr=0 inside the while loop.
Check post#2

--ahamed
# 7  
Old 04-09-2013
Yes Ahamed! I figured it out and wanted to update the post!!

Thanks again Ahamed! This worked for me and I get the desrired output!!

Regards,
a1_win
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How do I extract specific column in multiple csv files?

file1: Name,Threshold,Curr Samples,Curr Error%,Curr ART GETHome,100,21601,0.00%,47 GETregistry,100,21592,0.00%,13 GEThomeLayout,100,30466,0.00%,17 file2: Name,Threshold,Curr Samples,Curr Error%,Curr ART GETHome,100,21601,0.00%,33 GETregistry,100,21592,0.00%,22... (6 Replies)
Discussion started by: Raghuram717
6 Replies

2. Shell Programming and Scripting

Row bind multiple csv files having different column headers

All, I guess by this time someone asked this kind of question, but sorry I am unable to find after a deep search. Here is my request I have many files out of which 2 sample files provided below. File-1 (with A,B as column headers) A,B 1,2 File-2 (with C, D as column headers) C,D 4,5 I... (7 Replies)
Discussion started by: ks_reddy
7 Replies

3. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

4. Shell Programming and Scripting

Checking data in csv file after headers

I had a requirement to check if data exists after headers (typically row 2 & so on) in csv file. please help how we can check through shellscript in linux. Thank you !! (1 Reply)
Discussion started by: chandu123
1 Replies

5. Shell Programming and Scripting

Insterting column in csv from multiple files

Hello, I have a spec file that contains a lot of strings that looks like this: PC DELL OptiPlex 3010MT i3 3220/2GB/500GB/DVD-RW/FREE DOS / 5Y NBD Intel i3 3220 (Dual Core, 3.30GHz, 3MB, w/ HD2500 Graphics), 2GB (1x2GB) DDR3 PC3-1600MHz, 500GB HDD SATA III 7200rpm, DVD+/-RW (16x),... (9 Replies)
Discussion started by: g9100
9 Replies

6. UNIX for Dummies Questions & Answers

Using sed command to remove multiple instances of repeating headers in one file?

Hi, I have catenated multiple output files (from a monte carlo run) into one big output file. Each individual file has it's own two line header. So when I catenate, there are multiple two line headers (of the same wording) within the big file. How do I use the sed command to search for the... (1 Reply)
Discussion started by: rebazon
1 Replies

7. Shell Programming and Scripting

Multiple headers in a file

Hi , I have a .txt file in which I have multiple headers, the header record starts with $ symbol...like the first column name is $Account. I have to keep the header in the first line and delete all the remaining headers which are in the file. I tried using sort adc.txt | uniq -u , but my... (7 Replies)
Discussion started by: gaur.deepti
7 Replies

8. Shell Programming and Scripting

Combine Multiple text or csv files column-wise

Hi All I am trying to combine columns from multiple text files into a single file using paste command but the record length being unequal in the different files the data is running over to the closest empty cell on the left. Please see below. What can i do to resolve this ? File 1 File... (15 Replies)
Discussion started by: venky_ibm
15 Replies

9. Shell Programming and Scripting

Merging of files with different headers to make combined headers file

Hi , I have a typical situation. I have 4 files and with different headers (number of headers is varible ). I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only). For example - File 1 H1|H2|H3|H4 11|12|13|14 21|22|23|23... (1 Reply)
Discussion started by: marut_ashu
1 Replies

10. UNIX for Dummies Questions & Answers

Option in sql script to include column headers when spooling file to .csv format

Can anyone help me how to include COLUMN HEADER when spooling file to .CSV format through SQL statement. Thanks, Akbar (4 Replies)
Discussion started by: s1a2m3
4 Replies
Login or Register to Ask a Question