Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parsing file: preparing data for charts (gnuplot, calc,...)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing file: preparing data for charts (gnuplot, calc,...)
# 1  
Old 11-22-2011
Parsing file: preparing data for charts (gnuplot, calc,...)
Hi there,

I have files (tsv) like this:
Code:
CTLPort	IO Rate(IOPS)	Read Hit(%)	Write Hit(%)	Timestamp
0A	136	97	100	 09:36:48
0B	3	100	100	 09:36:48
0C	88	35	100	 09:36:48
0A	87	100	100	 09:37:49
0B	3	97	100	 09:37:49
0C	83	45	100	 09:37:49
0A	108	83	100	 09:38:48
0B	3	100	100	 09:38:48
0C	190	49	100	 09:38:48

I can transform it to the following view:
Code:
CTLPort	IO Rate(IOPS)	Read Hit(%)	Write Hit(%)	Timestamp
0A	136	97	100	 09:36:48
0A	87	100	100	 09:37:49
0A	108	83	100	 09:38:48
0B	3	100	100	 09:36:48
0B	3	97	100	 09:37:49
0B	3	100	100	 09:38:48
0C	88	35	100	 09:36:48
0C	83	45	100	 09:37:49
0C	190	49	100	 09:38:48

with next commands:
Code:
head -1 sample2 > sample4
awk 'BEGIN {FS="\t"; OFS="\t"} NR==1 {next} 1' sample2 |sort -k 1,1 -k 5 >> sample4

but I need for all fields except $1 & $NF something like this:
Code:
Timestamp	0A	0B	0C	0D
09:36	97	100	35	44
09:37	100	97	45	39
09:38	83	100	49	45

Any ideas?

PS number of ""timestamps" is known (fixed) value.
PPS preferred method is awk
# 2  
Old 11-25-2011
bump Smilie

I found out how to create array in which indices are uniqe combination of two fields and value is intersecting of these field, f.e.:

array[$NF, $1] = $2

But I stuck how to print it out in the END part of awk script.

Any guesses?

---------- Post updated at 12:18 PM ---------- Previous update was at 11:13 AM ----------

Okay. I have done next step:
Code:
NR==1 {next}
{
a[$1]++
b[$NF]++
c[$NF,$1]=$2
}
END {
        for (i in b) {
        printf "%-8s", i
                for (j in a) {
                printf "\t%3d", c[i,j]
                }
        printf  "\n"
        }
}

but I stuck with header.... it should be:
Timestamp 0A 0B 0C ...
(correlating to the fields)
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX and Linux Applications

Gnuplot Same Data Two Axes Different Units)

I'm pretty familiar with Gnuplot, but I've never actually come across a reasonable solution to this problem and I'm hoping someone can help me out! I think it's because I don't know how to pose the problem neatly, so please bare with me. Also new to the forum....so yeah... I have one data set (x... (3 Replies)
Discussion started by: TC69
3 Replies

2. Shell Programming and Scripting

Gnuplot Time Data Question

I have a data file of the following format: servername,2013-05-11 17:46:03,SomeText,195,195,11,202 servername,2013-05-11 17:47:03,SomeText,192,192,23,103 servername,2013-05-11 17:48:03,SomeText,189,190,14,117 servername,2013-05-11 17:49:03,SomeText,196,195,24,231 ... ... I want to... (0 Replies)
Discussion started by: BeeryM
0 Replies

3. Shell Programming and Scripting

Parsing file data

Hey Guys, I'm a novice at shell scripts and i need some help parsing file data. Basically, I want to write a script that retrieves URLs. Here is what I have so far. #!/bin/bash echo "Please enter start date (format: yyyy-mm-dd):\c" read STARTDATE echo "Please enter end date... (7 Replies)
Discussion started by: silverdust
7 Replies

4. Shell Programming and Scripting

Parsing data using keys from one file

I have 2 text files where I need to parse data from file 2 using the data from file 1. Below are my sample files File 1 (tab delimited) 257 350 670 845 725 1025 767 820 ... .... .... file 2 (tab delimited) 220..450 TA AB650 ABCED 520..850 GA AB720 ABCDE 700..1100 TC AB820 ABCDE... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

5. Shell Programming and Scripting

Parsing txt, xml files and preparing csv file

Hi, I need to parse text, xml files to get the statistic numbers and prepare summary csv file. What is the best way to parse these file and prepare csv file. Any idea you have , please? Regards, (2 Replies)
Discussion started by: LinuxLearner
2 Replies

6. Shell Programming and Scripting

placeholder in gnuplot data

My input data has occasional holes in it, spots where a sensor couldn't be read. These are ERR in the data file instead of a floating point number. What should I change them to, for gnuplot to ignore these values instead of whining about them? (1 Reply)
Discussion started by: Corona688
1 Replies

7. Shell Programming and Scripting

parsing data and incorporating it into another file

Hi, I have a folder that contains many (multiple) files 1.fasta 2.fasta 3.fasta 4.fasta 5.fasta . . 100's of files Each such file have data in the following format for example: vi 1.fasta >AB_1 gi|15835212|ref|NP_296971.1| preprotein translocase subunit SecE... (3 Replies)
Discussion started by: Lucky Ali
3 Replies

8. Shell Programming and Scripting

parsing data and incorporating it into another file

Hi All I have two files: file 1 >AB_1 MLKKPIIIGVTGGSGGGKTSVSRAILDSFPNARIAMIQHDSYYKDQSHMSFEERVKTNYDHPLAFDTDFM IQQLKELLAGRPVDIPIYDYKKHTRSNTTFRQDPQDVIIVEGILVLEDERLRDLMDIKLFVDTDDDIRII RRIKRDMMERGRSLESIIDQYTSVVKPMYHQFIEPSKRYADIVIPEGVSNVVAIDVINSKIASILGEV >AB_2... (5 Replies)
Discussion started by: Lucky Ali
5 Replies

9. Shell Programming and Scripting

Parsing the data in a file

Hi, I have file (FILE.tmp) having contents, FILE.tmp ======== filename=menudata records=0000000000037 ldbname=pinsys timestamp=2005/05/14-18:32:33 I want to parse it bring a new file which will look like, filename records ldbname timestamp... (2 Replies)
Discussion started by: Omkumar
2 Replies
Login or Register to Ask a Question