Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract portion of data

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract portion of data
# 1  
Old 06-02-2012
Extract portion of data
Hi Gurus,

I need some help in extracting some of these information and massage it into the desired output as shown below.

I need to extract the last row with the header in below sample which is usually the most recent date, for example:
2012-06-01 142356 mb 519 -219406 mb 1 -77049 mb

Notice in the sampe output below, there a new column called: "Label" being added.

FYI, this is just a sample data, the actual data will might contains more data and the listing will be longer.

Sample data:

Code:
Date          New Data #AB       Removed #CD    Net Change
----------  ---------- -----  ---------- -----  ----------
2012-05-27   100046 mb 580    -236329 mb 1      -136282 mb
2012-05-28   117905 mb 628    -223216 mb 1      -105310 mb
2012-05-29   153561 mb 706    -214508 mb 1       -60946 mb
2012-05-30   216141 mb 629    -222977 mb 1        -6835 mb
2012-05-31   180524 mb 468    -221210 mb 1       -40685 mb
2012-06-01   142356 mb 519    -219406 mb 1       -77049 mb
----------  ---------- -----  ---------- -----  ----------
Average      157957 mb        -158746 mb           -789 mb

Top 10 Servers Detected:
------------------------
Total for all servers                  4106890 mb      100.0%
  server1.abc.com	               1298172 mb      31.6%
  server2.abc.com                      708845 mb       17.3%

Sample output:
Code:
Label	     Date        New Data   #AB    Removed    #CD    Net Change
-------	     ----------  ---------- -----  ---------- -----  ----------
Statistic    2012-06-01  142356 mb  519    -219406 mb 1       -77049 mb


Is it possible to generate this kind of output?

Would appreciate for any of you help and advise.

Thank you.

- Jack
# 2  
Old 06-02-2012
Try:
Code:
awk 'NR==1{print "Label\t\t" $0}NR==2{print "-------\t\t" $0} NR>2 && /^--/{print "Statistic\t"p; exit}{p=$0}' infile

This User Gave Thanks to Scrutinizer For This Post:
superHonda123
# 3  
Old 06-03-2012
Hi Scrutinizer,

Thanks for your response and help.

I've tested and it's working as expected. Great Smilie

However, I've just realised that there are a couple of data that I need to compile in my script and append the similar data from other sources.

Is it possible to extract the last row ONLY without the header and dash line in below sample? The last row usually indicates the most recent date.

For example:
Code:
2012-06-01 142356 mb 519 -219406 mb 1 -77049 mb

Greatly appreciate for your advise.

Thank you.


- Jack
Last edited by superHonda123; 06-03-2012 at 08:51 AM.. Reason: code tags
# 4  
Old 06-03-2012
Given your sample input, this should work:

Code:
awk ' /^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]/ { k = $0 } END { print k }' t49.data



It will have problems if other lines (after the section you indicated) have a date in the first field.
# 5  
Old 06-03-2012
Or stripping the script in #2 would give:
Code:
awk 'NR>2 && /^--/{print p; exit}{p=$0}' infile

This User Gave Thanks to Scrutinizer For This Post:
superHonda123
# 6  
Old 06-03-2012
Hi Scrutinizer,

Thanks for your response.

I've tried the second awk and it's working OK.

However, I forgot to mention that there is a need to insert the word: "Statistic".

So, the correct output should be:
Code:
Statistic    2012-06-01  142356 mb  519    -219406 mb 1       -77049 mb

The spacing and alignment of this output should be aligned with the first output which includes the header and dash so that it's easier to append its output to the next line.

Could you advise on how to include this word in the second awk?

Thanks a lot.


- Jack
# 7  
Old 06-03-2012
Hi Jack,

I think that if you compare #5 to #2 , you should be able to make this change yourself, no?

S.
This User Gave Thanks to Scrutinizer For This Post:
superHonda123
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Extract a portion of string from each line in Linux

Hi I have to extract the destination path information from each record the file is of variable length so I will not be able to use the print command.The search should start on variable "destinationPath" and it should end at immediate "," also the first field has to be printed Input File:... (7 Replies)
Discussion started by: rkakitapalli
7 Replies

2. Shell Programming and Scripting

How to extract portion of a string?

Hi Gurus, Would like to seek some help on how to extract a portion of string from log's output as shown below. Sample of raw data: piece handle=/test123/disk_dump/test123/df0_cntrl_PCPFCI20120404_68498 tag=TAG20120404T180035 comment=NONE piece... (13 Replies)
Discussion started by: superHonda123
13 Replies

3. Shell Programming and Scripting

Extract portion of log info based on specific word

Hi Gurus, I'm using HP-UX B.11.23 operating system. I've been trying to extract a specific wording for example: "A tool used by tp produced warnings" from my below log data, but could not find a way to solve it. My intention is, if the log contain the word: "A tool used by tp produced... (9 Replies)
Discussion started by: superHonda123
9 Replies

4. Shell Programming and Scripting

Removing a portion of data in a file

Hi, I have a folder that contains many (multiple) files 1.fasta 2.fasta 3.fasta 4.fasta 5.fasta . . 100's of files Each such file have data in the following format for example: vi 1.fasta Code: >AB_1 MLKKPIIIGVTGGSGGGKTSVSRAILDSFPNARIAMIQHDSYYKDQSHMSFEERVKTNYDHPLAFDTDFM (6 Replies)
Discussion started by: Lucky Ali
6 Replies

5. Shell Programming and Scripting

How to extract a text portion from a file

Can some one help me with shell script to extract a text block between two known strings. The given input file is as below: Name: abs Some tesxt.... Some tesxt.... Some tesxt.... end of text Name: xyz Some tesxt.... Some tesxt.... Some tesxt.... end of text Name: efg Some... (5 Replies)
Discussion started by: ejazs0
5 Replies

6. Shell Programming and Scripting

extract string portion from filename using sed

Hi All, I posted something similar before but I now have a another problem. I have filenames as below TOP_TABIN240_20090323.200903231830 TOP_TABIN235_1_20090323.200903231830 i need to extract the dates as in bold. Using bash v 3.xx Im trying to using the print sed command but... (11 Replies)
Discussion started by: santam
11 Replies

7. Shell Programming and Scripting

extract string portion using sed

Hi All I have 3 files as listed below and highlighted in bold the portions of the filenames I need to extract: TOS_TABIN218_20090323.200903231830 TOS_TABIN219_1_20090323.200903231830 TOS_TABIN219_2_20090323.200903231830 I tried source_tabin_name=`echo $fname | sed 's/_.*//'` but I... (6 Replies)
Discussion started by: santam
6 Replies

8. UNIX for Dummies Questions & Answers

How to extract a portion of text from a log file

I am using Unix on Mac OS X 10.5.6. I am trying to extract the last entry of a log (text) file. As seen below, each log entry looks like the following (date and time change with each log entry): I want the script to extract everything quoted above, including the "===" dividers. ... (2 Replies)
Discussion started by: atilano
2 Replies

9. Shell Programming and Scripting

extract date portion from file

Hi, I have a file where there is a date field (single line variable length file) how to extract just the date portion from it the position of date field may vary anywhere in the line but will always have the format mm-dd-yyyy for eg . xxxxxxxxxxxxxxx09-10-2006xxxxxxxxxxxxxxxxxxxx (5 Replies)
Discussion started by: misenkiser
5 Replies

10. Shell Programming and Scripting

extract a portion of log file

hello, I want to grep the log file according to time and get the portion of log from one particular time to other. I can grep for individual lines by time but how should I print lines continuously from given start time till end till given end time. Appreciate your ideas, Thanks chandra (8 Replies)
Discussion started by: chandra004
8 Replies
Login or Register to Ask a Question