Sponsored Content
Top Forums Shell Programming and Scripting Extraction of data from multiple text files, and creation of a chart Post 302330729 by ackheron on Thursday 2nd of July 2009 07:00:02 AM
Old 07-02-2009
Lightbulb Extraction of data from multiple text files, and creation of a chart

Hello dear friends,

My problem as explained below seems really basic.
Fact is that I'm totally new to programming, and have only a week to produce a script ( CShell or Perl ? ) to perform this action.

While searching on the forums, I found a command that could help me, but I don't know how to use it properly :s.

But first of all . . . the context.
I have 7300 text files, each containing the same type of information.
Below is shown how the text files are written :


3 PARAMETERS ON 05JAN01.
1 AZU1 STA X -0.247297943318988E+07 +- 0.241394876950605E-02
2 AZU1 STA Y -0.467133800760033E+07 +- 0.374164031593362E-02
3 AZU1 STA Z 0.355810775092403E+07 +- 0.257172727253398E-02
2 1 0.571958818322482E+00
3 1 -0.659378461245129E+00
3 2 -0.847505892381651E+00
AZU1 ANTENNA LC 0.1636 0.0000 0.0000 !up north east (m)


In order to analyze all the data, I need to extract the bold values, and put each one in a different column of an Excel file (or equivalent) or in a text file ( but then with the three values on a line, and maybe try to create a csv file ).

To finish, I have a file like this in a different folder . . . . and there's 3 or 4 levels of difference in the arborescence. As :

desktop/data_2005/05JAN01/out/solution/05JAN01AZU1_fix.txt
desktop/data_2005/05JAN02/out/solution/05JAN02AZU1_fix.txt
and it goes on for the whole year . . .

The idea would be to use the 'grep' command I guess :
I tried with this "egrep "1|2|3" < 05JAN01AZU1_fix.stacov.txt | awk '{ print $5 }'>> output.txt" but it doesn't work properly. It did output cleanly the 3 values that I want when I changed the "1|2|3" to "10|20|30" and did the changed the values 1,2,3 to 10,20,30before the three values I wanted to extract.

So, basically . . . I have a text file for each day of the year, containing three values that I want to put in three separate columns of an excel file, or in the same line of the csv file.

I guess that I have to find the way to extract correctly my values, but also that I have to write a kind of loop, to go through all the folders(days) of the year.

CShell or Perl ?

Does anyone has one or more ideas to help me go further ?
If you do, I'll be pleased if you drop it there Smilie
Thanks . . .

P.S :I don't know who wrote the rules of the forum . . . the point n°15 "No smoking in the forums" made me smile . . . Smilie
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

extraction of data from a text file which follows certain pattern

hi everybody, i have a file, in it I need to extract some data that follows a particular pattern.. For example: my file contains like now running Speak225 sep 22 mon 16:34:05 2008 -------------------------------- ... (4 Replies)
Discussion started by: mohkris
4 Replies

2. Shell Programming and Scripting

Selective extraction of data from a files

Hi, I would like to seek for methods to do selective extraction of line froma file. The scenario as follows: I have a file with content: message a received on 11:10:00 file size: 10 bytes send by abc message b received on 11:20:00 file size: 10 bytes send by abc (3 Replies)
Discussion started by: dwgi32
3 Replies

3. UNIX for Dummies Questions & Answers

Data storing and chart

I apologize if this has been posted before but I have been doing a lot of searching and I have not found any real answers, this is quite possibly due to that I may not be searching for the right terms or something. Little bit of info about my situation is that I'm a newbie as you probably guessed... (1 Reply)
Discussion started by: ummSomeNameHere
1 Replies

4. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

5. UNIX for Dummies Questions & Answers

Need help for data extraction if files

Hello all, I want to extract some particular data from a files and than add all the values . but i m not able to cut the particular word(USU-INOCT and USU-OUTOCT) as it is coming not in column. and than able to add values coming in it . can anyone help me Please cat <file name> ... (7 Replies)
Discussion started by: anamdev
7 Replies

6. Shell Programming and Scripting

Column extraction from multiple files to multiple files

I have roughly ~30 .txt files in a directory which all have unique names. These files all contain text arranged in columns separated by whitespace (example file: [#YY MM DD hh mm WDIR WSPD GST WVHT DPD APD MWD PRES ATMP WTMP DEWP VIS TIDE #yr mo dy hr mn degT m/s m/s m sec ... (5 Replies)
Discussion started by: aozgaa
5 Replies

7. Shell Programming and Scripting

Read multiple text files and copy data to csv

hi i need to extract lines from multiple files to a csv file. for example, i have these 3 files file1.txt date:29dec1980 caller:91245824255 called:8127766 file2.txt date:11apr2014 caller:9155584558 called:8115478 file3.txt date:25jun2015 caller:445225552 called:8117485 (30 Replies)
Discussion started by: lp.descamps
30 Replies

8. UNIX for Beginners Questions & Answers

Extraction of data between parenthesis using multiple pattern

i want to extract all data with in parenthesis from a file by passing a pattern from another file.i have some sql statements in my file and i want to extract those ddl by refering to a pattern which is in another file and before writting into file i need some transformation to do.Basically i want... (1 Reply)
Discussion started by: raj121
1 Replies

9. Shell Programming and Scripting

In PErl script: need to read the data one file and generate multiple files based on the data

We have the data looks like below in a log file. I want to generat files based on the string between two hash(#) symbol like below Source: #ext1#test1.tale2 drop #ext1#test11.tale21 drop #ext1#test123.tale21 drop #ext2#test1.tale21 drop #ext2#test12.tale21 drop #ext3#test11.tale21 drop... (5 Replies)
Discussion started by: Sanjeev G
5 Replies
Spreadsheet::WriteExcel::Chart::Bar(3pm)		User Contributed Perl Documentation		  Spreadsheet::WriteExcel::Chart::Bar(3pm)

NAME
Bar - A writer class for Excel Bar charts. SYNOPSIS
To create a simple Excel file with a Bar chart using Spreadsheet::WriteExcel: #!/usr/bin/perl -w use strict; use Spreadsheet::WriteExcel; my $workbook = Spreadsheet::WriteExcel->new( 'chart.xls' ); my $worksheet = $workbook->add_worksheet(); my $chart = $workbook->add_chart( type => 'bar' ); # Configure the chart. $chart->add_series( categories => '=Sheet1!$A$2:$A$7', values => '=Sheet1!$B$2:$B$7', ); # Add the worksheet data the chart refers to. my $data = [ [ 'Category', 2, 3, 4, 5, 6, 7 ], [ 'Value', 1, 4, 5, 2, 1, 5 ], ]; $worksheet->write( 'A1', $data ); __END__ DESCRIPTION
This module implements Bar charts for Spreadsheet::WriteExcel. The chart object is created via the Workbook "add_chart()" method: my $chart = $workbook->add_chart( type => 'bar' ); Once the object is created it can be configured via the following methods that are common to all chart classes: $chart->add_series(); $chart->set_x_axis(); $chart->set_y_axis(); $chart->set_title(); These methods are explained in detail in Spreadsheet::WriteExcel::Chart. Class specific methods or settings, if any, are explained below. Bar Chart Methods There aren't currently any bar chart specific methods. See the TODO section of Spreadsheet::WriteExcel::Chart. EXAMPLE
Here is a complete example that demonstrates most of the available features when creating a chart. #!/usr/bin/perl -w use strict; use Spreadsheet::WriteExcel; my $workbook = Spreadsheet::WriteExcel->new( 'chart_bar.xls' ); my $worksheet = $workbook->add_worksheet(); my $bold = $workbook->add_format( bold => 1 ); # Add the worksheet data that the charts will refer to. my $headings = [ 'Number', 'Sample 1', 'Sample 2' ]; my $data = [ [ 2, 3, 4, 5, 6, 7 ], [ 1, 4, 5, 2, 1, 5 ], [ 3, 6, 7, 5, 4, 3 ], ]; $worksheet->write( 'A1', $headings, $bold ); $worksheet->write( 'A2', $data ); # Create a new chart object. In this case an embedded chart. my $chart = $workbook->add_chart( type => 'bar', embedded => 1 ); # Configure the first series. (Sample 1) $chart->add_series( name => 'Sample 1', categories => '=Sheet1!$A$2:$A$7', values => '=Sheet1!$B$2:$B$7', ); # Configure the second series. (Sample 2) $chart->add_series( name => 'Sample 2', categories => '=Sheet1!$A$2:$A$7', values => '=Sheet1!$C$2:$C$7', ); # Add a chart title and some axis labels. $chart->set_title ( name => 'Results of sample analysis' ); $chart->set_x_axis( name => 'Test number' ); $chart->set_y_axis( name => 'Sample length (cm)' ); # Insert the chart into the worksheet (with an offset). $worksheet->insert_chart( 'D2', $chart, 25, 10 ); __END__ AUTHOR
John McNamara jmcnamara@cpan.org COPYRIGHT
Copyright MM-MMX, John McNamara. All Rights Reserved. This module is free software. It may be used, redistributed and/or modified under the same terms as Perl itself. perl v5.10.1 2010-02-02 Spreadsheet::WriteExcel::Chart::Bar(3pm)
All times are GMT -4. The time now is 12:30 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy