The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Selective extraction of data from a files dwgi32 Shell Programming and Scripting 3 06-26-2009 04:52 AM
Extract multiple repeated data from a text file apjneeraj Shell Programming and Scripting 5 04-14-2009 02:40 AM
extraction of data from a text file which follows certain pattern mohkris UNIX for Advanced & Expert Users 4 09-23-2008 07:09 AM
Read the data from multiple files and sum the value nvuradi Shell Programming and Scripting 9 09-11-2008 01:08 PM
grep multiple text files in folder into 1 text file? coppertone UNIX for Dummies Questions & Answers 7 08-23-2002 02:50 PM

Reply
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 07-02-2009
ackheron ackheron is offline
Registered User
  
 

Join Date: Jul 2009
Posts: 2
Lightbulb Extraction of data from multiple text files, and creation of a chart

Hello dear friends,

My problem as explained below seems really basic.
Fact is that I'm totally new to programming, and have only a week to produce a script ( CShell or Perl ? ) to perform this action.

While searching on the forums, I found a command that could help me, but I don't know how to use it properly :s.

But first of all . . . the context.
I have 7300 text files, each containing the same type of information.
Below is shown how the text files are written :


3 PARAMETERS ON 05JAN01.
1 AZU1 STA X -0.247297943318988E+07 +- 0.241394876950605E-02
2 AZU1 STA Y -0.467133800760033E+07 +- 0.374164031593362E-02
3 AZU1 STA Z 0.355810775092403E+07 +- 0.257172727253398E-02
2 1 0.571958818322482E+00
3 1 -0.659378461245129E+00
3 2 -0.847505892381651E+00
AZU1 ANTENNA LC 0.1636 0.0000 0.0000 !up north east (m)


In order to analyze all the data, I need to extract the bold values, and put each one in a different column of an Excel file (or equivalent) or in a text file ( but then with the three values on a line, and maybe try to create a csv file ).

To finish, I have a file like this in a different folder . . . . and there's 3 or 4 levels of difference in the arborescence. As :

desktop/data_2005/05JAN01/out/solution/05JAN01AZU1_fix.txt
desktop/data_2005/05JAN02/out/solution/05JAN02AZU1_fix.txt
and it goes on for the whole year . . .

The idea would be to use the 'grep' command I guess :
I tried with this "egrep "1|2|3" < 05JAN01AZU1_fix.stacov.txt | awk '{ print $5 }'>> output.txt" but it doesn't work properly. It did output cleanly the 3 values that I want when I changed the "1|2|3" to "10|20|30" and did the changed the values 1,2,3 to 10,20,30before the three values I wanted to extract.

So, basically . . . I have a text file for each day of the year, containing three values that I want to put in three separate columns of an excel file, or in the same line of the csv file.

I guess that I have to find the way to extract correctly my values, but also that I have to write a kind of loop, to go through all the folders(days) of the year.

CShell or Perl ?

Does anyone has one or more ideas to help me go further ?
If you do, I'll be pleased if you drop it there
Thanks . . .

P.S :I don't know who wrote the rules of the forum . . . the point n°15 "No smoking in the forums" made me smile . . .
  #2 (permalink)  
Old 07-02-2009
jim mcnamara jim mcnamara is offline Forum Staff  
...@...
  
 

Join Date: Feb 2004
Location: NM
Posts: 5,643
Don't use cshell - sounds like you are in university... Do you have bash or ksh or even Bourne? There are very valid reasons for avoiding csh.

Here is a non- Cshell answer
Code:
find /desktop/data_2005  -name '*AZU1_fix.txt' |
while read filename
do
     awk '{ printf("%s ", $5)} END {printf("\n")} ' $filename
done | tr -s ' ' ',' > data_2005.csv
  #3 (permalink)  
Old 07-05-2009
ackheron ackheron is offline
Registered User
  
 

Join Date: Jul 2009
Posts: 2
Thank you very much Jim, it does work perfectly
I still have to search to understand fully how it works . . . I mean the syntax and everything . . . I would like to output the dates e.g. "05JAN01" in the first column aswell . . .

I do notice a difference between the values of the txt files and the values in the csv . . .
For instance when the first value is " -0.247297943318988E+07" (see the exemple in my first post ), it becomes "-2472979.90616501" (decimal part, it seems). I don't know yet if I actually need that precision . . . I'm checking this.

Any ideas ?

Your code will help me A LOT . . . Thank you again !
Ackheron.

Last edited by ackheron; 07-05-2009 at 07:46 AM..
Sponsored Links
Reply

Bookmarks

Tags
extraction text files multiple

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 06:41 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language translation by Google.
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0