awk - extracting data from a series of files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk - extracting data from a series of files
# 8  
Old 03-24-2011
Are you sure your looking at "outa.txt" and not "outa1.txt" or "outa2.txt" from the previous version of the program?
# 9  
Old 03-25-2011
Tools

Quote:
Originally Posted by Chubler_XL
Are you sure your looking at "outa.txt" and not "outa1.txt" or "outa2.txt" from the previous version of the program?
Hi Chubler.

I ran the script after changing the names - to avoid the confusion. The result is the same, I see only the data extracted from out3.log. Thanks! Smilie
# 10  
Old 03-28-2011
Bug

Quote:
Originally Posted by p_sun
Hi Chubler,
It doesn't do the trick. I guess it is overwriting the first output. Any other suggestions? Thank youSmilie
Any suggestions please.. Eager to try out you suggestions so that it takes care of my looong analysis. Thanks a lotSmilie
# 11  
Old 03-28-2011
Try this and check your outa.log file then:

Code:
rm -f outa.txt
for i in out1 out2 out3; do 
   echo "===== Processing file $i =====" >> outa.txt
   awk '/ test-file*/{print;m=0}' $i.log >> outa.txt
   echo "===== Finished with file $i =====" >> outa.txt
   awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4}m&&c==13{print $2 " " $3 " " $4 ;m=0}' $i.log > ${i}c.txt
done

# 12  
Old 03-29-2011
Debian

Quote:
Originally Posted by Chubler_XL
Try this and check your outa.log file then:

Code:
rm -f outa.txt
for i in out1 out2 out3; do 
   echo "===== Processing file $i =====" >> outa.txt
   awk '/ test-file*/{print;m=0}' $i.log >> outa.txt
   echo "===== Finished with file $i =====" >> outa.txt
   awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4}m&&c==13{print $2 " " $3 " " $4 ;m=0}' $i.log > ${i}c.txt
done

Thank you Chubler. The code worked great. I appreciate your time and help!!!Smilie
However, if I have to repeat the same for second awk (I have made a separate
script), it doesn't work.

rm -f outb.txt
for i in out1 out2 out3; do
echo "===== Processing file $i =====" >> outb.txt
awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4}m&&c==13{print $2 " " $3 " " $4 ;m=0}' $i.log >> outb.txt
echo "===== Finished with file $i =====" >> outb.txt
done

Last edited by p_sun; 03-31-2011 at 03:32 PM..
# 13  
Old 03-29-2011
cat out*.log | awk ....
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Problem in extracting data using cut/awk command

Hi Everyone, I have a very simple problem and i am stuck in that from last 8 days. I tried many attempts, googled my query but all in vain. I have a text file named "test.txt" In that suppose i have contents like: Java: 1 Object oriented programming language 2 Concepts of Abstraction... (5 Replies)
Discussion started by: Abhijeet Anand
5 Replies

2. Shell Programming and Scripting

Extracting part of data from files

Hi All, I have log files as below. log1.txt <table name="content_analyzer" primary-key="id"> <type="global" /> </table> <table name="content_analyzer2" primary-key="id"> <type="global" /> </table> Time taken: 1.008 seconds ID = gd54321bbvbvbcvb <table name="content_analyzer"... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies

3. Programming

Python script for extracting data using two files

Hello, I have two files. File 1 is a list of interested IDs Ex1 Ex2 Ex3File 2 is the original file with over 8000 columns and 20 millions rows and is a compressed file .gz Ex1 xx xx xx xx .... Ex2 xx xx xx xx .... Ex2 xx xx xx xx ....Now I need to extract the information for all the IDs of... (4 Replies)
Discussion started by: nans
4 Replies

4. Programming

Resample time series data with replacement any way to do this in awk or just bash script

I have some time series data that I need to resample or downsample at some specific time intervals. The firs column is time in decimal hours. I am tryiong to resample this data every 3 minutse. So I need a data value ever 0.05. Here is the example data and as you can see, there time slot for 0.1500... (3 Replies)
Discussion started by: malandisa
3 Replies

5. Shell Programming and Scripting

Using Awk for extracting data in specific format

please help me writing a awk script 001_r.pdb 0.0265185 001_r.pdb 0.0437049 001_r.pdb 0.0240642 001_r.pdb 0.0310264 001_r.pdb 0.0200482 001_r.pdb 0.0146746 001_r.pdb 0.0351344 001_r.pdb 0.0347856 001_r.pdb 0.036119 001_r.pdb 1.49 002_r.pdb 0.0281011 002_r.pdb 0.0319908 002_r.pdb... (5 Replies)
Discussion started by: phoenix_nebula
5 Replies

6. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,... (1 Reply)
Discussion started by: Boltzmann
1 Replies

7. UNIX for Dummies Questions & Answers

Iterate a min/max awk script over time-series temperature data

I'm trying to iterate a UNIX awk script that returns min/max temperature data for each day from a monthly weather data file (01_weath.dat). The temperature data is held in $5. The temps are reported each minute so each day contains 1440 temperature enteries. The below code has gotten me as far as... (5 Replies)
Discussion started by: jgourley
5 Replies

8. Shell Programming and Scripting

calculate the average of time series data using AWK

Hi, I have two time series data (below) merged into a file. t1 and t2 are in unit of second I want to calculate the average of V1 every second and count how many times "1" in V2 is occur within a second Input File: t1 V1 t2 V2 10.000000... (5 Replies)
Discussion started by: nica
5 Replies

9. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies

10. Shell Programming and Scripting

Perl - extracting data from .csv files

PROJECT: Extracting data from an employee timesheet. The timesheets are done in excel (for user ease) and then converted to .csv files that look like this (see color code key below): ,,,,,,,,,,,,,,,,,,, 9/14/2003,<-- Week Ending,,,,,,,,,,,,,,,,,, Craig Brennan,,,,,,,,,,,,,,,,,,,... (3 Replies)
Discussion started by: kregh99
3 Replies
Login or Register to Ask a Question