Sponsored Content
Top Forums Shell Programming and Scripting awk - extracting data from a series of files Post 302508694 by Chubler_XL on Monday 28th of March 2011 06:33:22 PM
Old 03-28-2011
Try this and check your outa.log file then:

Code:
rm -f outa.txt
for i in out1 out2 out3; do 
   echo "===== Processing file $i =====" >> outa.txt
   awk '/ test-file*/{print;m=0}' $i.log >> outa.txt
   echo "===== Finished with file $i =====" >> outa.txt
   awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4}m&&c==13{print $2 " " $3 " " $4 ;m=0}' $i.log > ${i}c.txt
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl - extracting data from .csv files

PROJECT: Extracting data from an employee timesheet. The timesheets are done in excel (for user ease) and then converted to .csv files that look like this (see color code key below): ,,,,,,,,,,,,,,,,,,, 9/14/2003,<-- Week Ending,,,,,,,,,,,,,,,,,, Craig Brennan,,,,,,,,,,,,,,,,,,,... (3 Replies)
Discussion started by: kregh99
3 Replies

2. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies

3. Shell Programming and Scripting

calculate the average of time series data using AWK

Hi, I have two time series data (below) merged into a file. t1 and t2 are in unit of second I want to calculate the average of V1 every second and count how many times "1" in V2 is occur within a second Input File: t1 V1 t2 V2 10.000000... (5 Replies)
Discussion started by: nica
5 Replies

4. UNIX for Dummies Questions & Answers

Iterate a min/max awk script over time-series temperature data

I'm trying to iterate a UNIX awk script that returns min/max temperature data for each day from a monthly weather data file (01_weath.dat). The temperature data is held in $5. The temps are reported each minute so each day contains 1440 temperature enteries. The below code has gotten me as far as... (5 Replies)
Discussion started by: jgourley
5 Replies

5. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,... (1 Reply)
Discussion started by: Boltzmann
1 Replies

6. Shell Programming and Scripting

Using Awk for extracting data in specific format

please help me writing a awk script 001_r.pdb 0.0265185 001_r.pdb 0.0437049 001_r.pdb 0.0240642 001_r.pdb 0.0310264 001_r.pdb 0.0200482 001_r.pdb 0.0146746 001_r.pdb 0.0351344 001_r.pdb 0.0347856 001_r.pdb 0.036119 001_r.pdb 1.49 002_r.pdb 0.0281011 002_r.pdb 0.0319908 002_r.pdb... (5 Replies)
Discussion started by: phoenix_nebula
5 Replies

7. Programming

Resample time series data with replacement any way to do this in awk or just bash script

I have some time series data that I need to resample or downsample at some specific time intervals. The firs column is time in decimal hours. I am tryiong to resample this data every 3 minutse. So I need a data value ever 0.05. Here is the example data and as you can see, there time slot for 0.1500... (3 Replies)
Discussion started by: malandisa
3 Replies

8. Programming

Python script for extracting data using two files

Hello, I have two files. File 1 is a list of interested IDs Ex1 Ex2 Ex3File 2 is the original file with over 8000 columns and 20 millions rows and is a compressed file .gz Ex1 xx xx xx xx .... Ex2 xx xx xx xx .... Ex2 xx xx xx xx ....Now I need to extract the information for all the IDs of... (4 Replies)
Discussion started by: nans
4 Replies

9. Shell Programming and Scripting

Extracting part of data from files

Hi All, I have log files as below. log1.txt <table name="content_analyzer" primary-key="id"> <type="global" /> </table> <table name="content_analyzer2" primary-key="id"> <type="global" /> </table> Time taken: 1.008 seconds ID = gd54321bbvbvbcvb <table name="content_analyzer"... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies

10. Shell Programming and Scripting

Problem in extracting data using cut/awk command

Hi Everyone, I have a very simple problem and i am stuck in that from last 8 days. I tried many attempts, googled my query but all in vain. I have a text file named "test.txt" In that suppose i have contents like: Java: 1 Object oriented programming language 2 Concepts of Abstraction... (5 Replies)
Discussion started by: Abhijeet Anand
5 Replies
UNZIP(1)						    BSD General Commands Manual 						  UNZIP(1)

NAME
unzip -- extract files from a ZIP archive SYNOPSIS
unzip [-aCcfjLlnopqtuv] [-d dir] zipfile DESCRIPTION
The following options are available: -a When extracting a text file, convert DOS-style line endings to Unix-style line endings. -C Match file names case-insensitively. -c Extract to stdout/screen. When extracting files from the zipfile, they are written to stdout. This is similar to -p, but does not suppress normal output. -d dir Extract files into the specified directory rather than the current directory. -f Update existing. Extract only files from the zipfile if a file with the same name already exists on disk and is older than the former. Otherwise, the file is silently skipped. -j Ignore directories stored in the zipfile; instead, extract all files directly into the extraction directory. -L Convert the names of the extracted files and directories to lowercase. -l List, rather than extract, the contents of the zipfile. -n No overwrite. When extracting a file from the zipfile, if a file with the same name already exists on disk, the file is silently skipped. -o Overwrite. When extracting a file from the zipfile, if a file with the same name already exists on disk, the existing file is replaced with the file from the zipfile. -p Extract to stdout. When extracting files from the zipfile, they are written to stdout. The normal output is suppressed as if -q was specified. -q Quiet: print less information while extracting. -t Test: do not extract anything, but verify the checksum of every file in the archive. -u Update. When extracting a file from the zipfile, if a file with the same name already exists on disk, the existing file is replaced with the file from the zipfile if and only if the latter is newer than the former. Otherwise, the file is silently skipped. -v List verbosely, rather than extract, the contents of the zipfile. This differs from -l by using the long listing. Note that most of the data is currently fake and does not reflect the content of the archive. -x pattern Exclude files matching the pattern pattern. -Z mode Emulate zipinfo(1L) mode. Enabling zipinfo(1L) mode changes the way in which additional arguments are parsed. Currently only zipinfo(1L) mode 1 is supported, which lists the file names one per line. Note that only one of -n, -o, and -u may be specified. If specified filename is "-", then data is read from stdin. ENVIRONMENT
If the UNZIP_DEBUG environment variable is defined, the -q command-line option has no effect, and additional debugging information will be printed to stderr. COMPATIBILITY
The unzip utility aims to be sufficiently compatible with other implementations to serve as a drop-in replacement in the context of the ports(7) system. No attempt has been made to replicate functionality which is not required for that purpose. For compatibility reasons, command-line options will be recognized if they are listed not only before but also after the name of the zipfile. Normally, the -a option should only affect files which are marked as text files in the zipfile's central directory. Since the archive(3) library reads zipfiles sequentially, and does not use the central directory, that information is not available to the unzip utility. Instead, the unzip utility will assume that a file is a text file if no non-ASCII characters are present within the first block of data decompressed for that file. If non-ASCII characters appear in subsequent blocks of data, a warning will be issued. The unzip utility is only able to process ZIP archives handled by libarchive(3). Depending on the installed version of libarchive, this may or may not include self-extracting archives. SEE ALSO
libarchive(3) HISTORY
The unzip utility appeared in FreeBSD 8.0. AUTHORS
The unzip utility and this manual page were written by Dag-Erling Smorgrav <des@FreeBSD.org>. It uses the archive(3) library developed by Tim Kientzle <kientzle@FreeBSD.org>. BSD
May 10, 2012 BSD
All times are GMT -4. The time now is 07:55 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy