Sponsored Content
Top Forums Shell Programming and Scripting awk - extracting data from a series of files Post 302507780 by p_sun on Thursday 24th of March 2011 06:58:24 PM
Old 03-24-2011
Bug

Quote:
Originally Posted by Chubler_XL
Code:
rm -f outa.txt
for i in out1 out2 out3; do 
   awk '/ test-file*/{print;m=0}' $i.log >> outa.txt
   awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4}m&&c==13{print $2 " " $3 " " $4 ;m=0}' $i.log > ${i}c.txt
done

Hi Chubler,
It doesn't do the trick. I guess it is overwriting the first output. Any other suggestions? Thank youSmilie
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl - extracting data from .csv files

PROJECT: Extracting data from an employee timesheet. The timesheets are done in excel (for user ease) and then converted to .csv files that look like this (see color code key below): ,,,,,,,,,,,,,,,,,,, 9/14/2003,<-- Week Ending,,,,,,,,,,,,,,,,,, Craig Brennan,,,,,,,,,,,,,,,,,,,... (3 Replies)
Discussion started by: kregh99
3 Replies

2. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies

3. Shell Programming and Scripting

calculate the average of time series data using AWK

Hi, I have two time series data (below) merged into a file. t1 and t2 are in unit of second I want to calculate the average of V1 every second and count how many times "1" in V2 is occur within a second Input File: t1 V1 t2 V2 10.000000... (5 Replies)
Discussion started by: nica
5 Replies

4. UNIX for Dummies Questions & Answers

Iterate a min/max awk script over time-series temperature data

I'm trying to iterate a UNIX awk script that returns min/max temperature data for each day from a monthly weather data file (01_weath.dat). The temperature data is held in $5. The temps are reported each minute so each day contains 1440 temperature enteries. The below code has gotten me as far as... (5 Replies)
Discussion started by: jgourley
5 Replies

5. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,... (1 Reply)
Discussion started by: Boltzmann
1 Replies

6. Shell Programming and Scripting

Using Awk for extracting data in specific format

please help me writing a awk script 001_r.pdb 0.0265185 001_r.pdb 0.0437049 001_r.pdb 0.0240642 001_r.pdb 0.0310264 001_r.pdb 0.0200482 001_r.pdb 0.0146746 001_r.pdb 0.0351344 001_r.pdb 0.0347856 001_r.pdb 0.036119 001_r.pdb 1.49 002_r.pdb 0.0281011 002_r.pdb 0.0319908 002_r.pdb... (5 Replies)
Discussion started by: phoenix_nebula
5 Replies

7. Programming

Resample time series data with replacement any way to do this in awk or just bash script

I have some time series data that I need to resample or downsample at some specific time intervals. The firs column is time in decimal hours. I am tryiong to resample this data every 3 minutse. So I need a data value ever 0.05. Here is the example data and as you can see, there time slot for 0.1500... (3 Replies)
Discussion started by: malandisa
3 Replies

8. Programming

Python script for extracting data using two files

Hello, I have two files. File 1 is a list of interested IDs Ex1 Ex2 Ex3File 2 is the original file with over 8000 columns and 20 millions rows and is a compressed file .gz Ex1 xx xx xx xx .... Ex2 xx xx xx xx .... Ex2 xx xx xx xx ....Now I need to extract the information for all the IDs of... (4 Replies)
Discussion started by: nans
4 Replies

9. Shell Programming and Scripting

Extracting part of data from files

Hi All, I have log files as below. log1.txt <table name="content_analyzer" primary-key="id"> <type="global" /> </table> <table name="content_analyzer2" primary-key="id"> <type="global" /> </table> Time taken: 1.008 seconds ID = gd54321bbvbvbcvb <table name="content_analyzer"... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies

10. Shell Programming and Scripting

Problem in extracting data using cut/awk command

Hi Everyone, I have a very simple problem and i am stuck in that from last 8 days. I tried many attempts, googled my query but all in vain. I have a text file named "test.txt" In that suppose i have contents like: Java: 1 Object oriented programming language 2 Concepts of Abstraction... (5 Replies)
Discussion started by: Abhijeet Anand
5 Replies
hexec(1)																  hexec(1)

NAME
hexec - a process execution hooking tool SYNOPSIS
hexec <OPTIONS> [expr] [cmd] [args...] DESCRIPTION
hexec is a tool to hook into process exececution calls (exec family of syscalls). You can define an expression that is executed against any hooked exec call. This expression may also contain a replacement exec call. OPTIONS SUMMARY
Here is a summary of the options to hexec. --help | -h Print a options/expr summary page --version | -v Print hexec version --log-out | -lo set output file for error and -print output OPTIONS
-h Print a options/expr summary page --help Print a options/expr summary page -lo Set the output file for error and -print output. This can be required if writing to stderr could cause malfunction because some processes read from stderr and expect a well defined output. EXPRESSIONS
The expression is executed against all process execution calls. If the expression returns true, the original call is skipped. <expr> -and <expr> <expr> -a <expr> <expr> <expr> Returns true if both expressions return true. If the left returns false, the right expression is never executed. <expr> -or <expr> <expr> -o <expr> Returns true if one of both expressions returns true. If the left returns true, the right expression is never executed. -path <pattern> Returns true if the path of the executable matches <pattern>. <pattern> is a bash compatible wild card pattern. -ipath <pattern> Same as -path, but case insensitive. -name <pattern> Returns true if the base name of the executable matches <pattern>. <pattern> is a bash compatible wild card pattern. -iname <pattern> Same as -name, but case insensitive. -contains <str> Returns true if the path of the executable containes the string <str>. -icontains <str> Same as -contains, but case insensitive. -print Print all arguments to the called process. Returns always true. -exec <cmd> [args...] ; Executes <cmd> with [args...] as arguments. This expression must be terminated with a semicolon. You can use argument placeholders in <cmd> and [args...] (see below). This expression always returns true. NOTE: Please be aware that you may need to escape or quote the terminating semicolon to not confuse your shell. -sh <script> Interprets <script> as a shell script by invoking /bin/sh with the arguments -c '<script>'. You can use argument placeholders inside <script>. Please note that -sh only expects a single argument and not a variable list of arguments (as -exec does). Using -sh is the same as using -exec sh -c <script>. PLACEHOLDERS FOR -exec EXPRESSION Every -exec expression can use placeholders in the argument list to obtain information from the original exec call. Each placeholder starts with { and ends with }. Use { if you want to use a { in your argument list. The placeholders are replaced when the -exec expression is evaluated. Placeholder types: {n} Will be replaced with the number of arguments in the original call. {<idx>} Will be replaced with the original argument at index <idx>. Example: {1} would give the first argument. You can prepend <idx> with placeholder flags. {} Will be replaced with all arguments from the original call. Each argument is seperated with a space. You can use placeholder flags. NOTE: The executable name is also considered as argument. This means that a call like "echo test" will result in two arguments, "echo" and "test". PLACEHOLDER FLAGS
s Every argument is inserted as single argument instead of concatenating all arguments. Consider the arguments 'a', 'b' and 'c'. Without s, -exec {} ; would result in -exec 'a b c' ;, which in many cases is not what you want. -exec {s} ; howewer would result in -exec 'a' 'b' 'c' ;. q Quote every single argument. e Escape all non alpha-numeric characters. This flag is very useful when using a "sh -c <...>" in the -exec expression. Examples -exec echo {} ; with the call 'a' 'b' 'c' results in: 'echo' 'a b c' -exec echo {q} ; with the call 'a' 'b' 'c' results in: 'echo' 'a' 'b' 'c' -exec sh -c 'echo {}; {}' ; with the call 'sh' '-c' 'gcc d.c >> log.txt' results in: 'sh' '-c' 'echo sh -c gcc d.c >> log.txt; sh -c gcc d.c >> log.txt' Please note that this will not do what you may expect, as the >> is handled wrong in this case. -exec sh -c 'echo {e}; {}' ; with the call 'sh' '-c' 'gcc d.c >> log.txt' results in: 'sh' '-c' 'echo sh -c gcc d.c >> log.txt; sh -c gcc d.c >> log.txt' CHANGES TO PROCESSES
hexec will add some environment variables to the hooked processes. These are (may not be complete): LD_PRELOAD hexec adds libhexec-hook.so to the list of preloaded libraries. HEXEC_EXPR_SHM Contains the name of the internal shared memory object. HEXEC_LOG_FD Contains the file descriptor for error and -print output. Please do never modify these environment variables. Also take care when you use these variables, because the name and content of the vari- ables may change in the future. EXAMPLES
hexec -name 'gcc' -exec ccache {s} ; make Calls make, which will then call gcc several times. The executable name of each hooked process execution is tested against the file pattern "gcc" and "ccache {s}" is called each time a match is found. '{s}' will be replaced with the original (the hooked) call. In this example, a call to "gcc -o test.o test.c" would be replaced with "ccache gcc -o test.o test.c" HOW IT WORKS
TODO BUGS
I'm sure there are alot...it's still beta :) AUTHOR
hexec was written by Alexander Block http://blocksoftware.net/ If you wish to report a problem or make a suggestion then please email ablock@blocksoftware.net hexec is released under the GNU General Public License version 2 or later. Please see the file COPYING for license details. November 2008 hexec(1)
All times are GMT -4. The time now is 09:23 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy