Sponsored Content
Full Discussion: Processing Multiple Files
Top Forums Shell Programming and Scripting Processing Multiple Files Post 302263128 by hakim on Sunday 30th of November 2008 10:41:01 PM
Old 11-30-2008
Processing Multiple Files

Hello Everyone,

I am new to scripting and confused with how to do this efficiently. I am trying to use AWK to do this.

I have a lot of files in a folder which has the data of my throughput measurements in two columns i.e. Serial # and Throughput. like this
177.994 847.9
178.996 858.0
179.996 847.8
180.992 851.2
.
.
.
I want to do the following with these files.

1. Separate the second column, so I only have the Throughput column.
2. Calculate the Max/Min/Average and Standard Deviation of that Column.
3. Save the Min, Max, Average, SD in a separate file with the FileName from which they were calculated

The file names are like this

p1-x20-y0-w
p1-x20-y0-wo
p1-x20-y10-w
p1-x20-y10-wo
p1-x20-y20-w
p1-x20-y20-wo
.
.
.
values with 'y' change from 0 to 100 in steps of 10
values with 'x' change from 20 to 40 in steps of 10

I have to submit a project report and tons to data to process before that. Can anybody help me with that?

Thanks,
Shahjehan
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

multiple file processing

I need to process a directory which will have a different amount of files in it from time to time. This is an error directory, I need to process each file indvidually in to one log file, then display the file to the user. So I would like to display the file name as well in the log file. example... (4 Replies)
Discussion started by: jagannatha
4 Replies

2. UNIX for Dummies Questions & Answers

Multiple excel files processing on unix

Hi all, I am faced with a rather unusual problem regarding interaction between NT and UNIX. I am using an ETL (Extract-Transform-Load) tool on unix that has the capability to read .xls files. So, when I FTP an excel (.xls) file from a windows server to unix and attempt to read it with this... (3 Replies)
Discussion started by: ucode_2482
3 Replies

3. UNIX for Dummies Questions & Answers

Script Problem-Processing multiple scripts

Hi All, I have four scripts to run. My 1st script will make script2 and script3 to run. I am setting a cron job for this script1 to run continuously. This script1 will check for 2 text files and based on the existance of those text files it will initiate the script2 and script3. Now my doubt... (2 Replies)
Discussion started by: RSC1985
2 Replies

4. UNIX for Dummies Questions & Answers

single output of awk script processing multiple files

Helllo UNIX Forum :) Since I am posting on this board, yes, I am new to UNIX! I read a copy of "UNIX made easy" from 1990, which felt like a making a "computer-science time jump" backwards ;) So, basically I have some sort of understanding what the basic concept is. Problem Description:... (6 Replies)
Discussion started by: Kasimir
6 Replies

5. Shell Programming and Scripting

multiple groups of files processing

I have five directories, dir1 to dir5 for each directory, I have all same number-named folders. There are four types of folders, {1..10}, {20..30}, { 40..50}, {60..70} Now for each types of folder, I will do the same thing, here is the code for i in {1..5} do cd dir$i mkdir temp1 for... (5 Replies)
Discussion started by: ksgreen
5 Replies

6. Shell Programming and Scripting

Processing multiple files awk

hai i need my single awk script to act on 4 trace files of ns2 and to calculate througput and it should print result from each trace file in a single trace file. i tried with the following code but it doesnt work awk -f awkscript inputfile1 inputfile2 inputfile3 inputfile4>outputfile ... (4 Replies)
Discussion started by: sarathyy
4 Replies

7. Shell Programming and Scripting

Passing multiple files to awk for processing in bash script

Hi, I'm using awk command in bash script. I'm able to pass multiple files to awk for processing.The code i can use is as below(sample code) #!/bin/bash awk -F "," 'BEGIN { ... ... ... }' file1 file2 file3 In the above code i'm passing the file names manually and it is fine till my... (7 Replies)
Discussion started by: shree11
7 Replies

8. Shell Programming and Scripting

Processing multiple files

Hello I have a program cfxfrwb which is designed to remove headers from reports files. The cfxfrwb is located in the following directory /u01/efin/v40/live/bin I run the program against a single report file in the temp directory and it does it's job../cfxfrwb... (2 Replies)
Discussion started by: captainrhodes
2 Replies

9. Shell Programming and Scripting

Processing multiple files (environment setting)

Hello I posted on here a while ago about processing multiple files within a script. See original post below: I have a program cfxfrwb which is designed to remove headers from reports files. The cfxfrwb is located in the following directory /u01/efin/v40/live/bin I run the program against... (8 Replies)
Discussion started by: captainrhodes
8 Replies
mlib_ImageGridWarp(3MLIB)				    mediaLib Library Functions					 mlib_ImageGridWarp(3MLIB)

NAME
mlib_ImageGridWarp - grid-based image warp SYNOPSIS
cc [ flag... ] file... -lmlib [ library... ] #include <mlib.h> mlib_status mlib_ImageGridWarp(mlib_image *dst, const mlib_image *src, const mlib_f32 *xWarpPos, const mlib_f32 *yWarpPos, mlib_d64 post- ShiftX, mlib_d64 postShiftY, mlib_s32 xStart, mlib_s32 xStep, mlib_s32 xNumCells, mlib_s32 yStart, mlib_s32 yStep, mlib_s32 yNumCells, mlib_filter filter, mlib_edge edge); DESCRIPTION
The mlib_ImageGridWarp() function performs a regular grid-based image warp. The images must have the same type, and the same number of channels. The images can have 1, 2, 3, or 4 channels. The data type of the images can be MLIB_BYTE, MLIB_SHORT, MLIB_USHORT, or MLIB_INT. The two images may have different sizes. The image pixels are assumed to be centered at .5 coordinate points. For example, the upper-left corner pixel of an image is located at (0.5, 0.5). For each pixel in the destination image, its center point D is, first, backward mapped to a point S in the source image; then the source pixels with their centers surrounding point S are selected to do one of the interpolations specified by the filter parameter to generate the pixel value for point D. The mapping from destination pixels to source positions is described by bilinear interpolation between a rectilinear grid of points with known mappings. Given a destination pixel coordinate (x, y) that lies within a cell having corners at (x0, y0), (x1, y0), (x0, y1) and (x1, y1), with source coordinates defined at each respective corner equal to (sx0, sy0), (sx1, sy1), (sx2, sy2) and (sx3, sy3), the source position (sx, sy) that maps onto (x, y) is given by the formulas: xfrac = (x - x0)/(x1 - x0) yfrac = (y - y0)/(y1 - y0) s = sx0 + (sx1 - sx0)*xfrac t = sy0 + (sy1 - sy0)*xfrac u = sx2 + (sx3 - sx2)*xfrac v = sy2 + (sy3 - sy2)*xfrac sx = s + (u - s)*yfrac - postShiftX sy = t + (v - t)*yfrac - postShiftY In other words, the source x and y values are interpolated horizontally along the top and bottom edges of the grid cell, and the results are interpolated vertically: (x0, y0) -> (x1, y0) -> (sx0, sy0) (sx1, sy1) +------------+---------+ | /| | | (s, t) | | | | | | | | | | | | | | | (x, y) -> | | | (sx, sy)--+ | | | | | | | | | (u, v) | | |/ | +------------+---------+ (x0, y1) -> (x1, y1) -> (sx2, sy2) (sx3, sy3) The results of above interpolation are shifted by (-postShiftX, -postShiftY) to produce the source pixel coordinates. The destination pixels that lie outside of any grid cells are kept intact. The grid is defined by a set of equal-sized cells. The grid starts at (xStart, yStart). Each cell has width equal to xStep and height equal to yStep, and there are xNumCells cells horizontally and yNumCells cells vertically. The degree of warping within each cell is defined by the values in xWarpPos and yWarpPos parameters. Each of these parameters must contain (xNumCells + 1)*(yNumCells + 1) values, which, respectively, contain the source X and source Y coordinates that map to the upper-left cor- ner of each cell in the destination image. The cells are enumerated in row-major order. That is, all the grid points along a row are enu- merated first, then the grid points for the next row are enumerated, and so on. For example, suppose xNumCells is equal to 2 and yNumCells is equal to 1. Then the order of the data in the xWarpPos would be: x00, x10, x20, x01, x11, x21 and in the yWarpPos: y00, y10, y20, y01, y11, y21 for a total of (2 + 1)*(1 + 1) = 6 elements in each table. PARAMETERS
The function takes the following arguments: dst Pointer to destination image. src Pointer to source image. xWarpPos A float array of length (xNumCells + 1)*(yNumCells + 1) containing horizontal warp positions at the grid points, in row- major order. yWarpPos A float array of length (xNumCells + 1)*(yNumCells + 1) containing vertical warp positions at the grid points, in row-major order. postShiftX The displacement to apply to source X positions. postShiftY The displacement to apply to source Y positions. xStart The minimum X coordinate of the grid. xStep The horizontal spacing between grid cells. xNumCells The number of grid cell columns. yStart The minimum Y coordinate of the grid. yStep The vertical spacing between grid cells. yNumCells The number of grid cell rows. filter Type of resampling filter. It can be one of the following: MLIB_NEAREST MLIB_BILINEAR MLIB_BICUBIC MLIB_BICUBIC2 edge Type of edge condition. It can be one of the following: MLIB_EDGE_DST_NO_WRITE MLIB_EDGE_SRC_PADDED RETURN VALUES
The function returns MLIB_SUCCESS if successful. Otherwise it returns MLIB_FAILURE. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Interface Stability |Evolving | +-----------------------------+-----------------------------+ |MT-Level |MT-Safe | +-----------------------------+-----------------------------+ SEE ALSO
mlib_ImageGridWarp_Fp(3MLIB), mlib_ImageGridWarpTable(3MLIB), mlib_ImageGridWarpTable_Fp(3MLIB), attributes(5) SunOS 5.10 10 Nov 2004 mlib_ImageGridWarp(3MLIB)
All times are GMT -4. The time now is 04:52 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy