extracting columns falling within specific ranges for multiple files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting extracting columns falling within specific ranges for multiple files
# 1  
Old 02-07-2012
extracting columns falling within specific ranges for multiple files

Hi, I need to create weekly files from daily records stored in individual monthly filenames from 1999-2010. my sample file structure is like the ones below:
Code:
daily record stored per month: 
199901.xyz, 199902.xyz, 199903.xyz, 199904.xyz ...199912.xyz

records inside 199901.xyz (original data has no header)
lat lon wind year month day
12.5 121.0 3.4 1999 1 1
12.4 121.1 2.2 1999 1 1
12.1 122.1 1.2 1999 1 2
12.3 124.3 1.0 1999 1 3 
12.5 122.2 1.8 1999 1 3...
12.1 123.7 2.3 1999 1 31

I want to output the values of lat,lon,wind based on a 7-day range and end up with this below:
Code:
199901_wk01.txt contains the values of lat,lon,wind for days from 1-7
199901_wk02.txt contains the values of lat,lon,wind for days 8-14
199901_wk03.txt contains the values of lat,lon,wind for days 15-21
199901_wk04.txt contains the values of lat,lon,wind for days 22-28

199902_wk01.txt contains the values of lat,lon,wind for days from 1-7
199902_wk02.txt contains the values of lat,lon,wind for days from 8-14
199902_wk03.txt contains the values of lat,lon,wind for days from 15-21
199902_wk04.txt contains the values of lat,lon,wind for days from 22-28

and so on for the rest of the files.

Thanks much in advance.
# 2  
Old 02-07-2012
Code:
awk 'BEGIN { MIN=-6 }; FNR==1 { MIN+=7 }; ($6 >= MIN) && ($6 <= (MIN+7))' file1 file2 file3 ...

Every time it switches files, it'll bump the value of MIN by 7.

If that's not the output you wanted, you'll have to be more specific -- show the exact output you want and how you calculated it, not just the things you want in it...
# 3  
Old 02-07-2012
Hi, thanks for your reply. Am sorry for the unclear requirement I have earlier posted. The exact output I wanted would contain these values. The daily records for each month vary and so I need to print all the column values depending on whether the day column is within a specific weekly bin.

Code:
199901_wk01.txt contains
12.5 121.0 3.4 1999 1 1 
12.4 121.1 2.2 1999 1 1 
12.1 122.1 1.2 1999 1 2 
12.3 124.3 1.0 1999 1 3  
12.5 122.2 1.8 1999 1 3
12.5 122.5 1.6 1999 1 5
12.3 121.4 1.2 1999 1 7

# 4  
Old 02-07-2012
Code:
#! /bin/bash

for file in *.txt
do
    f=`basename $file .txt`
    while read x
    do
        day=`echo $x | awk '{print $NF}'`
        if [ $day -ge 1 ] && [ $day -le 7 ]
        then
            echo $x >> ${f}_wk01.txt
        elif [ $day -ge 8 ] && [ $day -le 14 ]
        then
            echo $x >> ${f}_wk02.txt
        elif [ $day -ge 15 ] && [ $day -le 21 ]
        then
            echo $x >> ${f}_wk03.txt
        elif [ $day -ge 22 ] && [ $day -le 28 ]
        then
            echo $x >> ${f}_wk04.txt
        elif [ $day -ge 29 ] && [ $day -le 31 ]
        then
            echo $x >> ${f}_wk05.txt
        fi
    done < $file
done

This User Gave Thanks to balajesuri For This Post:
# 5  
Old 02-07-2012
thanks much balajesuri. i have a question on the code, the files has no header, which part on the code scans the specific column corresponding to days?

oh ok. am sorry, its ok now. thanks much as always for being helpful.Smilie

Last edited by ida1215; 02-07-2012 at 09:50 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Derivation of values falling on date ranges

Hi Guys, I am having below tables used in oracle bal ID BALANCE BAL_DATE 1 -11.71 01-JAN-05 00.00.00 1 -405.71 02-JAN-05 00.00.00 1 -760.71 03-JAN-05 00.00.00 ref_table PRODUCT EFF_FROM_DATE EFF_TO_DATE TYPE MIN_AMT MAX_AMT CHARGE 12 01-JAN-05 00.00.00 01-JAN-06... (6 Replies)
Discussion started by: rohit_shinez
6 Replies

2. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies

3. Shell Programming and Scripting

Sum values of specific column in multiple files, considering ranges defined in another file

I have a file (let say file B) like this: File B: A1 3 5 A1 7 9 A2 2 5 A3 1 3 The first column defines a filename and the other two define a range in that specific file. In the same directory, I have also three more files (File A1, A2 and A3). Here is 10 sample lines... (3 Replies)
Discussion started by: Bastami
3 Replies

4. Shell Programming and Scripting

Extracting specific files from multiple .tgz files

Hey, I have number of .tgz files and want to extract the file with the ending *results.txt from each one. I have tried for file in *.tgz; do tar --wildcards -zxf $file *results.txt; doneas well as list=$(ls *.tgz) for i in $list; do tar --wildcards -zxvf $i *.results.txt; done... (1 Reply)
Discussion started by: jfern
1 Replies

5. UNIX for Dummies Questions & Answers

Extracting rows from a text file based on the values of two columns (given ranges)

Hi, I have a tab delimited text file with multiple columns. The second and third columns include numbers that have not been sorted. I want to extract rows where the second column includes a value between -0.01 and 0.01 (including both numbers) and the first third column includes a value between... (1 Reply)
Discussion started by: evelibertine
1 Replies

6. Shell Programming and Scripting

Extracting multiple columns with awk

Hi everyone!! I need to apply a simple command to extract columns from a matrix, but I need to extract contemporary from the first to the tenth columns, than from the eleventh to the twentyth and so on... how can i do that? (1 Reply)
Discussion started by: gabrysfe
1 Replies

7. Shell Programming and Scripting

getting files between specific date ranges in solaris

hi ! how can i get files in a directory between certain date ranges ? say all files created/modified between Jan24 - Jan31 thanks (10 Replies)
Discussion started by: aliyesami
10 Replies

8. Shell Programming and Scripting

Extracting columns from multiple files with awk

hi everyone! I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next} {print a, $2}' file1 file2 I added the file3, file4 and... (10 Replies)
Discussion started by: orcaja
10 Replies

9. UNIX for Dummies Questions & Answers

Extracting columns from multiple files with awk

hi everyone! I already posted it in scripts, I'm sorry, it's doubled I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next}... (1 Reply)
Discussion started by: orcaja
1 Replies

10. Shell Programming and Scripting

extracting multiple consecutive columns using awk

Hello, I have a matrix 200*10,000 and I need to extract the columns between 40 and 77. I dont want to write in awk all the columns. eg: awk '{print $40, $41, $42,$43 ... $77}'. I think should exist a better way to do this. (10 Replies)
Discussion started by: auratus42
10 Replies
Login or Register to Ask a Question