Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Count on grep for more than two values in a row of fixed length file

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Count on grep for more than two values in a row of fixed length file
# 8  
Old 02-13-2013
Quote:
Originally Posted by princetd001
How do I incorporate this condition as well in above script?
No need to incorporate this in the awk program.

Using a while loop construct with a find command will work:
Code:
find 2013/ -name "file*.txt" | while read file
do
   # Awk code here - pass "$file" as argument
done

Or if you have the list of files with absolute path in a file:
Code:
while read file
do
  # Awk code here - pass "$file" as argument
done < file_list

# 9  
Old 02-13-2013
Quote:
Originally Posted by bipinajith
No need to incorporate this in the awk program.

Using a while loop construct with a find command will work:
Code:
find 2013/ -name "file*.txt" | while read file
do
   # Awk code here - pass "$file" as argument
done

Or if you have the list of files with absolute path in a file:
Code:
while read file
do
  # Awk code here - pass "$file" as argument
done < file_list


I may not be knowing the file names. my scripts should run through each folder recurssively and find files and do the awk command that you provided.

my folder structure will be as follows

it is yyyy/mm/dd format for folder structure and each dd[date] folder will have files.

Code:
 
2013/02/01
2013/02/02
2013/02/03
.
..
.
2013/02/12
2013/02/13
.
.

# 10  
Old 02-13-2013
Did you even try my suggestion using find command?

Try it and let us know if you are facing any issues.
# 11  
Old 02-13-2013
Quote:
Originally Posted by bipinajith
Did you even try my suggestion using find command?

Try it and let us know if you are facing any issues.

Will below code work? I have not tried and but I wrote a draft.

Code:
 
#!/bin/ksh
printf "count\trecord\n" > file_prov_coutry_time.csv
find 2013/02/0[1-9] 2013/02/1[0-9] -name 'sasmm_fsbc_durds_id000[0-9]*_t?*.dat*' | while read path
do
file=$(basename "$path")
awk ' NR>1 && substr($0,4,2) == "bc" && substr($0,6,3) == "can" {
        TS = substr($0,9,14) + 0;
        if(TS>=12022013000000&&TS<=13022013000000) {
                R = R RS $0;
                c++;
        }
 } END {
        printf "Count=%d\n", c;
        printf "Records%s", R;
        printf "\n"
}' file
#       printf "%s\t%s\n" "$c" "$R"
        echo "$R" "$file" >> file_prov_coutry_time.csv
done

# 12  
Old 02-13-2013
Try this:
Code:
find 2013/02/[0-9][0-9] -name 'sasmm_fsbc_durds_id000[0-9]*_t?*.dat*' | while read file
do
awk ' NR>1 && substr($0,4,2) == "bc" && substr($0,6,3) == "can" {
        TS = substr($0,9,14) + 0;
        if(TS>=12022013000000&&TS<=13022013000000) {
                R = R RS $0;
                c++;
        }
 } END {
        printf "Count=%d\n", c;
        printf "Records%s", R;
        printf "\n"
}' "$file" >> file_prov_coutry_time.csv
done

# 13  
Old 02-20-2013
Code:
#!/usr/bin/ksh
##################################################################
#
# Remove Control M characters and invalid line break from the specified fil
# usage: removeInvalidChar FileName 
#
####################################################################
find 2013/02/14 -name 'SASMMF*_FISUP_DURDSARCHIVE_ID*.dat*' | while read file
do
 echo "1. in loop."
 IXT = substr($0,869,869);
 echo $IXT
awk ' NR>1 && substr($0,869,869) == "1" {
 
 echo "2. loop 2"
        
        
        
        TS = substr($0,11,27) + 0;
        if(TS>=20130116000000000&&TS<=20130119000000000) {
                R = R RS $0;
                c++;
        }
 } END {
        printf "Count=%d\n", c;
        printf "Records%s", R;
        printf "\n"
}' "$file" >> file_prov_coutry_time.csv
done

The above code gives me an error ")" not allowed @ line # 13

I f I want to check to see if substr($0,869,869) returns how many 1s and how mnay 0s and how many 2s from each file and print the statistics in a file along with the file name.

Code:
RETURN VALUE =1 COUNT = 30 FILE NAME = 'SASMMF1_FISUP_DURDSARCHIVE_ID1.dat
RETURN VALUE =0 COUNT = 10 FILE NAME = 'SASMMF1_FISUP_DURDSARCHIVE_ID1.dat
 
RETURN VALUE =2 COUNT = 5 FILE NAME = 'SASMMF1_FISUP_DURDSARCHIVE_ID1.dat
 
RETURN VALUE =0 COUNT = 2 FILE NAME = 'SASMMF2_FISUP_DURDSARCHIVE_ID2.dat

# 14  
Old 02-20-2013
I do see some errors in your code:

Why do you have substr function outside awk? It is not a valid function in KSH:
Code:
IXT = substr($0,869,869);
echo $IXT
awk ' NR>1 && substr($0,869,869) == "1" {

Also why there is an echo inside awk? It is also not valid, use print or printf instead:
Code:
awk ' NR>1 && substr($0,869,869) == "1" { 
echo "2. loop 2"

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Copy columns from one file into another and get sum of column values and row count

I have a file abc.csv, from which I need column 24(PurchaseOrder_TotalCost) to get the sum_of_amounts with date and row count into another file say output.csv abc.csv- UTF-8,,,,,,,,,,,,,,,,,,,,,,,,, ... (6 Replies)
Discussion started by: Tahir_M
6 Replies

2. Shell Programming and Scripting

Fixed Length file from a SQL script

Hi, I have a DB2 UDB 9.7 SQL script, as follows: I need to pass the script into Unix and generate a fixed length file from this. Can someone kindly provide a script to achieve it? SELECT CAST(COALESCE(CL_ID,'000000000') AS CHAR(9)) AS CL_ID ,STATUS... (5 Replies)
Discussion started by: ebsus
5 Replies

3. UNIX for Dummies Questions & Answers

Length of a fixed width file

I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on... (2 Replies)
Discussion started by: Amrutha24
2 Replies

4. UNIX for Dummies Questions & Answers

Fixed length file extracting values in columns

How do I extract values in a few columns in a row of a fixed length file? If there are 8 columns and I need to extract values of 2nd,4th and 6 th columns, how do i do that? I used cut command, this I used only for one column. How do I do it more than one column? The below command will give... (1 Reply)
Discussion started by: princetd001
1 Replies

5. Shell Programming and Scripting

Finding multiple column values and match in a fixed length file

Hi, I have a fixed length file where I need to verify the values of 3 different fields, where each field will have a different value. How can I do that in a single step. (6 Replies)
Discussion started by: naveen_sangam
6 Replies

6. Shell Programming and Scripting

how to grep only particular length of numeric values

hi i have two types of file 1. temp.0000000001.data (10 digit numeric) 2. temp.000000001.data (9 digit numeric) i want to search a file which is having 10 digit numeric in between the file name. i use command like this.. ls | grep temp.^*.data but this will give both the files as... (2 Replies)
Discussion started by: somi2yoga
2 Replies

7. UNIX for Dummies Questions & Answers

Convert a tab delimited/variable length file to fixed length file

Hi, all. I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file: 10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783 19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657 And this is the expected... (2 Replies)
Discussion started by: Everton_Silveir
2 Replies

8. UNIX for Dummies Questions & Answers

What the command to find out the record length of a fixed length file?

I want to find out the record length of a fixed length file? I forgot the command. Any body know? (9 Replies)
Discussion started by: tranq01
9 Replies

9. Shell Programming and Scripting

convert fixed length file to CSV

Newbie Looking for a script to convert my input file to delimited text file. Not familier with AWK or shell programing. Below is sample record in my input file and the expected output format. My OS is HPUX 11.23. Thanks in advance for your assistance. tbtbs input file:... (12 Replies)
Discussion started by: tbtbs
12 Replies

10. Shell Programming and Scripting

creating a fixed length output from a variable length input

Is there a command that sets a variable length? I have a input of a variable length field but my output for that field needs to be set to 32 char. Is there such a command? I am on a sun box running ksh Thanks (2 Replies)
Discussion started by: r1500
2 Replies
Login or Register to Ask a Question