Bc in shell script is creating blank files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bc in shell script is creating blank files
# 1  
Old 05-15-2014
Bc in shell script is creating blank files

Hi,

I am creating a shell script which will check the processing of the main script if the error count is more than 2% of the record count in a feed. Part of the code is below:

Code:
err_tol=`echo $feed_cnt \* 0.02 |bc`

while read line
do
   err_cnt=$(grep -i "$line" $err_log | wc -l)

        if [ $err_cnt > $err_tol ]
        then
             echo "Errors exceed the tolerance level"
             exit 1
        fi
done <"$err_list"

feed_cnt - is the word count of the incoming feed
err_list - is a file with all errors that can be raised by the process

I am using bc in err_tol calculation as I need to multiply decimal numbers. This statement is creating blank files with the divided number value.

I have tried a few other methods and they seem to be creating blank files as well.
err_tol=`echo "$feed_cnt 0.02 * p" |dc`
err_tol=$(echo "scale=2; $feed_cnt * 0.02" | bc)


Can anyone help with this issue? Not sure what I am doing wrong.

Last edited by Don Cragun; 05-15-2014 at 11:03 PM.. Reason: Add CODE and ICODE tags.
# 2  
Old 05-15-2014
There are several issues here.
  1. What operating system are you using?
  2. What shell are you using?
  3. What errors are you getting from your script?
  4. Is err_list the name of a file, or is it the name of a variable that contains the name of a file?
  5. Are you trying to see if there are more than 2% of a specific type of error as specified by one line in the file named by $err_list, or all types of errors specified by all of the lines in the file named by $err_list?
  6. How is feed_cnt set?
  7. The command:
    Code:
    [ $err_cnt > $err_tol ]

    creates a file named by the expansion $err_tol, and stores the output of the command:
    Code:
    [ $err_cnt ]

    in that file. If you were trying to do a numeric comparison, that would be:
    Code:
    [ "$err_cnt" -gt "$err_tol" ]

    but (unless you're using a 1993 or later version of ksh) that only works for integer values (and, if you're using a 1993 or later ksh, you don't need to use bc to do any of there calculations used here).

Last edited by Don Cragun; 05-15-2014 at 11:37 PM.. Reason: Change order to make list work.
This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 05-15-2014
Hi Don - Thanks for your reply.

What operating system are you using?
-- Unix

What shell are you using?
-- ksh

What errors are you getting from your script?
-- I am not getting any error from the script as such and is working as expected.

Is err_list the name of a file, or is it the name of a variable that contains the name of a file?
-- err_list is the name of a file which contains different error types a file can have.

Are you trying to see if there are more than 2% of a specific type of error as specified by one line in the file named by $err_list , or all types of errors specified by all of the lines in the file named by $err_list ?
-- I want to if all types of errors specified by all of the lines in the file name $err_list is more than 2%

How is feed_cnt set? -- This value is wordcount on the file (wc -l)

Changing the condition to -gt eliminated creating the blank files.
Any thoughts on how I can check the sum of all types of errors is more than 2% of the feed count?
# 4  
Old 05-16-2014
Test it

Code:
awk -v fc=$feed_cnt 'NR==FNR { f[$0]++; next } $0 in f { total += f[$0] }; END { if (total > (0.02 * fc)) print "Errors exceed the tolerance level" }' $err_log $err_list

This User Gave Thanks to Aia For This Post:
# 5  
Old 05-16-2014
You didn't say which version of UNIX you're using and you didn't say which version of ksh you're using. If you just want to use ksh and wc (instead of awk), you can also try the following:
Code:
#!/bin/ksh
#set -xv
err_log="err_log"
err_list="err_list"
feed_cnt=$(wc -l < "$err_log")
err_tol=$(((feed_cnt + 25) / 50))
# grep -iFf "$err_list" "$err_log"
err_cnt=$(grep -iFcf "$err_list" "$err_log")
if [ "$err_cnt" -gt "$err_tol" ]
then
	echo "Errors exceed the tolerance level"
	exit 1
fi

Note that Aia's awk script looks for exact (case sensitive) full line matches; the code above looks for case insensitive matches where the complete contents of any line from err_list appears anywhere on a line in err_log.

If you want the behavior of the script above, but want to use awk instead, the changes to Aia's awk script to get that behavior are minor.

Since you didn't give us any sample data, I tried to match the options you gave to grep. But, I assumed you were looking for fixed strings rather than having a list of basic regular expressions in err_list.

If we guessed wrong, give us details of your real requirements.

If you're using a 1993 or later version of ksh, we can use floating point arithmetic to compute err_tol instead of playing games with rounding using integer arithmetic.

If you have an old UNIX system with a grep that doesn't accept the -Foption, change grep -iFcf to fgrep -icf.
# 6  
Old 05-16-2014
Thanks Aia and Don. grep -iFcf worked like a charm.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Space and blank line from file in UNIX shell script

I have below file. I want to remove space at begining of every line and then after also remove blank line from file. I use below code for each operation. sed -e 's/^*//' < check.txt > check1.txt sed '/^\s*$/d' < check1.txt > check2.txt above code not remove all the space... (12 Replies)
Discussion started by: Mohin Jain
12 Replies

2. UNIX for Dummies Questions & Answers

Creating a blank string of a specified size

I want to have a string which has n blank spaces For example set N = 3 create str = " " So the length depends on the value of N. I am in tcsh. (4 Replies)
Discussion started by: kristinu
4 Replies

3. Shell Programming and Scripting

Need help creating a script to FTP files to a server and then delete the files that were transfered.

I am trying to FTP files to a Windows server through my Linux machine. I have setup the file transfer with no problems but am having problem deleting those files from the Linux box. My current non-working solution is below. Any ideas, anyone?? :wall: Please be gentle, I'm fairly new to this... (4 Replies)
Discussion started by: jmalfhs
4 Replies

4. Shell Programming and Scripting

shell script to check blank line?

Can someone help to transform the below logic into a shell script, might be easy for some of you. I have a file with below text, I need if the line has the ":" and the above to it is not a blank line should print " <text>: is incorrect format" Apple: vitamin = A... (0 Replies)
Discussion started by: usyseng
0 Replies

5. Shell Programming and Scripting

Problem with blank spaces in shell script

Hi, I did the following script: DIR=`pwd`;for i in `find . -name GESTION -type d`;do cd $i/..;setfacl -R -m g:directores:rwx, GESTION;echo $DIR'/'$i;cd $DIR;done This code do the following actions: 1. Starting inside a folder, it's searching for any folder called "GESTION" 2. Every time... (3 Replies)
Discussion started by: argie01
3 Replies

6. Shell Programming and Scripting

Dynamically creating text files using shell script

Hi All, I want to create a shell script which dynamically create text files. i am using the following script $i=1 while do cat > test_$i.txt done but while running the script it was stopping(the cursor not going to next step, i have to enter ctrl+c to make it stop). it is creating only... (2 Replies)
Discussion started by: KiranKumarKarre
2 Replies

7. Shell Programming and Scripting

Blank Space is not appending in each row of CSV File - Shell Script

I am calling SQL script in my UNIX Shell script and trying to create the CSV file and my last column value of each row is 23 blank spaces. In my SQL script,the last column is like below. RPAD(' ',23,' ') -- Padding 23 blank Spaces The CSV file is generated but the sapce(23 spaces) is... (2 Replies)
Discussion started by: praka
2 Replies

8. Shell Programming and Scripting

Shell script: Remove blank lines

Hi gurus, I have this file with blank lines in it. How do i remove them in shell? I tried these commands but not working: sed '/^ *$/d' or sed '/^$/d' Anybody has a better idea pls? Also there are lines which starts with a single space, how do we remove the space in those lines?... (3 Replies)
Discussion started by: gholdbhurg
3 Replies

9. UNIX for Dummies Questions & Answers

Script to move blank files

Anyone could give me an example of scrip to move blank files found into a dir? Thanks, Leandro Takeda (3 Replies)
Discussion started by: letakeda
3 Replies

10. Shell Programming and Scripting

Argument in shell script never blank

Hi All, I have a script which accepts a parameter which can either be blank, a specific value, or a wildcard value. But it never seems to be blank and the wildcard option seems to return the names of matching files in my directory. This happens even with the worlds simplest script that just... (1 Reply)
Discussion started by: cdines
1 Replies
Login or Register to Ask a Question