Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Tricky Shell script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Tricky Shell script
# 57  
Old 08-17-2007
I think there should be an issue with the sed command in the line which is added new, so remove the sed part.

And check if that for loop is getting the correct line from the input file by printing the line. If not then redirect the output of "cat $inpFile | sort | uniq" to a new file and process that file.

I have commented out the redirection of my script, just for illustration.

Code:
#!/bin/ksh

inpFile=$1
outFile="$HOME/Output.txt"
errFile="$HOME/Error.txt"
tmpFile1="/tmp/sample1.txt"
tmpFile2="/tmp/sample2.txt"
sortFile="/tmp/sortfile.txt"

TOTAL_FILEDS=9
COUNT=1
PRINT_FLAG=0
LINE_NO=1
check_lines=0
rm -f $outFile $errFile
sort $inpFile | uniq -d # >> $errFile
#sort $inpFile | uniq -d | sed "s/$/& : Duplicate/g" # >> $errFile
for line in $(cat $inpFile | sort | uniq )
do
        count_fields=$(echo $line | sed "s/00,*//g;s/,$//g" | wc -c)
        cnt_flds1=$(echo $line | cut -d',' -f1-9 | sed "s/00,*//g;s/,$//g" | wc -c )
        cnt_flds2=$(echo $line | cut -d',' -f10-18 | sed "s/00,*//g;s/,$//g" | wc -c )
        cnt_flds3=$(echo $line | cut -d',' -f19-27 | sed "s/00,*//g;s/,$//g" | wc -c )
        if [[ $count_fields -eq 45 && $cnt_flds1 -eq 15 && $cnt_flds2 -eq 15 && $cnt_flds3 -eq 15 ]]
        then
                PRINT_FLAG=0
#               echo $line | sed "s/\(.\{26\}\),\(.\{26\}\),\(.\{26\}\).*/\1\n\2\n\3/g" > $sortFile
                echo $line | cut -d',' -f1-9 > $sortFile
                echo $line | cut -d',' -f10-18 >> $sortFile
                echo $line | cut -d',' -f19-27 >> $sortFile

                while [[ $COUNT -le $TOTAL_FILEDS ]]
                do
                        cut -d',' -f$COUNT $sortFile | grep -v 00 > $tmpFile1
                        cut -d',' -f$COUNT $sortFile | grep -v 00 | sort > $tmpFile2
                        diff $tmpFile1 $tmpFile2 > /dev/null
                        if [[ $? -eq 1 || ! -s $tmpFile1 ]]
                        then
                                PRINT_FLAG=1
                                break
                        fi
                COUNT=$((COUNT+1))
                done
                if [[ $PRINT_FLAG -eq 0 ]]
                then
                        echo "$line : SUCCESS" # >> $outFile
                else
                        echo "$line : ERROR in $COUNT" # >> $errFile
                fi
        else
                echo "$line : Out of Scope" # >> >> $errFile
        fi
        COUNT=1
        check_lines=$((check_lines+count_fields))
        if [[ $LINE_NO -eq 6 ]]
        then
                LINE_NO=0
                if [[ $check_lines -eq 270 ]]
                then
                        echo " **** Success ***** "
                else
                        echo "----- Error,Check the above -----"
                fi
                check_lines=0
        fi
        LINE_NO=$((LINE_NO+1))
done
rm -f $tmpFile1 $tmpFile2 $sortFile
exit 0

Output

Code:
$ >ksh final.ksh input2.txt
00,00,00,31,44,50,67,73,00,04,19,22,32,00,00,00,00,86,06,00,00,00,48,59,69,76,00 : SUCCESS
00,00,21,34,00,51,61,00,83,08,18,00,00,43,00,00,78,88,00,00,26,36,00,59,62,79,00 : SUCCESS
00,10,23,33,00,00,63,00,82,02,00,00,34,40,59,00,77,00,06,16,24,00,00,00,66,00,84 : SUCCESS
00,11,21,00,00,55,00,73,83,07,00,22,39,43,00,66,00,00,00,15,00,00,44,58,67,00,87 : SUCCESS
00,12,00,35,00,55,00,70,82,00,13,23,00,48,00,60,00,87,08,00,00,00,00,58,66,71,88 : SUCCESS
00,12,23,00,48,50,00,00,84,08,00,00,36,00,00,60,73,88,00,16,00,37,49,52,00,75,00 : SUCCESS
 **** Success ***** 
00,12,25,00,00,50,61,00,85,01,15,00,31,00,00,00,79,88,00,00,29,00,41,52,62,00,90 : SUCCESS
00,14,00,30,43,53,00,73,00,04,00,27,00,48,00,67,00,83,00,19,00,35,00,54,00,74,87 : SUCCESS
00,14,00,31,00,57,63,70,00,04,00,24,00,45,59,00,00,90,00,15,25,35,00,00,66,71,00 : SUCCESS
03,00,00,00,41,56,00,73,84,04,17,22,32,00,00,00,76,00,00,19,29,00,49,00,64,00,86 : SUCCESS
03,00,24,34,00,50,00,79,00,00,16,00,00,46,54,62,00,90,07,00,28,38,49,00,68,00,00,54 : Out of Scope
05,00,22,00,00,55,64,00,80,06,11,26,00,41,00,00,72,00,00,13,00,32,00,00,67,78,83 : SUCCESS
----- Error,Check the above -----
07,00,23,00,46,00,62,00,83,00,10,00,34,47,00,66,72,00,09,13,29,00,00,52,00,00,90,09 : Out of Scope
08,00,21,00,44,00,65,72,00,09,17,22,32,00,00,00,00,81,00,00,00,37,46,58,68,76,00,87,65,85 : Out of Scope
11,00,20,30,00,00,65,79,00,03,00,21,00,46,51,00,00,85,07,17,00,33,47,00,00,00,87,87,00 : Out of Scope
52,14,24,30,00,00,00,70,00,00,00,00,32,47,50,60,74,00,04,16,27,38,00,00,00,00,89 : ERROR in 1
56,10,00,36,40,00,00,00,80,00,12,25,00,00,51,62,72,00,09,00,00,37,49,00,69,00,84,07 : Out of Scope
66,11,00,34,41,00,63,00,00,07,00,29,00,00,57,00,72,81,00,15,00,00,49,58,68,76,00,87 : Out of Scope
----- Error,Check the above -----
77,00,00,30,40,57,00,75,00,00,00,25,37,00,58,68,00,81,00,16,00,00,46,00,69,77,89,12 : Out of Scope
88,12,00,38,42,00,00,00,85,00,13,23,00,00,52,60,74,00,06,00,00,39,44,00,67,00,87,86 : Out of Scope

Try to add flavour as per your requirement.

Bye Smilie
# 58  
Old 08-20-2007
MySQL
lorcan,
In this script--
#!/usr/bin/ksh

inpFile=$1
outFile="$HOME/Output.txt"
errFile="$HOME/Error.txt"
dupFile="$HOME/Duplicate.txt"
tmpFile1="/tmp/sample1.txt"
tmpFile2="/tmp/sample2.txt"
sortFile="/tmp/sortfile.txt"

TOTAL_FILEDS=9
COUNT=1
PRINT_FLAG=0
LINE_NO=1
check_lines=0
rm -f $outFile $errFile
sort $inpFile | uniq -d > $dupFile
#sort $inpFile | uniq -d | sed "s/$/& : Duplicate/g" >> $errFile
for line in $(cat $inpFile)
do
count_fields=$(echo $line | sed "s/00,*//g;s/,$//g" | wc -c)
echo "count_fields is: $count_fields"
cnt_flds1=$(echo $line | cut -d',' -f1-9 | sed "s/00,*//g;s/,$//g" | wc -c )
echo "cnt_flds1 is : $cnt_flds1"
cnt_flds2=$(echo $line | cut -d',' -f10-18 | sed "s/00,*//g;s/,$//g" | wc -c )
echo "cnt_flds2 is : $cnt_flds2"
cnt_flds3=$(echo $line | cut -d',' -f19-27 | sed "s/00,*//g;s/,$//g" | wc -c )
echo "cnt_flds3 is : $cnt_flds3"
if [[ $count_fields -eq 45 && $cnt_flds1 -eq 15 && $cnt_flds2 -eq 15 && $cnt_flds3 -eq 15 ]]
then
PRINT_FLAG=0

echo $line | cut -d',' -f1-9 > $sortFile
echo $line | cut -d',' -f10-18 >> $sortFile
echo $line | cut -d',' -f19-27 >> $sortFile

while [[ $COUNT -le $TOTAL_FILEDS ]]
do
cut -d',' -f$COUNT $sortFile | grep -v 00 > $tmpFile1
cut -d',' -f$COUNT $sortFile | grep -v 00 | sort > $tmpFile2
diff $tmpFile1 $tmpFile2 > /dev/null
if [[ $? -eq 1 || ! -s $tmpFile1 ]]
then
PRINT_FLAG=1
break
fi
COUNT=$((COUNT+1))
done
echo "Value of print flag is : $PRINT_FLAG"
if [[ $PRINT_FLAG -eq 0 ]]
then
echo "$line : SUCCESS" >> $outFile
else
echo "$line : ERROR in $COUNT" >> $errFile
fi
else
echo "$line : Out of Scope" >> $errFile
fi
COUNT=1
check_lines=$((check_lines+count_fields))
if [[ $LINE_NO -eq 6 ]]
then
LINE_NO=0
if [[ $check_lines -eq 270 ]]
then
echo " **** Success ***** "
else
echo "----- Error,Check the above -----"
fi
check_lines=0
fi
LINE_NO=$((LINE_NO+1))
done
rm -f $tmpFile1 $tmpFile2 $sortFile
exit 0


When i am running the script,the value of PRINT_FLAG is getting 1 that means the data is not in sort order,trying to debug but i am not able.Please help me out in this.

The input File is --
03,00,00,00,41,56,00,73,84,04,17,22,32,00,00,00,76,00,00,19,29,00,49,00,64,00,86
77,00,00,30,40,57,00,75,00,00,00,25,37,00,58,68,00,81,00,16,00,00,46,00,69,77,89,12
03,00,24,34,00,50,00,79,00,00,16,00,00,46,54,62,00,90,07,00,28,38,49,00,68,00,00,54
00,12,00,35,00,55,00,70,82,00,13,23,00,48,00,60,00,87,08,00,00,00,00,58,66,71,88
07,00,23,00,46,00,62,00,83,00,10,00,34,47,00,66,72,00,09,13,29,00,00,52,00,00,90,09
00,00,00,31,44,50,67,73,00,04,19,22,32,00,00,00,00,86,06,00,00,00,48,59,69,76,00
11,00,20,30,00,00,65,79,00,03,00,21,00,46,51,00,00,85,07,17,00,33,47,00,00,00,87,87,00
00,14,00,31,00,57,63,70,00,04,00,24,00,45,59,00,00,90,00,15,25,35,00,00,66,71,00
00,12,23,00,48,50,00,00,84,08,00,00,36,00,00,60,73,88,00,16,00,37,49,52,00,75,00
05,00,22,00,00,55,64,00,80,06,11,26,00,41,00,00,72,00,00,13,00,32,00,00,67,78,83
66,11,00,34,41,00,63,00,00,07,00,29,00,00,57,00,72,81,00,15,00,00,49,58,68,76,00,87
00,11,21,00,00,55,00,73,83,07,00,22,39,43,00,66,00,00,00,15,00,00,44,58,67,00,87
00,12,25,00,00,50,61,00,85,01,15,00,31,00,00,00,79,88,00,00,29,00,41,52,62,00,90
00,00,21,34,00,51,61,00,83,08,18,00,00,43,00,00,78,88,00,00,26,36,00,59,62,79,00
# 59  
Old 08-23-2007
Quote:
Originally Posted by namishtiwari
lorcan,
thanks for the input.

when i ran the script it is telling

./testingscript testingdata
Check the 1 field of the INPUT FILE

it is not incrementing the COUNT value so it is printing only a single record at a time.

You mentioned ---
This script checks for the 15 tickets in a row and also checks if there are 5 tickets within every 9 columns.

And as a whole i am checking whether there are atleast one ticket in a column and arranged in ascending order.

The first one you mentioned is perfectly fine.

the second line is somewhat like this--
00,11,21,00,00,55,00,73,83,07,00,22,39,43,00,66,00,00,00,15,00,00,44,58,67,00,87

this is one data in a file,it consists of 3 rows and 9 columns.we can break
00,11,21,00,00,55,00,73,83
07,00,22,39,43,00,66,00,00
00,15,00,00,44,58,67,00,87

We need to check in this that each column should have min 1 value and if more than that then it should be in ascending order not as a whole.

Thanks for your kind help.
/* Error messages ;
“COL”i”FNW”=Column numberx Total Field amount less than 1 or more than 2
“COL””i”NA” =Column numberx Not ascending
“RW1NA”=Row1 not ascending
“R1FN5 " = Row1 Total field amount is less or more than 5
*/
/*NR is line number of inputfile */

Try this code ;

cat inputfile | awk -F',' '{
last1=0;last2=0;last3=0;i=0;k=0;l=0;m=0;msg="";r1=0;r2=0;r3=0;
while ( i < 9 )
{
i++;n=0;
if ( $i != 0 )
{
n++;k++;
if ( $i > last1 ) {last1=$i}else { r1=1}};

if ( $(i+9) != 0 )
{
n++;l++;
if ( $(i+9) > last2 ) {last2=$(i+9)}else { r2=1 }};

if ( $(i+18) != 0 )
{
n++;m++;
if ( $(i+18) > last3 ) {last3=$(i+18)}else { r3=1 }};

if ( n < 1 )
{ msg=msg"COL"i"FNW " }
else
{
if (( $i != 0 ) && ( $(i+9) != 0 ) && ( $(i+9) < $i )){msg=msg"COL"i"NA "}
else {
if (( $i != 0 ) && ( $(i+18) != 0 ) && ( $(i+18) < $i )){msg=msg"COL"i"NA "}
else {
if (( $(i+9) != 0 ) && ( $(i+18) != 0 ) && ( $(i+18) < $(i+9) )){msg=msg"COL"i
"NA " }
else {
if ((( $i != 0 ) && ( $(i+9) != 0 ) && ( $(i+18) != 0 )) && ( ( $i > $(i+9) )
|| ( $i > $(i+18) ) || ( $(i+9) > $ (i+18) ) )) { msg=msg"COL"i"NA " } ;}}}}};

if ( r1 != 0 ) {msg=msg"RW1NA "};
if ( r2 != 0 ) {msg=msg"RW2NA "};
if ( r3 != 0 ) {msg=msg"RW3NA "};
if ( k != 5 ) {msg=msg"R1FN5 "};
if ( l != 5 ) {msg=msg"R2FN5 "};
if ( m != 5 ) {msg=msg"R3FN5 "};
if ( msg != "" ) {printf ( "%d " "%s" "\n",NR,msg)};
}'
# 60  
Old 08-23-2007
Quote:
Originally Posted by namishtiwari
lorcan,
thanks for the input.

when i ran the script it is telling

./testingscript testingdata
Check the 1 field of the INPUT FILE

it is not incrementing the COUNT value so it is printing only a single record at a time.

You mentioned ---
This script checks for the 15 tickets in a row and also checks if there are 5 tickets within every 9 columns.

And as a whole i am checking whether there are atleast one ticket in a column and arranged in ascending order.

The first one you mentioned is perfectly fine.

the second line is somewhat like this--
00,11,21,00,00,55,00,73,83,07,00,22,39,43,00,66,00,00,00,15,00,00,44,58,67,00,87

this is one data in a file,it consists of 3 rows and 9 columns.we can break
00,11,21,00,00,55,00,73,83
07,00,22,39,43,00,66,00,00
00,15,00,00,44,58,67,00,87

We need to check in this that each column should have min 1 value and if more than that then it should be in ascending order not as a whole.

Thanks for your kind help.
/* Error messages ;
“COL”i”FNW”=Column numberx Total Field amount less than 1 or more than 2
“COL””i”NA” =Column numberx Not ascending
“RW1NA”=Row1 not ascending
“R1FN5 " = Row1 Total field amount is less or more than 5
*/
/*NR is line number of inputfile */

Try this code ;

cat inputfile | awk -F',' '{
last1=0;last2=0;last3=0;i=0;k=0;l=0;m=0;msg="";r1=0;r2=0;r3=0;
while ( i < 9 )
{
i++;n=0;
if ( $i != 0 )
{
n++;k++;
if ( $i > last1 ) {last1=$i}else { r1=1}};

if ( $(i+9) != 0 )
{
n++;l++;
if ( $(i+9) > last2 ) {last2=$(i+9)}else { r2=1 }};

if ( $(i+18) != 0 )
{
n++;m++;
if ( $(i+18) > last3 ) {last3=$(i+18)}else { r3=1 }};

if ( n < 1 )
{ msg=msg"COL"i"FNW " }
else
{
if (( $i != 0 ) && ( $(i+9) != 0 ) && ( $(i+9) < $i )){msg=msg"COL"i"NA "}
else {
if (( $i != 0 ) && ( $(i+18) != 0 ) && ( $(i+18) < $i )){msg=msg"COL"i"NA "}
else {
if (( $(i+9) != 0 ) && ( $(i+18) != 0 ) && ( $(i+18) < $(i+9) )){msg=msg"COL"i
"NA " }
else {
if ((( $i != 0 ) && ( $(i+9) != 0 ) && ( $(i+18) != 0 )) && ( ( $i > $(i+9) )
|| ( $i > $(i+18) ) || ( $(i+9) > $ (i+18) ) )) { msg=msg"COL"i"NA " } ;}}}}};

if ( r1 != 0 ) {msg=msg"RW1NA "};
if ( r2 != 0 ) {msg=msg"RW2NA "};
if ( r3 != 0 ) {msg=msg"RW3NA "};
if ( k != 5 ) {msg=msg"R1FN5 "};
if ( l != 5 ) {msg=msg"R2FN5 "};
if ( m != 5 ) {msg=msg"R3FN5 "};
if ( msg != "" ) {printf ( "%d " "%s" "\n",NR,msg)};
}'
# 61  
Old 08-23-2007
fazliturk,
when i ran the script it tells--

bash-2.05$ ksh script5
awk: syntax error near line 33
awk: illegal statement near line 33
awk: syntax error near line 35
awk: bailing out near line 35

I just added a line at the beginning
#!/bin/ksh

Thanks,
Namish
# 62  
Old 08-23-2007
Quote:
Originally Posted by namishtiwari
fazliturk,
when i ran the script it tells--

bash-2.05$ ksh script5
awk: syntax error near line 33
awk: illegal statement near line 33
awk: syntax error near line 35
awk: bailing out near line 35

I just added a line at the beginning
#!/bin/ksh

Thanks,
Namish
yes you are right. I've tried this code in korn shell . İn korn shell I haven't get any programing error messages. It runs coorectly and I thing it s code what you need . Did you try it in korn shell.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing xml - tricky one...need help!!

Hi, i am new to linux programming fraternity but looks like starting with a big thing... yes..xml parsing (it is indeed tough for a beginner like me) so need your kind help... The snippet of xml looks like: <snapshot> <tag1> <key>1234</key> <keytype>abcd</keytype> </tag1> <tag2>... (11 Replies)
Discussion started by: rookie2014
11 Replies

2. Shell Programming and Scripting

Tricky sed required

Hi All I need to put some sed together for a task and its a bit advanced for me, so I thought I'd ask if anyone here could help. I have a csv file with content like this - "","abcde","","" "'","abcde","","" "","","","1234" "'e'","","","" I need to remove any single quotes that fall... (17 Replies)
Discussion started by: steadyonabix
17 Replies

3. Solaris

Tricky egrep

Hi folks! My first post here. I'm working on a script that retrieves a range of files from a list depending on a range of time. UPDATE: I've seen it could be difficult to read all this thing, so I'll make a summarize it.. How come I do this and take a result.. grep "..\:.." lista.new |... (4 Replies)
Discussion started by: kl0x
4 Replies

4. Shell Programming and Scripting

Tricky data manipulation...

Hi everyone.. I am new here, hello.. I hope this doesn't come across to you folks as a stupid question, I'm somewhat new to scripting :) I'm seeking some help in finding a way to manipulate data output for every two characters - example: numbers.lst contains the following output:... (3 Replies)
Discussion started by: explicit
3 Replies

5. Shell Programming and Scripting

Tricky - Need help on Shell script variables

Hi, I have a requirement in which i have to read a csv file and put data in certain set of variables: File content: VP-DTL-REC-CNT, ,854840,0.00,VP-PAID-AMT, ,0,32280885.17,VP-PAT-PAID-AMT, ,0,9930244.32,VP-PAID-REV-CNT, ,484927,0.00,VP-REJ-CNT, ,369913,0.00, , ,0,0.00, , ,0,0.00, , ,0,0.00, ,... (3 Replies)
Discussion started by: shantoshkumar
3 Replies

6. Shell Programming and Scripting

Linux: Writing a tricky script to check connectivity

So, first and foremost, I'm having issues with my internet connection. Periodically, the connection drops across the network. The fix is simple enough: restart the modem. However, this gets old when the connection dies out every hour. I can hit my surfboard on 192.168.100.1, and navigate to a... (5 Replies)
Discussion started by: kungfujoe
5 Replies

7. UNIX for Dummies Questions & Answers

Tricky Quotation Question

Hi, I am at a point in my script where I defined the number of the command line parameter I would like to set a variable equal to: parameter_number=14 I would then like to set a variable equal to the correct parameter: variable=$parameter_number The issue here is that {} is required... (2 Replies)
Discussion started by: msb65
2 Replies

8. Shell Programming and Scripting

Tricky script question

Hi, I'm in the midst of writing a UNIX script that sftp's files to an external host and am stuck with a problem. The problem is that the files created on my server as a order number that correlates to a sequence of directories on the remote host which is where the file should be ftp'ed. ... (3 Replies)
Discussion started by: budrito
3 Replies

9. Shell Programming and Scripting

Tricky Sed

Hello. I am trying to convert occurrences of 'NULL' from a datafile. The 'NULL' occurences appears at this: |NULL| NULL|NULL| NULL|NULL| NULL|NULL| NULL| There should be 52 fields per line. I would like any occurrence of | NULL| or |NULL| to appear as '||' Currently I am using this sed... (2 Replies)
Discussion started by: bestbuyernc
2 Replies

10. Windows & DOS: Issues & Discussions

Tricky one...

Here's my problem: I have a laptop running Windows XP Pro with no internal CD or Floppy drives. I want to install Linux on it. I don't care about the Windows XP Pro installation, in fact I would like to install Linux over the entirety of the HD. However I cannot boot from any external CD drive... (1 Reply)
Discussion started by: saabir
1 Replies
Login or Register to Ask a Question