Tricky Shell script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Tricky Shell script
# 43  
Old 08-16-2007
CPU & Memory

Perdarabo,

Quote:
Originally Posted by Perderabo
or change the line
exec < data
to
exec < testingdata.txt

or change it to:

exec < $1

and then pass the name of file as an argument

or eliminate the line entirely and redirect the input

Ok I got it,but when i ran the script with the changed name of the input file it gave me an error

./script
./script[82]: item: bad number

The change i did in the script is only this

exec < testingdata

The requiremnt is a bot diffrent,i am mentioning the points for you..

# scan the 27 data items to make sure they are numbers
# and remove leading zeros while we are at it

This req is like this---

1... We are not bothered about the leading zeroes ie 01,02,03-------------- In my requirement they all are valid numbers,we should ignore the zeroes only ie 00 where ever they are,they are just for filling purpose dont have any meaning.

2..# note each column with a non-zero--

here leading zero is fine ie 01,02,03----- they are fine.Here i need to check that the colums must have a valid number and if more than that numbers are there then it should be in ascending order,if not the data is ascending order in column it should print that data as an error.

3. There should be 27 entries in a row that is fine but in that there must be exactly 15 numbers leaving all 00.

4. When i ran the script in debug mode that is ksh -x script
it tells--
ksh -x script
ksh: script: cannot execute

I am using solaris machine.

Your apporoach to the requirement is very correct but just some changes.Rest of the code is fullfilling the requirement.I am new to shell script.

Thanks
Namish
# 44  
Old 08-16-2007
I am using an Linux server, so might be due to this there is a difference.
But anyway.

1. What is the output of

Code:
echo "00,11,21,00,00,55,00,73,83,07,00,22,39,43,00,66,00,00,00,15,00,00,44,58,67,00,87" | sed "s/\(.\{26\}\),\(.\{26\}\),\(.\{26\}\).*/\1\n\2\n\3/g"

if this does not report like
Code:
00,11,21,00,00,55,00,73,83
07,00,22,39,43,00,66,00,00
00,15,00,00,44,58,67,00,87

Then edit the code by removing the line and add the following

Code:
echo $line | cut -d',' -f1-9 > $sortFile
echo $line | cut -d',' -f10-18 >> $sortFile
echo $line | cut -d',' -f19-27 >> $sortFile


2. uncomment the removal of temp files tmpFile1 and check the contents of it.
# 45  
Old 08-16-2007
Lightbulb

I changed the syntax for

echo "00,11,21,00,00,55,00,73,83,07,00,22,39,43,00,66,00,00,00,15,00,00,44,58,67,00,87" | sed "s/\(.\{26\}\),\(.\{26\}\),\(.\{26\}\).*/\1\n\2\n\3/g"

to the one you mentioned now the output is coming.

Lorcan,
I mentioned earlier if i need to check in this script---

if the data not in ascending order in column then it should print that record with some error message or all the columns which are not in ascending order they should be redirected to a diffrent file.
That will be a help.
# 46  
Old 08-16-2007
Quote:
Originally Posted by namishtiwari

1... We are not bothered about the leading zeroes ie 01,02,03----------
If you want my script to work, leave it alone. I know what I am doing. A leading zero tells ksh that a number is octal thus rendering 08 and 09 as non-numbers. Switch you input data to octal rather than decimal or accept the fact that leading zeros must be removed for arithmetic to work properly.

In the script I posted line 82 is "while ((nitem < 9)) ; do" and the only way that could fail is if line 80 "nitem=0" was damaged. But your error message says "item" rather than "nitem". Another one of your "corrections"?

Delete your copy of my script. Copy it again from the website. And this time, leave it alone. You are introducing bugs with your changes.
# 47  
Old 08-17-2007
Quote:
Originally Posted by namishtiwari
Lorcan,
I mentioned earlier if i need to check in this script---

if the data not in ascending order in column then it should print that record with some error message or all the columns which are not in ascending order they should be redirected to a diffrent file.
That will be a help.
Script

Code:
#!/bin/ksh

inpFile=$1
outFile="$HOME/Output.txt"
errFile="$HOME/Error.txt"
tmpFile1="/tmp/sample1.txt"
tmpFile2="/tmp/sample2.txt"
sortFile="/tmp/sortfile.txt"

TOTAL_FILEDS=9
COUNT=1

rm -f $outFile $errFile

for line in $(cat $inpFile)
do
        count_fields=$(echo $line | sed "s/00,*//g;s/,$//g" | wc -c)
        cnt_flds1=$(echo $line | cut -d',' -f1-9 | sed "s/00,*//g;s/,$//g" | wc -c )
        cnt_flds2=$(echo $line | cut -d',' -f10-18 | sed "s/00,*//g;s/,$//g" | wc -c )
        cnt_flds3=$(echo $line | cut -d',' -f19-27 | sed "s/00,*//g;s/,$//g" | wc -c )
        if [[ $count_fields -eq 45 && $cnt_flds1 -eq 15 && $cnt_flds2 -eq 15 && $cnt_flds3 -eq 15 ]]
        then
                PRINT_FLAG=0
                echo $line | cut -d',' -f1-9 > $sortFile
                echo $line | cut -d',' -f10-18 >> $sortFile
                echo $line | cut -d',' -f19-27 >> $sortFile                
                while [[ $COUNT -le $TOTAL_FILEDS ]]
                do
                        cut -d',' -f$COUNT $sortFile | grep -v 00 > $tmpFile1
                        cut -d',' -f$COUNT $sortFile | grep -v 00 | sort > $tmpFile2
                        diff $tmpFile1 $tmpFile2 > /dev/null
                        if [[ $? -eq 1 || ! -s $tmpFile1 ]]
                        then
                                PRINT_FLAG=1
                                break
                        fi
                COUNT=$((COUNT+1))
                done
                if [[ $PRINT_FLAG -eq 0 ]]
                then
                        echo "$line" >> $outFile # Output file 
                else
                        echo "$line : Error in $COUNT field " >> $errFile # Error file with records that does not match
                fi
        else
                echo "$line : Out of Scope " >> $errFile # Error file with records that does not match
        fi
        COUNT=1
done
rm -f $tmpFile1 $tmpFile2 $sortFile
exit 0


Last edited by lorcan; 08-17-2007 at 02:26 AM.. Reason: Added an else loop to redirect as per latest req
# 48  
Old 08-17-2007
Lightbulb

lorcan,


Suppose my input file changes to--

00,11,21,00,00,55,00,73,83,07,00,22,39,43,00,66,00,00,00,15,00,00,44,58,67,00,87
52,14,24,30,00,00,00,70,00,00,00,00,32,47,50,60,74,00,04,16,27,38,00,00,00,00,89
56,10,00,36,40,00,00,00,80,00,12,25,00,00,51,62,72,00,09,00,00,37,49,00,69,00,84,07
99,10,23,33,00,00,63,00,82,02,00,00,34,40,59,00,77,00,06,16,24,00,00,00,66,00,84
00,14,00,30,43,53,00,73,00,04,00,27,00,48,00,67,00,83,00,19,00,35,00,54,00,74,87
08,00,21,00,44,00,65,72,00,09,17,22,32,00,00,00,00,81,00,00,00,37,46,58,68,76,00,87,65,85
00,12,25,00,00,50,61,00,85,01,15,00,31,00,00,00,79,88,00,00,29,00,41,52,62,00,90
60,00,21,34,00,51,61,00,83,08,18,00,00,43,00,00,78,88,00,00,26,36,00,59,62,79,00
88,12,00,38,42,00,00,00,85,00,13,23,00,00,52,60,74,00,06,00,00,39,44,00,67,00,87,86
03,00,00,00,41,56,00,73,84,04,17,22,32,00,00,00,76,00,00,19,29,00,49,00,64,00,86
00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,87

This file is having wrong data like
--
1. We have more than 27 entries in a line.That is a simple validation i can do that easily.
2. In a row that is consisting of 3rows and 9 columns,the column values are greater than the respective one.
3. The input file may have so many zeroes.
4. There can be duplicay of the rows.
5. If a row is having after 27 entries like-
03,00,00,00,41,56,00,73,84,04,17,22,32,00,00,00,76,00,00,19,29,00,49,00,64,00,86,00,00,00
Then also it should be redirected to the err file.


We need to do validations for all this and if the error is coming redirect to a file.Your script is working perfectly fine if the column value is greater than the respective one.
If you have any doubts regarding i can help you in clarifyng them.
Thanks.
# 49  
Old 08-17-2007
Try the code again, i have added an else part so that it the lines that are more than 27 entries are also redirected to error file
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing xml - tricky one...need help!!

Hi, i am new to linux programming fraternity but looks like starting with a big thing... yes..xml parsing (it is indeed tough for a beginner like me) so need your kind help... The snippet of xml looks like: <snapshot> <tag1> <key>1234</key> <keytype>abcd</keytype> </tag1> <tag2>... (11 Replies)
Discussion started by: rookie2014
11 Replies

2. Shell Programming and Scripting

Tricky sed required

Hi All I need to put some sed together for a task and its a bit advanced for me, so I thought I'd ask if anyone here could help. I have a csv file with content like this - "","abcde","","" "'","abcde","","" "","","","1234" "'e'","","","" I need to remove any single quotes that fall... (17 Replies)
Discussion started by: steadyonabix
17 Replies

3. Solaris

Tricky egrep

Hi folks! My first post here. I'm working on a script that retrieves a range of files from a list depending on a range of time. UPDATE: I've seen it could be difficult to read all this thing, so I'll make a summarize it.. How come I do this and take a result.. grep "..\:.." lista.new |... (4 Replies)
Discussion started by: kl0x
4 Replies

4. Shell Programming and Scripting

Tricky data manipulation...

Hi everyone.. I am new here, hello.. I hope this doesn't come across to you folks as a stupid question, I'm somewhat new to scripting :) I'm seeking some help in finding a way to manipulate data output for every two characters - example: numbers.lst contains the following output:... (3 Replies)
Discussion started by: explicit
3 Replies

5. Shell Programming and Scripting

Tricky - Need help on Shell script variables

Hi, I have a requirement in which i have to read a csv file and put data in certain set of variables: File content: VP-DTL-REC-CNT, ,854840,0.00,VP-PAID-AMT, ,0,32280885.17,VP-PAT-PAID-AMT, ,0,9930244.32,VP-PAID-REV-CNT, ,484927,0.00,VP-REJ-CNT, ,369913,0.00, , ,0,0.00, , ,0,0.00, , ,0,0.00, ,... (3 Replies)
Discussion started by: shantoshkumar
3 Replies

6. Shell Programming and Scripting

Linux: Writing a tricky script to check connectivity

So, first and foremost, I'm having issues with my internet connection. Periodically, the connection drops across the network. The fix is simple enough: restart the modem. However, this gets old when the connection dies out every hour. I can hit my surfboard on 192.168.100.1, and navigate to a... (5 Replies)
Discussion started by: kungfujoe
5 Replies

7. UNIX for Dummies Questions & Answers

Tricky Quotation Question

Hi, I am at a point in my script where I defined the number of the command line parameter I would like to set a variable equal to: parameter_number=14 I would then like to set a variable equal to the correct parameter: variable=$parameter_number The issue here is that {} is required... (2 Replies)
Discussion started by: msb65
2 Replies

8. Shell Programming and Scripting

Tricky script question

Hi, I'm in the midst of writing a UNIX script that sftp's files to an external host and am stuck with a problem. The problem is that the files created on my server as a order number that correlates to a sequence of directories on the remote host which is where the file should be ftp'ed. ... (3 Replies)
Discussion started by: budrito
3 Replies

9. Shell Programming and Scripting

Tricky Sed

Hello. I am trying to convert occurrences of 'NULL' from a datafile. The 'NULL' occurences appears at this: |NULL| NULL|NULL| NULL|NULL| NULL|NULL| NULL| There should be 52 fields per line. I would like any occurrence of | NULL| or |NULL| to appear as '||' Currently I am using this sed... (2 Replies)
Discussion started by: bestbuyernc
2 Replies

10. Windows & DOS: Issues & Discussions

Tricky one...

Here's my problem: I have a laptop running Windows XP Pro with no internal CD or Floppy drives. I want to install Linux on it. I don't care about the Windows XP Pro installation, in fact I would like to install Linux over the entirety of the HD. However I cannot boot from any external CD drive... (1 Reply)
Discussion started by: saabir
1 Replies
Login or Register to Ask a Question