Read file and remove max length


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Read file and remove max length
# 1  
Old 01-23-2007
Tools Read file and remove max length

Hi all,
I tried to write a shell to read huge file and eliminate max length record which is wrong generated record. But I get an error

Code:
remove_sp.sh: line 27: syntax error near unexpected token `else'
remove_sp.sh: line 27: `        else $LINE >> REJFILE'

My shell is here:
Code:
#!/bin/sh
FILENAME="$1"
TIMEFILE="/home/mediator/INNGPPS/temp/loopfile.out">$TIMEFILE
NEWFILE="/home/mediator/INNGPPS/temp/new0118">$TIMEFILE
REJFILE="/home/mediator/INNGPPS/temp/rej0118">$TIMEFILE
THIS_SCRIPT=$(basename $0)
######################################
function usage
{
echo "\nUSAGE: $THIS_SCRIPT file_to_process\n"
echo "OR - To send the output to a file use: "
echo "\n$THIS_SCRIPT file_to_process > output_file_name 2>&1 \n"
exit 1
}
######################################
function while_read_LINE
{
cat $FILENAME | while read LINE
do
 count=0
 for letter in $LINE
 do
  count=`expr $count + 1`
 done 
 if [ $count < 500 ] then 
  $LINE >> NEWFILE
 	else $LINE >> REJFILE
 fi
done
}
######################################
########### START OF MAIN ############
######################################
# Looking for exactly one parameter
(( $# == 1 )) || usage

# Does the file exist as a regular file?
[[ -f $1 ]] || usage

echo "\nStarting File Processing of each Method\n"
echo "Method 1:"
echo "\nfunction while_read_LINE\n" >> $TIMEFILE
echo "function while_read_LINE"
#time while_read_LINE >> $TIMEFILE
while_read_LINE

# 2  
Old 01-23-2007
$LINE >> NEWFILE
doesn't make sense. You need to do something like:
echo $LINE >> NEWFILE

Also...
if [ $count < 500 ] then
is not going to have the effect you think it will. Replace < with -lt to compare stuff. (If you want to open a file called 500 and make it available for your [ command to read, then you have the right syntax.)
# 3  
Old 01-23-2007
Thank you,
$count variables calculates the length of the line and if its length is fewer than 500 it writes to NEWFILE or to REJFILE. How can I write this

Code:
..
if [ $count < 500 ] then 
  $LINE >> NEWFILE
 	else $LINE >> REJFILE
....

# 4  
Old 01-23-2007
Quote:
Originally Posted by mr_bold
Thank you,
$count variables calculates the length of the line and if its length is fewer than 500 it writes to NEWFILE or to REJFILE. How can I write this

Code:
..
if [ $count < 500 ] then 
  $LINE >> NEWFILE
 	else $LINE >> REJFILE
....

Modify your function like this:
Code:
function while_read_LINE
{
while read LINE; do
 count=0
 count=`echo $line | wc -c` 
 if [ $count -lt 500 ]; then 
     echo $LINE >> NEWFILE
 else 
     echo $LINE >> REJFILE
 fi
done
} < $FILENAME

# 5  
Old 01-23-2007
Code:
awk '{ if (length < 500 ) { print $0 > "file1" }  else { print $0 > "file2" } }'  filename

# 6  
Old 01-24-2007
Quote:
Originally Posted by tayyabq8
Modify your function like this:
Code:
function while_read_LINE
{
while read LINE; do
 count=0
 count=`echo $line | wc -c` 
 if [ $count -lt 500 ]; then 
     echo $LINE >> NEWFILE
 else 
     echo $LINE >> REJFILE
 fi
done
} < $FILENAME


This will put lines with a length of 499 characters in the wrong file.

echo abc | wc -c will have 4 as result and not 3

use a korn or bash shell
typeset -i count
((count=`echo $line | wc -c`-1))
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read File and check records for length

I need a script that will run in unix to: 1) Read and input file with 1 column that contains for ex: 0123456789 1234567890 ...etc 2) Checks the first column if it is: a. Numeric from 0 - 9 b. if it is not less... (4 Replies)
Discussion started by: mrn6430
4 Replies

2. Shell Programming and Scripting

Remove characters from fixed length file

Hello I've question on the requirement I am working on. We are getting a fixed length file with "33" characters long. We are processing that file loading into DB. Now some times we are getting a file with "35" characters long. In this case I have to remove two characters (in 22,23... (14 Replies)
Discussion started by: manasvi24
14 Replies

3. UNIX for Dummies Questions & Answers

Modify the max username length

Hey Any one... Do u know any way I can modify the max username length in unix? I guess it is 32/64 characters by default. Suppose I want to increase it to 128. i hav tried /etc/skel but no use... How can I do that? (2 Replies)
Discussion started by: MayureshRisbud
2 Replies

4. UNIX for Advanced & Expert Users

How to increase max username length?

Hi, This is my first post to this site. So kindly forgive if I am writing in a wrong section. My query is that... I want to modify the max username length size. I guess it is 32/64 on CentOS. Now I want to change it to 128. Is there any way to do that? Thanks in advance!! :) (4 Replies)
Discussion started by: ajay303
4 Replies

5. Shell Programming and Scripting

how to read fixed length flat file....

Hi Gurus, Thanks in advance... I am new to writing shell scripting and help me out reading a flat file with fixed length. I have a fixed length flat file with storename(lenth 6) , emailaddress(lenth 15), location(10). There is NO delimiters in that file. Like the following str00001.txt... (2 Replies)
Discussion started by: willywilly
2 Replies

6. Shell Programming and Scripting

how to grep the max length word form a file?

Hi i have a requirement that is extract the max length word from a file ? plz reply (2 Replies)
Discussion started by: vankireddy
2 Replies

7. Shell Programming and Scripting

How to read max of 10 file at a time?

Hi All, Please advise . Welcome more suggestions. For examples, I have 1000 file with prefix x??? In fact, I want to convert them to x???.txt with max 10 files at a time. As such, I will need to call another script to read from those 10 *txt files and sleep 5000 to convert the next 10 again.... (10 Replies)
Discussion started by: cedrichiu
10 Replies

8. Shell Programming and Scripting

Counting the max length of string

Hi all, I have a flat file of 1000 rows. I want to check the length of the 5th column. The one having the longest length , I want to set it as DEFINED PARAMETER. So later I can check others with that particular number only. Any ideas ?? (2 Replies)
Discussion started by: ganesh123
2 Replies

9. Shell Programming and Scripting

what is the max length of args i can pass in shell?

i have a shell script which takes several args. what is the maximum length of string i can give as argument? (6 Replies)
Discussion started by: senthilk615
6 Replies

10. UNIX for Dummies Questions & Answers

Length of a Unix filepath max length

Hi Guys, Could anyone shed some light on the length of a Unix filepath max length pls ? thanks ! Wilson (3 Replies)
Discussion started by: wilsontan
3 Replies
Login or Register to Ask a Question