12 Ways to Parse a file


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Answers to Frequently Asked Questions Tips and Tutorials 12 Ways to Parse a file
# 1  
Old 04-06-2005
12 Ways to Parse a file

A common thing in shell scripting. I came across this script that will be useful for people learning to write script.

Code:
#!/usr/bin/ksh
#
# SCRIPT: 12_ways_to_parse.ksh.ksh
#
#
# REV: 1.2.A
#
# PURPOSE:  This script shows the different ways of reading
#       a file line by line.  Again there is not just one way
#       to read a file line by line and some are faster than
#       others and some are more intuitive than others.
#
# REV LIST:
#
#       03/15/2002 - Randy Michael
#       Set each of the while loops up as functions and the timing
#       of each function to see which one is the fastest.
#
#######################################################################
#
#       NOTE: To output the timing to a file use the following syntax:
#
#          12_ways_to_parse.ksh file_to_process  > output_file_name 2>&1
#
#       The actaul timing data is sent to standard error, file 
#       descriptor (2), and the function name header is sent
#       to standard output, file descriptor (1).
#
#######################################################################
#
# set -n  # Uncomment to check command syntax without any execution
# set -x  # Uncomment to debug this script
#

FILENAME="$1"
TIMEFILE="/tmp/loopfile.out"
>$TIMEFILE
THIS_SCRIPT=$(basename $0)

######################################
function usage
{
echo "\nUSAGE: $THIS_SCRIPT  file_to_process\n"
echo "OR - To send the output to a file use: "
echo "\n$THIS_SCRIPT  file_to_process  > output_file_name 2>&1 \n"
exit 1
}
######################################
function while_read_LINE
{
cat $FILENAME | while read LINE
do
        echo "$LINE"
        :
done
}
######################################
function while_read_LINE_bottom 
{
while read LINE
do
        echo "$LINE"
        :

done < $FILENAME
}
######################################
function while_line_LINE_bottom
{
while line LINE
do
        echo $LINE
        :
done < $FILENAME
}
######################################
function cat_while_LINE_line  
{
cat $FILENAME | while LINE=`line`
do
        echo "$LINE"
        :
done
}
######################################
function while_line_LINE
{
cat $FILENAME | while line LINE
do
        echo "$LINE"
        :
done
}
######################################
function while_LINE_line_bottom 
{
while LINE=`line`
do
        echo "$LINE"
        :

done < $FILENAME 
}
######################################
function while_LINE_line_cmdsub2 
{
cat $FILENAME | while LINE=$(line)
do
        echo "$LINE"
        :
done
}
######################################
function while_LINE_line_bottom_cmdsub2 
{
while LINE=$(line)
do
        echo "$LINE"
        :

done < $FILENAME 
}
######################################
function while_read_LINE_FD 
{
exec 3<&0
exec 0< $FILENAME
while read LINE
do
        echo "$LINE"
        :
done
exec 0<&3
}
######################################
function while_LINE_line_FD 
{
exec 3<&0
exec 0< $FILENAME
while LINE=`line`
do
        echo "$LINE"
        :
done
exec 0<&3
}
######################################
function while_LINE_line_cmdsub2_FD
{
exec 3<&0
exec 0< $FILENAME
while LINE=$(line)
do
        print "$LINE"
        :
done
exec 0<&3
}
######################################
function while_line_LINE_FD 
{
exec 3<&0
exec 0< $FILENAME

while line LINE
do
        echo "$LINE"
        :
done

exec 0<&3
}
######################################
########### START OF MAIN ############
######################################

# Test the Input

# Looking for exactly one parameter
(( $# == 1 )) || usage

# Does the file exist as a regular file?
[[ -f $1 ]] || usage

echo "\nStarting File Processing of each Method\n"

echo "Method 1:"
echo "\nfunction while_read_LINE\n" >> $TIMEFILE
echo "function while_read_LINE"
time while_read_LINE >> $TIMEFILE
echo "\nMethod 2:"
echo "\nfunction while_read_LINE_bottom\n" >> $TIMEFILE
echo "function while_read_LINE_bottom"
time while_read_LINE_bottom >> $TIMEFILE
echo "\nMethod 3:"
echo "\nfunction while_line_LINE_bottom\n" >> $TIMEFILE
echo "function while_line_LINE_bottom"
time while_line_LINE_bottom >> $TIMEFILE
echo "\nMethod 4:"
echo "\nfunction cat_while_LINE_line\n" >> $TIMEFILE
echo "function cat_while_LINE_line"
time cat_while_LINE_line >> $TIMEFILE
echo "\nMethod 5:"
echo "\nfunction while_line_LINE\n" >> $TIMEFILE
echo "function while_line_LINE"
time while_line_LINE >> $TIMEFILE
echo "\nMethod 6:"
echo "\nfunction while_LINE_line_bottom\n" >> $TIMEFILE
echo "function while_LINE_line_bottom"
time while_LINE_line_bottom >> $TIMEFILE
echo "\nMethod 7:"
echo "\nfunction while_LINE_line_cmdsub2\n" >> $TIMEFILE
echo "function while_LINE_line_cmdsub2"
time while_LINE_line_cmdsub2 >> $TIMEFILE
echo "\nMethod 8:"
echo "\nfunction while_LINE_line_bottom_cmdsub2\n" >> $TIMEFILE
echo "function while_LINE_line_bottom_cmdsub2"
time while_LINE_line_bottom_cmdsub2 >> $TIMEFILE
echo "\nMethod 9:"
echo "\nfunction while_read_LINE_FD\n" >> $TIMEFILE
echo "function while_read_LINE_FD"
time while_read_LINE_FD >> $TIMEFILE
echo "\nMethod 10:"
echo "\nfunction while_LINE_line_FD\n" >> $TIMEFILE
echo "function while_LINE_line_FD"
time while_LINE_line_FD >> $TIMEFILE
echo "\nMethod 11:"
echo "\nfunction while_LINE_line_cmdsub2_FD\n" >> $TIMEFILE
echo "function while_LINE_line_cmdsub2_FD"
time while_LINE_line_cmdsub2_FD >> $TIMEFILE
echo "\nMethod 12:"
echo "\nfunction while_line_LINE_FD\n" >> $TIMEFILE
echo "function while_line_LINE_FD"
time while_line_LINE_FD >> $TIMEFILE

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Different ways to check a file

to get the checksum of a file on unix systems, you can just use the "cksum" command. i discovered there are some watered down versions of unix systems i have to do some work on. surprisingly, these systems have perl installed on them and awk. so if the cksum command is not available on a... (1 Reply)
Discussion started by: SkySmart
1 Replies

2. Shell Programming and Scripting

How to parse a file?

Hi guys I have a very long file which looks like this: y893 89:send prctmgr exit 106:bas_services_dwn -------------------------------------------------- y895 90:send prctmgr exit 106:bas_services_dwn -------------------------------------------------- y897 90:send prctmgr exit... (3 Replies)
Discussion started by: aoussenko
3 Replies

3. Shell Programming and Scripting

parse fasta file to tabular file

Hello, A bioperl problem I thought could be done with awk: convert the fasta format (Note: the length of each row is not the same for each entry as they were combined from different files!) to tabular format. input.fasta: >YAL069W-1.334 Putative promoter sequence... (6 Replies)
Discussion started by: yifangt
6 Replies

4. Shell Programming and Scripting

Parse a file

FILE1 2917,065A,RDF1+TDEV,2917_3RAID5,05E:0_10E:0,BL_lmwsp02,0345,xxx,3452(DR) 2917,03EA,RDF1+TDEV,2917_3RAID5,03E:0_12E:0,BL_tv00p02,0455,xxx,3ee4(DR) 2917,03EB,RDF1+TDEV,2917_3RAID5,03E:0_12E:0,BL_tv00p02,0345,xxx,2d34(DR)... (7 Replies)
Discussion started by: greycells
7 Replies

5. Shell Programming and Scripting

need to parse the jil file into an excel file

Hi I have the following as input /* ----------------- backupJIL ----------------- */ insert_job: backupJIL job_type: c command: autorep -J ALL -q > /home/autosys/...p/autosys_jil_bk machine: machine owner: autosys@machine permission: gx,ge,wx,we date_conditions: 1 days_of_week:... (7 Replies)
Discussion started by: ramky79
7 Replies

6. Shell Programming and Scripting

Parse file from remote server to calculate count of string existence in that file

Hi I need to parse the file of same name which exist on different servers and calculate the count of string existed in both files. Say a file abc.log exist on 2 servers. I want to search for string "test" on both files and calculate the total count of search string's existence. For... (6 Replies)
Discussion started by: poweroflinux
6 Replies

7. UNIX for Dummies Questions & Answers

ways to open a file

How many ways are there to open a file in unix? wha i know is mentioned below suppose if xx is a filename, then 1) cat xx 2) more xx 3) view xx 4) less xx I heard there are seven ways to open a file. If so, can anyone tell me all the ways to open a file and correct me the above... (3 Replies)
Discussion started by: venkatesht
3 Replies

8. UNIX for Advanced & Expert Users

How to parse through a file and based on condition form another output file

I have one file say CM.txt which contains values like below.Its just a flat file 1000,A,X 1001,B,Y 1002,B,Z ... .. total around 4 million lines of entries will be in that file. Now i need to write another file CM1.txt which should have 1000,1 1001,2 1002,3 .... ... .. Here i... (6 Replies)
Discussion started by: sivasu.india
6 Replies

9. Shell Programming and Scripting

Need help to parse the file

# Start "ABC" SFFd 0 4 Time SFFT 4 8 {Sec} User SFFTimeVal 12 8 {Sec} # Start "CP" SFFT ... (3 Replies)
Discussion started by: navsharan
3 Replies

10. Shell Programming and Scripting

Parse file

Hi Friends, I have a file in the format shown (Name followed by address:) I need only the address part please see the output. I have tried using nawk but I am not getting the desired output. SAM ADDRS 64874 FRANKLYN DR IRVINE TX - 74394; 538 FRED ASSOCIATES PETER ADDRS 84734... (5 Replies)
Discussion started by: sbasetty
5 Replies
Login or Register to Ask a Question