Sponsored Content
Full Discussion: grep or awk?
Top Forums UNIX for Dummies Questions & Answers grep or awk? Post 1394 by on Friday 2nd of March 2001 03:42:53 PM
Old 03-02-2001
Here's my problem:
1. I have a file that contains say for this example, three records, each twenty bytes long:
CustNum CustName
1111111111abcdefghij
2222222222abcdefghij
3333333333abcdefghij

2. I have two other very large files (over 500,000 records) one is 500 bytes, the other is 200 bytes long. These two files contain the CustNum from the first file as well as MANY more that I don't want.

3. I want to extract out the CustNum from the first file, then do a loop thru the other two files, matching the CustNum and only writting out those matching records all 500 or 200 bytes.
Essentially, I want to reduce the 100,000 record file to a managable amount.

I have tried cut to extract the records out to a variable, then loop with grep, but the results produce a file of one continuous record. No newline? Should I use awk instead?

Any help would be appreciated......
Here is my code:

#! /bin/ksh
# get_bids_autorenew.sh

#== Local Variables ==#
datadir="/ias/users/app4dxh/data"
driver="$datadir/driver.dat"
file_1="tcpcsm2.data"
file_2="tcpvrm2.data"
num=0
stat=0



Get_Bids()
{
cd $biddir

if [ -s $driver ]
then
bids=`cut -c1-10 $driver`
stat=$?
else
stat=$?
echo "Function: $0 - No data found in $driver or file does not exist."
echo "Aborting script with a status of $stat"
exit 1
}



Match_Files()
{
#== For each bid picked up, check each of the CP and ==#
#== SM files for a match and just write those records. ==#

for i in $bids
do
match=`grep -s $i $file`
stat=$?
while :
do
case "$stat" in
0) echo $match >> file_$num.new
echo Status is $stat
break;;
1) break;;
2) echo "Function: $0 - The file $file is not accessible - grep status is $stat"
break;;
esac
done
done

}

#==Main==#
Get_Bids

#== For each File, execute the Match_Files function ==#
cd $datadir
for data in "$file_1" "$file_2"
do
set $data
file=$1
let num="$num + 1"
Match_Files
done
exit 0


 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[grep awk cut] > awk

Hi, I'm very new to scripting. grep $s $filename | awk '{print $2}' | cut -c 1-8 How can I optimize this using a single awk? I tried: awk '/$s/ {print $2}' $filename | cut -c 1-8 However didn't work, I think the awk is not recognizing $s and the verbal is something else. (6 Replies)
Discussion started by: firdousamir
6 Replies

2. UNIX for Dummies Questions & Answers

grep and awk

I have grep MHz psrinfo-v.out it gives The i386 processor operates at 3000 MHz, The i386 processor operates at 3000 MHz, The i386 processor operates at 3000 MHz, The i386 processor operates at 3000 MHz, how to get instead of these 4 lines: CPU speed: 3000 MHz i.e. CPU... (3 Replies)
Discussion started by: melanie_pfefer
3 Replies

3. Shell Programming and Scripting

MEM=`ps v $PPID| grep -i db2 | grep -v grep| awk '{ if ( $7 ~ " " ) { print 0 } else

Hi Guys, I need to set the value of $7 to zero in case $7 is NULL. I've tried the below command but doesn't work. Any ideas. thanks guys. MEM=`ps v $PPID| grep -i db2 | grep -v grep| awk '{ if ( $7 ~ " " ) { print 0 } else { print $7}}' ` Harby. (4 Replies)
Discussion started by: hariza
4 Replies

4. Shell Programming and Scripting

Is it better to grep and pipe to awk, or to seach with awk itself

This may just be a lack of experience talking, but I always assumed that when possible it was better to use a commands built in abilities rather than to pipe to a bunch of commands. I wrote a (very simple) script a while back that was meant to pull out a certain error code, and report back what... (4 Replies)
Discussion started by: DeCoTwc
4 Replies

5. Shell Programming and Scripting

Read content between xml tags with awk, grep, awk or what ever...

Hello, I trying to extract text that is surrounded by xml-tags. I tried this cat tst.xml | egrep "<SERVER>.*</SERVER>" |sed -e "s/<SERVER>\(.*\)<\/SERVER>/\1/"|tr "|" " " which works perfect, if the start-tag and the end-tag are in the same line, e.g.: <tag1>Hello Linux-Users</tag1> ... (5 Replies)
Discussion started by: Sebi0815
5 Replies

6. Shell Programming and Scripting

AWK/GREP: grep only lines starting with integer

I have an input file 12.4 1.72849432773174e+01 -7.74784188610632e+01 12.5 9.59432114416327e-01 -7.87018212757537e+01 15.6 5.20139995965960e-01 -5.61612429666624e+01 29.3 3.76696387248366e+00 -7.42896194101892e+01 32.1 1.86899877018077e+01 -7.56508762501408e+01 35 6.98857157014640e+00... (2 Replies)
Discussion started by: chrisjorg
2 Replies

7. Shell Programming and Scripting

awk / grep

how do I change this line to use the awk command RC19=`grep -c "Broken pipe" $FTP_OUT` Code tags please (5 Replies)
Discussion started by: mbmarciniak
5 Replies

8. UNIX for Dummies Questions & Answers

Piping grep into awk, read the next line using grep

Hi, I have a number of files containing the information below. """"" Fundallinfo 6.3950 14.9715 14.0482 """"" I would like to grep for Fundallinfo and use it to read the next line? I ideally would like to read the three numbers that follow in the next line and... (2 Replies)
Discussion started by: Paul Moghadam
2 Replies

9. Shell Programming and Scripting

sed and awk usage to grep a pattern 1 and with reference to this grep a pattern 2 and pattern 3

Hi , I have a file where i have modifed certain things compared to original file . The difference of the original file and modified file is as follows. # diff mir_lex.c.modified mir_lex.c.orig 3209c3209 < if(yy_current_buffer -> yy_is_our_buffer == 0) { --- >... (5 Replies)
Discussion started by: breezevinay
5 Replies

10. UNIX for Beginners Questions & Answers

Using awk instead of grep -f

Hi Guys. I am trying to count occurances of patterns(occurance can be anywhere in file2) from file1 in file2. file1 is god god pod rod file2 is iamgod iamgod podrod 123rod456 output should be god 2 god 2 pod 1 rod 2 I am not good at awk but i figured out this command.it doesnt... (7 Replies)
Discussion started by: ahfze
7 Replies
FILETYPE(3)								 1							       FILETYPE(3)

filetype - Gets file type

SYNOPSIS
string filetype (string $filename) DESCRIPTION
Returns the type of the given file. PARAMETERS
o $filename - Path to the file. RETURN VALUES
Returns the type of the file. Possible values are fifo, char, dir, block, link, file, socket and unknown. Returns FALSE if an error occurs. filetype(3) will also produce an E_NOTICE message if the stat call fails or if the file type is unknown. EXAMPLES
Example #1 filetype(3) example <?php echo filetype('/etc/passwd'); // file echo filetype('/etc/'); // dir ?> ERRORS
/EXCEPTIONS Upon failure, an E_WARNING is emitted. NOTES
Note The results of this function are cached. See clearstatcache(3) for more details. Tip As of PHP 5.0.0, this function can also be used with some URL wrappers. Refer to "Supported Protocols and Wrappers" to determine which wrappers support stat(3) family of functionality. SEE ALSO
is_dir(3), is_file(3), is_link(3), file_exists(3), mime_content_type(3), pathinfo(3), stat(3). PHP Documentation Group FILETYPE(3)
All times are GMT -4. The time now is 04:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy