Is there a way to make this more efficient


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Is there a way to make this more efficient
# 1  
Old 08-11-2008
Is there a way to make this more efficient

I have the following code.

printf "Test Message Report" > report.txt
while read line
do
msgid=$(printf "%n" "$line" | cut -c1-6000| sed -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*ex:Msg\(.*\)ex:Msg.*|\1|')
putdate=$(printf "%n" "$line" | cut -c1-6000| sed -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*PutDate\(.*\)PutTime.*|\1|')
puttime=$(printf "%n" "$line" | cut -c1-6000| sed -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*PutTime\(.*\)ApplOrgin.*|\1|')
timestamp=$(printf "%n" "$line" | cut -c1-6000| sed -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*CreationTimeStamp\(.*\)CreationTimeStamp.*|\1|')
printf "MsgId = $msgid" >> report.txt
printf "Put Date = $putdate" >> report.txt
printf "Put Time = $puttime" >> report.txt
printf "Timestamp = $timestamp" >> report.txt
done < temp01

Basically we are producing a report from the messages taken off an MQ Queue. The messages have some characters that throws sed whereby it sometimes extracts the data I am after and sometimes not. I got around this problem by cutting the portion of the data that contains the information I am after from each record, which equates to a message taken off the queue, and passing it through the first sed command (sed -e 's/[^a-zA-Z0-9+_:-]//g' ) to filter the characters of interest. Subsequently the data is put through the second sed command to extarct the required information. From each record I require seven different bits of information, I have shown four. However, the process is repetitive for the other three.

Is there a way of improving this piece of code? I wanted to save the reformatted (printf "%n" "$line" | cut -c1-7000| sed -e 's/[^a-zA-Z0-9+_:-]//g' ) portion of the data into a variable and then pass it to sed, but sed only accepts a file as an input. This would get rid off the repetitive code. Please note, I can not use perl.
# 2  
Old 08-11-2008
Quote:
Originally Posted by gugs
Please note, I can not use perl.
Because the teacher will give you an F? Why? Even if you're not root, you can always install Perl in your home directory.

Why is it important to do the cut?

Can you provide some sample input?
# 3  
Old 08-11-2008
Code:
#  echo "sed only accepts a file as input" | sed 's/ only\(.* a \).*\(as .*\)/\1stream \2/g' 
sed accepts a stream as input

# 4  
Old 08-11-2008
What I ment was a variable....

I was thinking of passing sed a variable, for example:

while read line
do
record=$(printf "%n" "$line" | cut -c1-6000| sed -e 's/[^a-zA-Z0-9+_:-]//g' )
puttime=$(sed<$record 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*PutTime\(.*\)ApplOrgin.*|\1|')
done < temp01

I get a message stating it "cannot locate directory or file". I have tried passing ${record} too.
# 5  
Old 08-11-2008
Code:
#  record="sed only accepts a file as input"

#  echo $record |sed 's/ only\(.* a \).*\(as .*\)/\1stream \2/g'
sed accepts a stream as input

# 6  
Old 08-11-2008
Code:
while read line
do
   record=$(printf "%n" "$line" | cut -c1-6000| sed -e 's/[^a-zA-Z0-9+_:-]//g' )
   puttime=$(echo "$record" | sed   's|.*PutTime\(.*\)ApplOrgin.*|\1|')
done < temp01

Jean-Pierre.
# 7  
Old 08-11-2008
Quote:
Originally Posted by Tytalus
Code:
#  record="sed only accepts a file as input"

Depends on your shell
Code:
#!/bin/bash
a=test
sed 's/t/b/' <<< $a


@OP
you should provide a sample input.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combining awk command to make it more efficient

VARIABLE="jhovan 5259 5241 0 20:11 ? 00:00:00 /proc/self/exe --type=gpu-process --channel=5182.0.1597089149 --supports-dual-gpus=false --gpu-driver-bug-workarounds=2,45,57 --disable-accelerated-video-decode --gpu-vendor-id=0x80ee --gpu-device-id=0xbeef --gpu-driver-vendor... (3 Replies)
Discussion started by: SkySmart
3 Replies

2. Programming

Help with make this Fortran code more efficient (in HPC manner)

Hi there, I had run into some fortran code to modify. Obviously, it was written without thinking of high performance computing and not parallelized... Now I would like to make the code "on track" and parallel. After a whole afternoon thinking, I still cannot find where to start. Can any one... (3 Replies)
Discussion started by: P_E_M_Lee
3 Replies

3. UNIX for Advanced & Expert Users

Efficient way to grep

Hi Experts, I've been trying simple grep to search for a string in a huge number of files in a directory. grep <pattern> * this gives the search results as well as the following - grep: <filename>: Permission denied grep: <filename>: Permission denied for files which I don't have... (4 Replies)
Discussion started by: sumoka
4 Replies

4. Emergency UNIX and Linux Support

Help to make awk script more efficient for large files

Hello, Error awk: Internal software error in the tostring function on TS1101?05044400?.0085498227?0?.0011041461?.0034752266?.00397045?0?0?0?0?0?0?11/02/10?09/23/10???10?no??0??no?sct_det3_10_20110516_143936.txt What it is It is a unix shell script that contains an awk program as well as... (4 Replies)
Discussion started by: script_op2a
4 Replies

5. UNIX for Dummies Questions & Answers

Is this regex efficient?

I want to match the red portion: 9784323456787-Unknown Phrase with punctuation "Some other PhrASE." Is this the best regex to match this? '978\{10\}-*' (4 Replies)
Discussion started by: glev2005
4 Replies

6. UNIX for Advanced & Expert Users

efficient repace

some of the data i receive has been typed in manually due to which there are often places where i find 8 instead of ( and the incorrect use of case what according to you is the best way to correct such data. The data has around 20,000 records. The value i want to change is in the 4th field.... (2 Replies)
Discussion started by: VGR
2 Replies

7. Shell Programming and Scripting

help on most efficient search

Hello, We have a directory with 15 sub-directories where each sub-directory contains 1.5 to 2 lakhs of files in it. Daily, around 300-500 files will be uploaded to each sub-directory. Now, i need to get the list of files received today in most efficient way. I tried using "find with newer... (16 Replies)
Discussion started by: prvnrk
16 Replies

8. Shell Programming and Scripting

Is there a more efficient way?

I'm using korn shell to connect to oracle, retrieve certain values, put them in a list, and iterate through them. While this method works, I can't help but think there is an easier method. If you know of one, please suggest a shorter, more efficient method. ############### FUNCTIONS ... (6 Replies)
Discussion started by: SelectSplat
6 Replies

9. UNIX for Advanced & Expert Users

Efficient Dispatching

Does anyone know what's new with Efficient dispatching in the Solaris 2.8 release (vs Solaris 2.6) release? Specifically, does anyone know of a good website to get detailed information on thread dispatching using efficient dispatching in solaris 2.8? Thank you. (1 Reply)
Discussion started by: uchachra
1 Replies
Login or Register to Ask a Question