The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Optimizing OSX deiphon OS X (Apple) 2 09-20-2008 02:20 AM
Optimizing Performance in a CEP System iBot Complex Event Processing RSS News 0 09-13-2008 02:40 PM
Optimizing query matrixmadhan UNIX and Linux Applications 15 09-04-2007 02:00 AM
Optimizing for a Speed-up switch Shell Programming and Scripting 3 04-07-2006 09:52 PM
optimizing disk performance J.P Filesystems, Disks and Memory 4 03-01-2005 12:32 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 09-20-2008
jolecanard jolecanard is offline
Registered User
  
 

Join Date: Sep 2008
Posts: 36
Optimizing sed command

Hi,

I would like to know if there is a command faster then sed, or a way to optimize my code...

Here is the description of my problem:
I have a file "file1" that is composed of 10 columns. The first one contain a list of ID, the second a list of number associated to the firs list of ID. The third one contains ID and the fourth the number associated, etc...

So I have:

Code:
ID  numb  ID  numb  ID   numb   ID   numb   ID   numb
ID  numb  ID  numb  ID   numb   ID   numb   ID   numb
...
...

The total number of ID is around 8000.

I want to find all the ID starting by a given string (eg. "460...") and extract the associated number. To do that I am simply using a grep command associated with cut and it is going pretty well.

Then I am doing an operation on the numbers extracted (MATLAB) and then I need to put the new values back in place in the file with 10 columns. To do that I am using:

Code:
x=file1
count=1
for k in $listID
do
  VALUE=`head -$count newval | tail -1`
  sed 's/'$k'  .\.....E.../'$k'  '$VALUE'/g' $x > extps
  \mv extps $x
  count=`expr $count + 1`
done

Where "newval" is the list of the new values computed, "listID" is the list of all the ID for which I need to change the associated number and "file1" is hte file with the 10 columns.

This code is working perfectly, but because I have a bunch of ID to change the repeted use of sed is taking A LOT of time (among 8000 ID I need to change about 500).

Does anybody know how could I speed up this code?

Thanks
  #2 (permalink)  
Old 09-21-2008
Franklin52 Franklin52 is offline Forum Staff  
Moderator
  
 

Join Date: Feb 2007
Posts: 4,342
This should be faster. Assuming the fields are separated by 2 spaces and you have the listID's separated with a space like something as below:


Code:
#!/bin/sh

listID="123 789"
VALUE="your_new_value"

awk -v l="$listID" -v var="$VALUE" '
BEGIN{FS=OFS="  ";split(l,arr," ");for(i in arr){a[arr[i]]=arr[i]}}
{for(i=1;i<10;i+=2)if($i in a){$(i+1)=var}}
{print}' file1

If you have the listID's line by line in a file (should be a better choice):


Code:
#!/bin/sh

VALUE="your_new_value"

awk -v var="$VALUE" 'BEGIN{FS=OFS="  "}
NR==FNR{a[$0]=$0; next}
{for(i=1;i<10;i+=2)if($i in a){$(i+1)=var}}
{print}' file_of_listIDs file1

If the output is correct you can redirect the output to a temporary file and replace file1 with the temporary file.

Use nawk, gawk or /usr/xpg4/bin/awk on Solaris.

Regards
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 12:44 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0