Sponsored Content
Top Forums Shell Programming and Scripting speeding up bash script with "while read line" Post 302526459 by origamisven on Tuesday 31st of May 2011 02:31:28 PM
Old 05-31-2011
speeding up bash script with "while read line"

Hello everybody,
I'm still slowly treading my way into bash scripting (without any prior programming experience) and hence my code is mostly what some might call "creative" if they meant well Smilie

I have created a script that serves its purpose but it does so very slowly, since it needs to work its way through ~1million lines of text file input and does so with a "while read line" loop, which slows down the process terribly.

If it's in any way possible I'd like to speed up the script and would appreciate any suggestions you may have.
Code:
x=0
MOLprev=0
while read line
do
    MOLcur=$(echo $line | awk '{print $5}')    
    if [[ "$MOLprev" -eq 9999 && "$MOLprev" -ne "$MOLcur" ]]
    then
        x=$((x+1))
        SCALE=$((x*10000))
    fi
    echo $line | awk -v var1=`echo $MOLprev` -v var2=`echo $SCALE` '{ 
        if (var1 -ge $5) 
            printf("%4s%7s%5s%4s%6s%12s%8s%8s\n",$1,$2,$3,$4,$5+var2,$6,$7,$8)
        else 
            printf("%4s%7s%5s%4s%6s%12s%8s%8s\n",$1,$2,$3,$4,$5,$6,$7,$8); 
        }'
    MOLprev=$MOLcur
done < atoms_done > $OUTPUT

It is supposed to work through a text file (.pdb - protein data bank file) called "atoms_done" in column 5, which consists of integer values that represent a molecule number. There are several molecule types (column 4) in the file (see example below).

The molecule number (column 4) increases until the value 9999 is reached, after which the number jumps back to 0 and is incremented again until 9999. I want to rewrite or edit the existing file and correct just that and have the numbering increase above 9999 until the end of the file is reached.

ATOM 11393 OW SOL 1997 10.570 25.370 66.140
ATOM 11394 HW1 SOL 1997 10.990 24.510 65.850
ATOM 11395 HW2 SOL 1997 11.260 25.970 66.540
ATOM 11396 OW SOL 1998 26.270 16.020 58.330
ATOM 11397 HW1 SOL 1998 27.210 16.140 58.670
ATOM 11398 HW2 SOL 1998 25.800 16.900 58.370
ATOM 11399 OW SOL 1999 7.760 28.120 61.090
ATOM 11400 HW1 SOL 1999 6.970 28.740 61.090
ATOM 11401 HW2 SOL 1999 8.260 28.210 61.950
ATOM 11402 OW SOL 2000 36.170 4.250 62.330
ATOM 11403 HW1 SOL 2000 35.280 3.810 62.490
ATOM 11404 HW2 SOL 2000 36.030 5.100 61.830
ATOM 11405 C1 MeO 2001 19.100 14.520 124.300
ATOM 11406 O1 MeO 2001 19.850 14.620 123.120
ATOM 11407 HO1 MeO 2001 19.520 15.360 122.630
ATOM 11408 HC1 MeO 2001 18.190 13.930 124.210
ATOM 11409 HC2 MeO 2001 19.740 14.210 125.120
ATOM 11410 HC3 MeO 2001 18.730 15.500 124.600
ATOM 11411 C1 MeO 2002 19.280 3.410 94.800
ATOM 11412 O1 MeO 2002 20.380 3.410 95.710
ATOM 11413 HO1 MeO 2002 21.020 3.970 95.290
ATOM 11414 HC1 MeO 2002 18.320 3.220 95.290


Thank you for any helpful comments and suggestions.

Cheers.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies

2. Shell Programming and Scripting

script to read a line with spaces bet " " and write to a file

Hi, I need a command in UNIX KSH below is the description... MAPPING DESCRIPTION ="Test Mapping for the calid inputs" ISVALID ="YES" NAME ="m_test_xml" OBJECTVERSION ="1" VERSIONNUMBER ="1" unix ksh command to read the DESCRIPTION and write to a file Test Mapping for the calid inputs... (3 Replies)
Discussion started by: perlamohan
3 Replies

3. Shell Programming and Scripting

read -p "prompt text" foo say "read: bad option(s)" in Bourne-Shell

Hallo, i need a Prompting read in my script: read -p "Enter your command: " command But i always get this Error: -p: is not an identifier When I run these in c-shell i get this error /usr/bin/read: read: bad option(s) How can I use a Prompt in the read command? (9 Replies)
Discussion started by: wiseguy
9 Replies

4. Shell Programming and Scripting

read -n1 -r -p "Press..." key / produces error in bash shell script

Hello! Sorry, for my not so perfect english! I want to stop bash shell script execution until any key is pressed. This line in a bash shell script read -n1 -r -p "Press any key to continue..." key produces this error When I run this from the command line usera@lynx:~$ read... (4 Replies)
Discussion started by: linuxinho
4 Replies

5. Shell Programming and Scripting

Simplify Bash Script Using "sed" Or "awk"

Input file: 2 aux003.net3.com error12 6 awn0117.net1.com error13 84 aux008 error14 29 aux001.ha.ux.isd.com error12 209 aux002.vm.ux.isd.com error34 21 alx0027.vm.net2.com error12 227 dux001.net5.com error123 22 us008.dot.net2.com error121 13 us009.net2.com error129Expected Output: 2... (4 Replies)
Discussion started by: sQew
4 Replies

6. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

7. UNIX for Dummies Questions & Answers

"Help with bash script" - "License Server and Patch Updates"

Hi All, I'm completely new to bash scripting and still learning my way through albeit vey slowly. I need to know where to insert my server names', my ip address numbers through out the script alas to no avail. I'm also searching on how to save .sh (bash shell) script properly.... (25 Replies)
Discussion started by: profileuser
25 Replies

8. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

9. Shell Programming and Scripting

Failure: if grep "$Var" "$line" inside while read line loop

Hi everybody, I am new at Unix/Bourne shell scripting and with my youngest experiences, I will not become very old with it :o My code: #!/bin/sh set -e set -u export IFS= optl="Optl" LOCSTORCLI="/opt/lsi/storcli/storcli" ($LOCSTORCLI /c0 /vall show | grep RAID | cut -d " "... (5 Replies)
Discussion started by: Subsonic66
5 Replies

10. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies
XSetWMProtocols(3)						  XLIB FUNCTIONS						XSetWMProtocols(3)

NAME
XSetWMProtocols, XGetWMProtocols - set or read a window's WM_PROTOCOLS property SYNTAX
Status XSetWMProtocols(Display *display, Window w, Atom *protocols, int count); Status XGetWMProtocols(Display *display, Window w, Atom **protocols_return, int *count_return); ARGUMENTS
display Specifies the connection to the X server. count Specifies the number of protocols in the list. count_return Returns the number of protocols in the list. protocols Specifies the list of protocols. protocols_return Returns the list of protocols. DESCRIPTION
The XSetWMProtocols function replaces the WM_PROTOCOLS property on the specified window with the list of atoms specified by the protocols argument. If the property does not already exist, XSetWMProtocols sets the WM_PROTOCOLS property on the specified window to the list of atoms specified by the protocols argument. The property is stored with a type of ATOM and a format of 32. If it cannot intern the WM_PRO- TOCOLS atom, XSetWMProtocols returns a zero status. Otherwise, it returns a nonzero status. XSetWMProtocols can generate BadAlloc and BadWindow errors. The XGetWMProtocols function returns the list of atoms stored in the WM_PROTOCOLS property on the specified window. These atoms describe window manager protocols in which the owner of this window is willing to participate. If the property exists, is of type ATOM, is of for- mat 32, and the atom WM_PROTOCOLS can be interned, XGetWMProtocols sets the protocols_return argument to a list of atoms, sets the count_return argument to the number of elements in the list, and returns a nonzero status. Otherwise, it sets neither of the return argu- ments and returns a zero status. To release the list of atoms, use XFree. XGetWMProtocols can generate a BadWindow error. PROPERTIES
WM_PROTOCOLS List of atoms that identify the communications protocols between the client and window manager in which the client is willing to participate. DIAGNOSTICS
BadAlloc The server failed to allocate the requested resource or server memory. BadWindow A value for a Window argument does not name a defined Window. SEE ALSO
XAllocClassHint(3), XAllocIconSize(3), XAllocSizeHints(3), XAllocWMHints(3), XFree(3), XSetCommand(3), XSetTransientForHint(3), XSet- TextProperty(3), XSetWMClientMachine(3), XSetWMColormapWindows(3), XSetWMIconName(3), XSetWMName(3), XSetWMProperties(3), XStringListTo- TextProperty(3) Xlib - C Language X Interface X Version 11 libX11 1.5.0 XSetWMProtocols(3)
All times are GMT -4. The time now is 01:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy