Help to simplify with awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help to simplify with awk
# 1  
Old 08-27-2008
Bug Help to simplify with awk

Hi,

Need your help guys. I'm trying to tweak my current shell-script to make it run faster. I think a part of the code that takes too long is the splitting of the 1st field of my CSV raw-file to date-time. Below is the 1st column of my CSV file:

$ awk -F"," {'print $1'} temp-*|head
20080826000059
20080826000325
20080826000400
20080826000044
20080826000332
20080826001014
20080826001932
20080826002833
20080826002107
20080826002148

When my script runs it becomes:

2008.08.26,00:00:59
2008.08.26,00:03:25
2008.08.26,00:04:00
2008.08.26,00:00:44
2008.08.26,00:03:32
2008.08.26,00:10:14
2008.08.26,00:19:32
2008.08.26,00:28:33
2008.08.26,00:21:07
2008.08.26,00:21:48

the part of my script that does this is as below:

for LINE in `cat $INPUT/temp-out.rgp3`
do
YEAR=`echo $LINE|awk -F"," {'print $1'}|cut -c 1-4`
MONTH=`echo $LINE|awk -F"," {'print $1'}|cut -c 5-6`
DEY=`echo $LINE|awk -F"," {'print $1'}|cut -c 7-8`
HOUR=`echo $LINE|awk -F"," {'print $1'}|cut -c 9-10`
MIN=`echo $LINE|awk -F"," {'print $1'}|cut -c 11-12`
SEC=`echo $LINE|awk -F"," {'print $1'}|cut -c 13-14`
done


Is there a way that I can simplify this to make the script run faster? Maybe use awk {gsub} or something similar?

Appreciate your replies. Smilie

Thanks.
# 2  
Old 08-27-2008
Try...
Code:
awk -F"," '{ 
       YEAR=substr($1,1,4)
       MONTH=substr($1,5,2)
       DEY=substr($1,7,2)
       HOUR=substr($1,9,2)
       MIN=substr($1,11,2)
       SEC=substr($1,13,2)
       print YEAR "." MONTH "." DEY "," HOUR ":" MIN ":" SEC
    }' $INPUT/temp-out.rgp3

# 3  
Old 08-27-2008
Also take a look at this solution: https://www.unix.com/shell-programmin...#post302228573
# 4  
Old 08-27-2008
Or even just

Code:
sed -e 's/\(....\)\(..\)\(..\)\(..\)\(..\)\(..\).*/\1.\2.\3,\4:\5:\6/' $INPUT/temp-out.rgp3

If the objective is to have these variables available to your shell script in some further processing before the "done", then maybe something like

Code:
while read line; do
  tail=${line#????}; YEAR=${line%$tail};  line=${line#$YEAR}
  tail=${line#??};   MONTH=${line%$tail}; line=${line#$MONTH}
  tail=${line#??};   DAY=${line%$tail};   line=${line#$DAY}
  tail=${line#??};   HOUR=${line%$tail};  line=${line#$HOUR}
  tail=${line#??};   MIN=${line%$tail};   line=${line#$MIN}
  tail=${line#??};   SEC=${line%$tail}
  echo $YEAR.$MONTH.$DAY,$HOUR:$MIN:$SEC
  : more stuff here
done <$INPUT/temp-out.rgp3

This isn't terribly well encapsulated; I'd suggest you turn this into a function to make it prettier.

Last edited by era; 08-27-2008 at 10:27 AM.. Reason: Alignment for improved legibility
# 5  
Old 08-29-2008
Code:
echo 20080826000059 | sed 's/\(....\)\(..\)\(..\)\(..\)\(..\)\(..\)/\1\.\2\.\3,\4:\5:\6/'

# 6  
Old 08-29-2008
works!

Quote:
Originally Posted by Ygor
Try...
Code:
awk -F"," '{ 
       YEAR=substr($1,1,4)
       MONTH=substr($1,5,2)
       DEY=substr($1,7,2)
       HOUR=substr($1,9,2)
       MIN=substr($1,11,2)
       SEC=substr($1,13,2)
       print YEAR "." MONTH "." DEY "," HOUR ":" MIN ":" SEC
    }' $INPUT/temp-out.rgp3

It works like a charm!

Thanks!!!!
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Programming

Simplify setter and getter of java class

I am trying to verify my understanding on setter and getter on java class with this example: //MaximumFinder2.java import java.util.Scanner; public class MaximumFinder2 { public static void main (String args) { Scanner input = new Scanner(System.in); ... (6 Replies)
Discussion started by: yifangt
6 Replies

2. Shell Programming and Scripting

How to simplify this code?

hi guys need your help...how to simplify this script... for i in `cat dmp.txt` do model=$i more $model | grep : | cut -d ":" -f 2- | grep : | grep -v "=" > temp1 more $model | grep / | cut -d ":" -f 2- > temp2 more $model | grep = | cut -d ":" -f 2- > temp3 more... (2 Replies)
Discussion started by: zulabc
2 Replies

3. Shell Programming and Scripting

Need to simplify code list files in directory

I have the following code that print certain files in my directory and groups things together. I would be good to tidy and simplify this code up and would greatly appreciate suggestions. if ($opt_raytrac == 1) then echo "" echo -n "\033*\)/ & /g' | sort -k 4n | sed 's/ //g' \ # |... (0 Replies)
Discussion started by: kristinu
0 Replies

4. Shell Programming and Scripting

simplify regular expressions

Hi can anyone help me with how to simplify this regular expression ---------- Post updated at 09:16 PM ---------- Previous update was at 09:11 PM ---------- IS THIS RIGHT ? (3 Replies)
Discussion started by: drew211
3 Replies

5. Shell Programming and Scripting

simplify/combine if statements would be nice

below is something i inherited: if && && ; then HOST_SELECT="-m quadcore" fi if && && ; then HOST_SELECT="-m quadcore" fi if && && ; then HOST_SELECT="-m octocore1" fibelow is what i changed it to: if && && ; then HOST_SELECT="-m quadcore"... (2 Replies)
Discussion started by: crimso
2 Replies

6. Shell Programming and Scripting

Simplify Bash Script Using "sed" Or "awk"

Input file: 2 aux003.net3.com error12 6 awn0117.net1.com error13 84 aux008 error14 29 aux001.ha.ux.isd.com error12 209 aux002.vm.ux.isd.com error34 21 alx0027.vm.net2.com error12 227 dux001.net5.com error123 22 us008.dot.net2.com error121 13 us009.net2.com error129Expected Output: 2... (4 Replies)
Discussion started by: sQew
4 Replies

7. Shell Programming and Scripting

Can I simplify this script?

Hi all, I have a script which runs every morning which clears down a series of directories. The structures of which are; /opt/feeds/failed/feed1 /opt/feeds/succeed/feed1 /opt/feeds/failed/feed2 /opt/feeds/succeed/feed2 /opt/feeds/failed/feed3 /opt/feeds/succeed/feed3 etc etc Files... (6 Replies)
Discussion started by: JayC89
6 Replies

8. Shell Programming and Scripting

How can I simplify the script

Hi all, How can I simplify following script, Logic is to find two strings (strings are case sensitive) from a file. if ; then if ; then Group=`echo $1_hostname` fi fi Please help me on this. Regards Sudhish s. kumar (8 Replies)
Discussion started by: sudhish
8 Replies

9. Shell Programming and Scripting

a script to simplify the use of grep

I'm trying to write a script that simplify the use of grep utility. these are the option that I'd like to use with the script " -c -i -l -n -v". When I execute the script none of these option works. I really appreciate any idea or tips regarding this problem. here the code echo " Enter 1-7:"... (2 Replies)
Discussion started by: kemobyte
2 Replies
Login or Register to Ask a Question