That is going to be extremely slow. There's much better ways to break data into fields than calling awk in backticks all the time!
You can feed arbitrary text into commands this way:
Code:
cat <<< "This text fed into cat"
For splitting that string:
Code:
read G G G G G G DATE G < test_BusinessDate.txt
IFS="/" read DD MM YY <<< "${DATE}"
echo "Date: ${DATE} or ${DD}/${MM}/${YY}"
Tested in BASH and ksh.
This avoids having to run any external programs at all, accomplishing everything with shell builtins. The IFS variable is a special variable that controls what letters the 'read' builtin splits apart its input on, and putting it right before 'read' like that changes it only for that one line.
Can't say I understand what exactly is going on in your code...
I want to pull a date from the txt file. It happens to be in the 7th column of that txt file... I then need to use that date and update two other files with it.. The problem is one file uses the format mm/dd/yy and the other uses yyyymmdd.
Trying to put a script together that will run daily.. The date will not be the current date, its a date that's based on a particular business date...
Can't say I understand what exactly is going on in your code...
I am using the <<< operator, which feeds whatever text into commands you tell it to. It's much more efficient than "echo whatever | command" and easier to read. Just try it and you'll see what it does.
Code:
$ cat <<< "this is a line of text"
this is a line of text
$
I am using the "read" builtin, which reads a single line from standard input (or whatever you redirect into it) and puts it into one or more shell variables, splitting it apart based on the IFS variable. By default, it splits on spaces, so you could do
Code:
read A B C <<< "first second third"
and have A get "first", B get "second", C get "third". Just try it and you'll see what it does.
It can split apart on other things than spaces by changing the IFS variable.
Code:
$ IFS=":" read A B C <<< "a:b:c"
$ echo $A
a
$ echo $B
b
$ echo $C
c
Here it is again in more detail.
Code:
# Read a line from test_businessDate.txt, splitting it apart into
# the variables G, G, G, G, G, G, DATE, and G.
# We don't care about anything in G.
read G G G G G G DATE G < test_BusinessDate.txt
# Read the DATE variable into the DD, MM, YY variables, splitting
# into different variables on the / character.
IFS="/" read DD MM YY <<< "${DATE}"
# Tadah.
echo "Date: ${DATE} or ${DD}/${MM}/${YY}"
The read builtin is very powerful, it can do all the splitting you're using awk for here but more simply and twenty times faster.
Code:
# Do it with read 1000 times, takes less than a tenth of a second
time for ((N=0; N<1000; N++))
do
read A B <<< "a b" && echo "$b"
done > /dev/null
real 0m0.096s
user 0m0.028s
sys 0m0.068s
# do it with AWK 1000 times, takes almost two seconds!
time for ((N=0; N<1000; N++))
do
b=`awk '{ print $2 }' <<< "a b"`
echo $b
done > /dev/null
real 0m1.930s
user 0m0.368s
sys 0m1.324s
External commands like grep, awk, and sed are only efficient when used on many lines of data. The speed difference between a builtin and an external command is significant enough that we frequently have people coming here to "optimize" their very slow scripts full of sed and grep and awk in backticks. If you find yourself running awk to process a single line, it'd be better to use a shell builtin.
Last edited by Corona688; 08-24-2010 at 05:55 PM..
I tried to test what you had shown but i get a syntax error..
Code:
#!/bin/ksh -x
read G G G G G G M1_BUS_DATE G G G G G < test_BusinessDate.txt
IFS="/" read DD MM YY <<< "$(M1_BUS_DATE)"
echo "M1 Business date: $(M1_BUS_DATE)"
echo "Edited date is: "20"$(YY)$(MM)$(DD)
+ read G G G G G G M1_BUS_DATE G G G G G
+ 0< test_BusinessDate.txt
read_test.ksh[5]: syntax error at line 5 : `<' unexpected
I tried to test what you had shown but i get a syntax error..
You've substituted round brackets for braces everywhere. You've also got a few extra quotation marks in there.
Code:
#!/bin/ksh -x
read G G G G G G M1_BUS_DATE G G G G G < test_BusinessDate.txt
IFS="/" read DD MM YY <<< "${M1_BUS_DATE}"
echo "M1 Business date: ${M1_BUS_DATE}"
echo "Edited date is: 20${YY}${MM}${DD}"
If you still get an error about unexpected <, you may be using an extremely old shell and forced to use the "echo stuff | read" trick instead(which only works in ksh afaik).
Im trying to search for a single variable in the first field and from that output use awk to extract out the lines that contain a value less than a value stored in another variable. Both the variables are associated with each other.
Any guidance is appreciated.
File that contains the... (6 Replies)
I have the following script, and I want to assign the output ($10 and $5) from awk to N and L:
grdinfo data.grd | awk '{print $10,$5}'| read N L
output from gridinfo data.grd is: data.grd 50 100 41 82 -2796 6944 0.016 0.016 3001 2461. where N and L is suppose to be 3001 and 100. I use... (8 Replies)
i have this variable:
varT="1--2--3--5"
i want to use awk to print field 3 from this variable. i dont want to do the "echo $varT".
but here's my awk code:
awk -v valA="$varT" "BEGIN {print valA}"
this prints the entire line. i feel like i'm so close to getting what i want. i... (4 Replies)
Hello experts,
I'm stuck with this script for three days now. Here's what i need.
I need to split a large delimited (,) file into 2 files based on the value present in the last field.
Samp: Something.csv
bca,adc,asdf,123,12C
bca,adc,asdf,123,13C
def,adc,asdf,123,12A
I need this split... (6 Replies)
Hi all,
Hope someone can help me out here.
I have this BASH script (see below)
My problem lies with the variable path.
The output of the command find will give me several fields. The 9th field is the path. I want to captured that and the I want to filter this to a specific level.
The... (6 Replies)
Hi all,
i have a data array as follows.
array=ertfgj2345
array=456ttygkd
.
.
.
array=errdjt3235
so number or elements in the array can varies depending on how big the data input is.
now i have a variable, and it is $1 (there are $2, $3 and so on, i am only interested in $1).
... (9 Replies)
Dear All,
we have a command output which looks like :
Total 200 queues in 30000 Kbytes
and we're going to get "200" and "30000" for further process. currently, i'm using :
numA=echo $OUTPUT | awk '{print $2}'
numB=echo $OUTPUT | awk '{print $5}'
my question is : can I use just one... (4 Replies)
Hi all,
I have been struggling with this all day, and it is key to a conversion database I have to write.
The data converts the information out of an array using AWK, and basically all I have to do is figure out how to get the value of a variable inside a variable.
Right now at its... (11 Replies)
