Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Read logline line by line with awk/sed

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read logline line by line with awk/sed
# 1  
Old 05-24-2008
Read logline line by line with awk/sed
Hello,

I have a logfile which is in this format:

Code:
1211667249500#3265
1211667266687#2875
1211667270781#1828

Is there a way to read the logfile line by line every time I execute the code and put the two numbers in the line in two separate variables?

Something like:
Code:
1211667249500#3265

Into:
Code:
var1=1211667249500 var2=3265

Thanks
# 2  
Old 05-24-2008
You mean you execute the script, and you get two variables? You can try this.
Code:
#!/bin/bash
#GetVarOnceALine.ksh <logfilename>

line=`head -1 $1`
if [ $line -eq "" ]; then
echo "Logfile is empty now"
exit 1;
fi

sed '2,$ p' <$1 >/tmp/tmpfile
mv /tmp/tmpfile $1

var1=`echo $line|awk -F# '{print $1}'`
var2=`echo $line|awk -F# '{print $2}'`

exit 0

# 3  
Old 05-25-2008
Save the following as "myscript"...
for i in `cat $1`
do
var1=$(echo $i | awk -F\# '{print $1}')
var2=$(echo $i | awk -F\# '{print $2}')
echo "var1=${var1} var2=${var2}"
done


Assuming your data is in "datafile"...
1211667249500#3265
1211667266687#2875
1211667270781#1828

Run the script....
myscript datafile

Output it produces...
var1=1211667249500 var2=3265
var1=1211667266687 var2=2875
var1=1211667270781 var2=1828
# 4  
Old 05-25-2008
Deleted by owner
Last edited by danmero; 05-26-2008 at 09:45 PM.. Reason: Out of scope.
# 5  
Old 05-25-2008
Hello,

it's almost working, except one thing. I was a bit unclear about this, sorry!
The logfile gets bigger every time I run a script.

first time:
Code:
1211667249500#3265

second time:
Code:
1211667249500#3265
1211667266687#2875

third time:
Code:
1211667249500#3265
1211667266687#2875
1211667270781#1828

Instead of this as a result from your scripts:
Code:
var1=1211667249500 var2=3265
var1=1211667266687 var2=2875
var1=1211667270781 var2=1828

I'd want this as a result:

First time:
Code:
var1=1211667249500 var2=3265

Second time:
Code:
var1=1211667266687 var2=2875

Third time:
Code:
var1=1211667270781 var2=1828

So it actually reads the logfile line by line, instead of all lines at once. But also reads it one line after the previous one.

Thanks for all the replies Smilie

ps.
When I do awk -F'#' '{print $2/1000}' log I get 3.265
and when I do awk -F'#' '{print $1/1000}' log I get 1.21167e+09 is there a way to get the full number instead?
Last edited by dejavu88; 05-25-2008 at 07:52 AM..
# 6  
Old 05-25-2008
If yout want to get the latest info. One option is to use "tail -f filename"

This will display the latest values as they happen...

tail -f datafile | for i in `cat -`
do
var1=$(echo $i | awk -F\# '{print $1}')
var2=$(echo $i | awk -F\# '{print $2}')
echo "var1=${var1} var2=${var2}"
done
# 7  
Old 05-26-2008
FlyingSquirrel, your last bit of code generates the same output. I found out there's more parameters for tail and tail -n 1 does the job for me.

Thanks everyone for the help, I learned a lot. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

2. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

3. Shell Programming and Scripting

Multiple line search, replace second line, using awk or sed

All, I appreciate any help you can offer here as this is well beyond my grasp of awk/sed... I have an input file similar to: &LOG &LOG Part: "@DB/TC10000021855/--F" &LOG &LOG &LOG Part: "@DB/TC10000021852/--F" &LOG Cloning_Action: RETAIN &LOG Part: "@DB/TCCP000010713/--A" &LOG &LOG... (5 Replies)
Discussion started by: KarmaPoliceT2
5 Replies

4. Shell Programming and Scripting

How to read a two files, line by line in UNIX script and how to assign shell variable to awk ..?

Input are file and file1 file contains store.bal product.bal category.bal admin.bal file1 contains flip.store.bal ::FFFF:BADC:CD28,::FFFF:558E:11C5,6,8,2,1,::FFFF:81C8:CA8B,::FFFF:BADC:CD28,1,0,0,0,::FFFF:81C8:11C5,2,1,0,0,::FFFF:81DC:3111,1,0,1,0 store.bal.... (2 Replies)
Discussion started by: veeruasu
2 Replies

5. Shell Programming and Scripting

sed command to replace a line at a specific line number with some other line

my requirement is, consider a file output cat output blah sdjfhjkd jsdfhjksdh sdfs 23423 sdfsdf sdf"sdfsdf"sdfsdf"""""dsf hellow there this doesnt look good et cetc etc etcetera i want to replace a line of line number 4 ("this doesnt look good") with some other line ... (3 Replies)
Discussion started by: vivek d r
3 Replies

6. Shell Programming and Scripting

how to read the contents of two files line by line and compare the line by line?

Hi All, I'm trying to figure out which are the trusted-ips and which are not using a script file.. I have a file named 'ip-list.txt' which contains some ip addresses and another file named 'trusted-ip-list.txt' which also contains some ip addresses. I want to read a line from... (4 Replies)
Discussion started by: mjavalkar
4 Replies

7. Shell Programming and Scripting

sed to read line by line and input into another file

I have two files. Fileone contains text string one text string two text string three Filetwo contains Name: Address: Summary: Name: Address: Summary: Name: Address: Summary: I would like to use sed to read each line of file one and put it at the end of the summary line of file... (3 Replies)
Discussion started by: dolacap
3 Replies

8. Shell Programming and Scripting

bash: read file line by line (lines have '\0') - not full line has read???

I am using the while-loop to read a file. The file has lines with null-terminated strings (words, actually.) What I have by that reading - just a first word up to '\0'! I need to have whole string up to 'new line' - (LF, 10#10, 16#A) What I am doing wrong? #make file 'grb' with... (6 Replies)
Discussion started by: alex_5161
6 Replies

9. Shell Programming and Scripting

awk;sed appending line to previous line....

I know this has been asked before but I just can't parse the syntax as explained. I have a set of files that has user information spread out over two lines that I wish to merge into one: User1NameLast User1NameFirst User1Address E-Mail:User1email User2NameLast User2NameFirst User2Address... (11 Replies)
Discussion started by: walkerwheeler
11 Replies

10. Shell Programming and Scripting

Using Sed to read one line only

I have a bash script which produces the desired output (mostly). tempTxt=`sed -n '79,/^$/p' <sourceFile.txt | cut -c 13-19` What this produces is the text from the 79th line of the source file and cuts the text to the 13th through the 19th chars. Then the script does the same thing... (4 Replies)
Discussion started by: roninuta
4 Replies
Login or Register to Ask a Question