Reading a log file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Reading a log file
# 1  
Old 08-19-2015
Reading a log file

Hi,

I'm trying to write a script to go through few folders read some log files and make a list from the data.

the log files are in few steps each step having a final energy but I need to read the last final energy coming after the keyword "hurray".

I have so far accomplished a code like this
Code:
$MYDIR=$(pwd)
DIRS=`ls -l $MYDIR | egrep '^d' | awk '{print $9}'`

for DIR in $DIRS
do
echo  ${DIR}
cd ${DIR}
optimized=0
while IFS= read -r line; do
  case $line in
    *"HURRAY"*)
      optimized=1;#echo nice
      continue
      ;;
    "FINAL SINGLE POINT ENERGY      "*)
      [ "$optimized" = 1 ] || continue
      #final_energy=`awk '{ print 5 }' $line`
      #echo "Found optimized final energy: $final_energy"
      ;;
    "final energy")
  esac
done <input-s.out

#Read the coordinates
#n=`awk '{ print $0;exit }' job1.xyz`; #echo $n
#awk -v VAR="$n" 'NR>=3 && NR<=2+VAR'  job1.xyz

cd ..
done

the code almost works fine but there are few errors
at first I get an error
Code:
line 1: =/Users/ray/Documents/orca/bu: No such file or directory

(this is the current folder)

and second awk is not able to split the variable it's just expecting a file how can I tell it to split a variable

update: here's a link containing an example log file
https://onedrive.live.com/redir?resi...int=file%2cout

Last edited by raymondg; 08-19-2015 at 02:46 PM.. Reason: adding more info
# 2  
Old 08-19-2015
The cd error is because you use $DIR without quoting it, which splits it upon spaces into two strings. You should put it in double quotes. But I don't think you need cd at all.

It'd be easier to search for your log files and read from them than finding dirs and cd-ing into them.

Please show a sample of your input data, too. Without that, I'm just wild-guessing. You should either do the whole thing or none of it in awk, using it on a single line is pointlessly wasteful.

Code:
# create a file, /tmp/$$ where $$ is this script's process id.
# it will have lines like /path/to/input-s.out final-energy
find . -name 'input-s.out' | while read FILENAME
do
        awk '/HURRAY/ { opt=1 } ; opt && /^FINAL SINGLE POINT ENERGY/ { print FILENAME, $5 }' OFS="\t" "$FILENAME"
done > /tmp/$$

while IFS=$'\t' read FILE ENERGY
do
        # Do whatever you want to do with this energy data
done < /tmp/$$

rm -f /tmp/$$

This User Gave Thanks to Corona688 For This Post:
# 3  
Old 08-19-2015
thanks so much Corona688 I updated the post adding an example log file.
# 4  
Old 08-19-2015
Where? I don't see it.

[edit] Oh, you used a dropbox link instead of pasting any text.
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 08-19-2015
Astonishingly, I seem to have guessed it right the first try, the awk bit at least. Does the rest work for you too?
This User Gave Thanks to Corona688 For This Post:
# 6  
Old 08-19-2015
ok, there's something here
I try to say it in a logical manner
inside my home folder there are subfolder and inside the subfolder there are log files.
each subfolder name is the formula of a compound and each compound has four log files inside of the folder and all their names start with the compound's formula same as the subfolder.
an example is:
CHSiH3 (subfolder) -> CHSiH3-t.out, CHSiH3-s.out, CHSiH3-CC-s.out, CHSiH3-CC-t.out

so actually by what you said simply by searching from the current directory for each subdirectory name there should be four log files found where all have the .out extension.

I want to classify the energies based on the name of the log files
t-> triplet, s or not t -> singlet, CC -> coupled cluser, if no CC -> DFT

so at the end I can have tabular data like CSV file

Code:
Compound  State Method Energy
CHSiH3       triplet  CC      10.3243
CHSiH3       singlet DFT      9.9498
....

Do you think you can help me with cause I feel abit lost here.

Last edited by Corona688; 08-19-2015 at 05:15 PM..
# 7  
Old 08-19-2015
So, even though there's four per folder, they can all be processed individually, as long as their name is considered? Or do you need them to be grouped in four?

I am also feeling a bit lost, because knowing there's four files per folder doesn't explain what you want done with them all.
This User Gave Thanks to Corona688 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh Script, Reading A File, Grepping A File Contents In Another File

So I'm stumped. First... APOLOGIES... my work is offline in an office that has zero internet connectivity, as required by our client. If need be, I could print out my script attempts and retype them here. But on the off chance... here goes. I have a text file (file_source) of terms, each line... (3 Replies)
Discussion started by: Brusimm
3 Replies

2. UNIX for Dummies Questions & Answers

Reading Xml file and print the values into the text file in columnwise?

hi guys, i want help... Reding XML file and print the values into the text file using linux shell script file as per below xml file <sequence> <Filename>aldorzum.doc</Filename> <DivisionCode>US</DivisionCode> <ContentType>Template</ContentType> <ProductCode>VIMZIM</ProductCode> </sequence>... (4 Replies)
Discussion started by: sravanreddy
4 Replies

3. UNIX for Dummies Questions & Answers

Reading XML file and print the values in the text file using Linux shell script

hi guys, i want help... Reding XML file and print the values into the text file using linux shell script file as per below xml file <sequence> <Filename>aldorzum.doc</Filename> <DivisionCode>US</DivisionCode> <ContentType>Template</ContentType> <ProductCode>VIMZIM</ProductCode> </sequence>... (1 Reply)
Discussion started by: sravanreddy
1 Replies

4. Shell Programming and Scripting

Reading line by line from live log file using while loop and considering only those lines start from

Hi, I want to read a live log file line by line and considering those line which start from time stamp; Below code I am using, which read line but throws an exception when comparing line that does not contain error code tail -F /logs/COMMON-ERROR.log | while read myline; do... (2 Replies)
Discussion started by: ketanraut
2 Replies

5. Shell Programming and Scripting

Searching for Log / Bad file and Reading and writing to a flat file

Need to develop a unix shell script for the below requirement and I need your assistance: 1) search for file.log and file.bad file in a directory and read them 2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file 4) concatinate each row from bad file as... (3 Replies)
Discussion started by: mlpathir
3 Replies

6. Shell Programming and Scripting

Menu / Log files - reading / display

So I'm pretty green still at this, and right now I don't even have the IF statements in just trying to get this part to work. I want to basically click optino 1,2,3,4 for which logs I want to search, and then be able to put in the variable I want to grep out? Sounds easy just not clear in my mind... (4 Replies)
Discussion started by: xgringo
4 Replies

7. Shell Programming and Scripting

Reading/Compressing of log file

Hello All Posted a similar thread in some other section too. Regrets if this section is not suitable for this post. Request all the members to be tolerant as i'm a newbie here :) Coming to the topic : I've this huge log files of size 20GB-30 GB in my unix server. I want to analyse the log... (2 Replies)
Discussion started by: sgbhat
2 Replies

8. UNIX for Dummies Questions & Answers

Log File Writing and Reading

Hi all, I have the following shell script code which tries to sftp and writes the log into the log file. TestConnection () { echo 'Connection to ' $DESTUSERNAME@$DESTHOSTNAME $SETDEBUG if ]; then rm $SCRIPT ; fi touch $SCRIPT echo "cd" $REMOTEDIR >> $SCRIPT echo "quit" >>... (10 Replies)
Discussion started by: valluvan
10 Replies

9. Shell Programming and Scripting

Reading file names from a file and executing the relative file from shell script

Hi How can i dynamically read files names from a list file and execute them from a single shell script. Please help its urgent Thanks in Advance (4 Replies)
Discussion started by: anushilrai
4 Replies
Login or Register to Ask a Question