Automate Log Monitoring Process


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Automate Log Monitoring Process
# 1  
Old 04-06-2009
Automate Log Monitoring Process

I am a new member of this forum and am also new to unix shell scripting.
I joined the forum to seek for help to achieve my task as this forum helps people.
here's what i do manually on daily basis
1)Loginto different unix box
2)Ftp the log files (morethan 50 each dir) to windows
3)use text pad and do find in files "Exception"
4)I get the file name , the exception entry
These tasks take much some since i need to pull the log files from 10 different server and by the time i find the exception , it 's almost the day is over.
Need your help in automating this process
So what i need a shell script to
1)from dev unix box, copy the logs files from 10 different servers
2)Use shell script/awk command to look thro' each directory
and find for particular string that i mention in the script.
It's kind of rule based name of the log file. date of the log , etc pass as a parameter.
Don't ask me what i have done. because i am new to unix scripting.
Help me so i can refer the forum to many people.
# 2  
Old 04-07-2009
Code:
cat << EOF |
server1 directory local_directory file_meta_chars*
server2 /somewhere /somewhereelse log_blue*
server3 /here /there log_green*
EOF
while read hostname directory local_directory file_meta ; do

ftp -nvd << EOF
open $hostname
user sharugan my-password
cd $directory
lcd $local_directory
prompt
mget $file_meta
bye
EOF

cd $local_directory
grep Exception $file_meta

done |
  tee /somepath/my-complete-log

.. . or something like that should do the trick.
# 3  
Old 04-07-2009
thank you so much for the script.I am going to try it out.
As i said i am new to shell script and don't know anything about shell script.
But your comments say "or something like that should do the trick"
wondering if this is not the exact script or i need to modify these.
if so , can you give me the exact script that does all the task for me as i mentioned above.
# 4  
Old 04-07-2009
as i am learing from you , can you add comments on the each block to understand what it does. for ex: I don't know what this block does.
cat << EOF |
server1 directory local_directory file_meta_chars*
server2 /somewhere /somewhereelse log_blue*
server3 /here /there log_green*
EOF
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Monitoring processes in parallel and process log file after process exits

I am writing a script to kick off a process to gather logs on multiple nodes in parallel using "&". These processes create individual log files. Which I would like to filter and convert in CSV format after they are complete. I am facing following issues: 1. Monitor all Processes parallelly.... (5 Replies)
Discussion started by: shunya
5 Replies

2. Shell Programming and Scripting

Script to automate recovery process

Hello All! First post... I am working on a script that is used to recover a crashed drive from an rsync backup. I'm down to the place where I need to create all of the directories in /mnt where I will then mount each of the volumes and begin the restore process to each volume... I have... (3 Replies)
Discussion started by: RogerBaran
3 Replies

3. UNIX for Dummies Questions & Answers

Automate sftp process using script

Hi, guys, I am trying to automate a sftp process using "expect" method (since the key authentication method is disabled in my company network, there is no helping it). In order to try, I type in the command manually: sftp @ > << EOF >cd >ls -l >EOF >Connecting to @servername password: ... (3 Replies)
Discussion started by: warmboy610
3 Replies

4. Shell Programming and Scripting

Automate Job monitoring

Hi, I have a project which i need to automate some manual work. We have a 100 jobs in Linux and Unix environment. We do job monitoring daily. that is taking more that 2 to 3 hrs for us because we use autorep -j <jobname> command for each and every job. That takes more time for us to do. We have... (16 Replies)
Discussion started by: cg_saran
16 Replies

5. UNIX for Dummies Questions & Answers

To automate a process

CAN ANYONE HELP TO SOLVE i wann write a script to automate a process .i.e, to search files in the FTP server and and if files are there and we hav to bring that files to our system. After copying the files in our system we have to upload the data in the tables. I have scripts to load the... (2 Replies)
Discussion started by: nani1984
2 Replies

6. UNIX for Advanced & Expert Users

TO Automate the FTP process

Plzz Some One Help in this matter I have scripts to load the data into tables and to copy files from ftp to our system. we use to run the scripts every day.... we hav the files in the FTP server and we hav to bring the files to our system and we hav to load the data into the tables. We... (0 Replies)
Discussion started by: nani1984
0 Replies

7. Shell Programming and Scripting

To automate a process

CAN ANYONE HELP TO SOLVE i wann write a script to automate a process .i.e, to search files in the FTP server and and if files are there and we hav to bring that files to our system. After copying the files in our system we have to upload the data in the tables. I have scripts to load the... (0 Replies)
Discussion started by: nani1984
0 Replies

8. Shell Programming and Scripting

Error in script to automate the daily monitoring process of UNIX server and it's proc

hi friends, I am trying to automate the daily monitoring process of UNIX server and it's processes. the script are below i executed the above script using ksh -x monitortest1.sh in root login . It shows error at some lines . 1. i logged in using root ,but it... (8 Replies)
Discussion started by: rdhaprakasam
8 Replies

9. Shell Programming and Scripting

PLEASE PLEASE Help - Automate text manipulation process

I am trying to automate a process (on windows OS) of manipulating a file by changing its column delimiters. I have a perl script ready but the problem i think is with the format of the file itself. code: #! usr/bin/perl -w foreach (<STDIN>) { #copy this line: my $line = $_; ... (1 Reply)
Discussion started by: salemh
1 Replies

10. Shell Programming and Scripting

How to automate an FTP process?

Hello script experts, I am newbie to shell script. But I have to write a shell script (ASAP) where I need to ftp a file on daily basis to a remote server, and send an email with final status. I I should have a properties file with hostname, Userid, and pwd. And a shall script file should read... (1 Reply)
Discussion started by: ksak
1 Replies
Login or Register to Ask a Question