Shell Programming and Scripting

Hello all
i need to implement demon that needs to extract data from db load the data to memory and according to this data
to perform actions like sending emails or write/update files this action needs to be preformed each 30 min's
now i really don't know what to decide or to compile c++ program that will to the task or to use scripts and misc Linux tools ( sed /awk )
what will be the fastest way to do this? to save cpu and memory .
the dilemma is about marinating this process if its script it does not need compilations and i can just drop it into any machine linux/unix
but if its native its more harder .
what do you think?

Use Perl

I suggest script!

We use a similar process to manage several hundred servers. We have taken 3 approaches over the last 3 years.

1. Daemon (c++) that listens on a port and runs the passed args as command.
2. Bash script using ssh to issue command using ssh keys to make this a password-less process.
3. Bash script using expect.

Using a compiled c++ app running as a listening daemon was great for ease but very bad for security. We had to add in the ability to only allow admin IP's set in a conf file and in the end realized that this was a real bad security risk even with IP based ACL's.

The second method, worked well also. Again having ssh keys laying around on a file system was not the best idea either.This method was also ultimately aborted. PCI compliance prohibited usage of both the above.

In the end we agreed that using expect was less of a risk. We can use bash to script the things we need and use expect (inline in the bash) to handle the ssh login and pass the command and exit.

We use the read command to take in user and password as the script runs and store the user/pass in a variable and then pass to expect as needed.

While this is what we use it may or may not be your best approach. This all boils down to YOUR needs and requirements.


Also note that using read to store user and pass and then passing to expect will allow a non-admin to see the password if they run the "ps" command. This is only possible for a split second while expect spawns the ssh command to the remote host and is FAR better than having an ssh key or worse a user/password hard coded in a file somewhere.

The c++ approach will expose a port that can be seen and runs all the time. It will only take minutes for a good hacker to target that port. If that daemon is running as root then you are in real jeopardy at that point of having your whole system compromised! To make matters worse if (as you say) you have multiple machines using this daemon then all are at risk of this compromise.

Not sure if you have heard of webmin, but many also use webmin for this purpose as webmin allows you to "cluster" groups of machines and send commands, update packages/applications, add/update/delete users, edit configurations, and way more of all machines you have added to webmin console at once.

Webmin is quite mature and has very good security tools (SSL login, and user/host based ACL's as well).



EDIT:

Also, another good thing to use, no matter which method you decide to use. If at all possibe, create a shared location (NFS?) to store scripts that are for admin type processes. Then all you have to do is cycle through the list of hosts and all the command from the common nfs location.


Example:

Create an NFS export on your NFS server.
On hosts 1-4 have them all mount the share as "/nfs1"

Place admin scripts in "/nfs1/admin" (/nfs1/admin/update_users.sh or whatever).

Now hosts 1-4 can all use the script from the same location so all you have to do is this:

Code:

HOSTS="host1 host2 host2 host4" 
# This can also be stored in a table in a database and retrived via sql so you do not have to change all files every time you add a host!

for SRV in $HOSTS
do
    ssh $SRV /nfs1/admin/update_users.sh
done

Of course this is assuming an ssh connection and NOT a daemon, and does not show any expect examples but you get the idea.

what about performance and cpu ?

That really depends on the commands you are running, and how well the app/script is written. Shell scripts and c++ apps (if written well) are fast and have a small foot print.

That said, if no attention is paid to what is being done with the app/script... things can get nasty quick.

Just watch for things that cause heavy I/O (searching the whole drive or worse nfs drive for text in a file), and also watch for things that will consume all memory or cpu.

Obviously test all scripts/apps on a test machine prior to running anywhere near production servers.

Watch the I/O, CPU, and Mem when you test.