Sponsored Content
Top Forums UNIX for Advanced & Expert Users Help Me - How to grep in multiple servers Post 302328222 by l_vbosch on Tuesday 23rd of June 2009 07:45:54 PM
Old 06-23-2009
If your're designing a _real_ search website I humbly think that you have to revise your whole strategy because searching that way is completely inefficient and resource wasteful.

After saying that I can suggest a few possibilities working with what you have.

1) If you want to search from a workstation through all the servers through SSH, firstly I suggest you configure a simple PKI infrastructure (private/public key). With that, you avoid typing the remote server password each time. In the client you need to have running a ssh-agent to login in behaf of your own user.

After thar you could use something like this:

for server in [ mumbai bangalore chennai delhi hyderabad other-hostnames ]; do echo $server; ssh $server zcat /path/to/voter.gz | grep -l "keyword"; done

You have to rewrite the above one-liner to your exact needs.


A quick work around for a better solution should be a script that parses that compressed files extracting all keywords and populating a MySQL (or any other) database. You're searches will be a lot faster and consistence because you have all the info in a single repository.

Ragards,
Leandro.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ftp to multiple servers

Hi folks. I am writing a ksh ftp script. The problem is, I need to transfer the files to several different servers. Is there a way to close a connection and move on to the next in one script or do I need to write a separate script for each one? Thanks, kristy (2 Replies)
Discussion started by: kristy
2 Replies

2. Shell Programming and Scripting

login into multiple servers thru script...

I need to login into multiple servers thru a script run couple commands and run find command as root. I only have ssh access to the servers as a user than I can "su" to root. If you have a similar script please post it. Also if you can suggest commands that I should consider please let me know. ... (1 Reply)
Discussion started by: avcert1998
1 Replies

3. Shell Programming and Scripting

rsh to change multiple ip in multiple servers?

good day. i jsut wanted to know what is the best script or the best way changing a lot of Ip's in all servers. Do you have any idea? im using awk to change IP,what if, you have lots of servers. You need to change it one by one? It will take time to change it manually. (2 Replies)
Discussion started by: kenshinhimura
2 Replies

4. UNIX for Advanced & Expert Users

grep for a line in a file on multiple unix servers

Hi, I need to grep(look) for a line in filename named /tmp/Test on all 90 unix servers. Without logging to each servers manually and grep'ing for the line, is there a way to pull this info in output file thru' the ksh/sh/bash script. Can someone pl. help to create this script for me? Ex.... (3 Replies)
Discussion started by: Mike1234
3 Replies

5. Shell Programming and Scripting

Deploying Cronjob to multiple servers

I figure that this is a shell scripting issue so I'll post here. I'm attempting to deploy a cronjob to a variety of servers, some 200-odd, of all walks of life. Mostly AIX, but some Solaris, some HP-UX, some Linux (of varying distributions), etc. I initially thought about utilizing a mass... (2 Replies)
Discussion started by: Magus Zeal
2 Replies

6. Shell Programming and Scripting

Checking running process status using "grep" on multiple servers in load sharing system.

Suppose i have 3 different servers say x,y and z. Im running some process say ABC and 40 instances for the same is being created. In load sharing suppose on server x, 20 instances are running server y, 10 instances are running server z, 10 instances are running. While checking the... (1 Reply)
Discussion started by: ankitknit
1 Replies

7. Shell Programming and Scripting

Grep from multiple patterns multiple file multiple output

Hi, I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command. Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns. Xargs -I {} grep... (3 Replies)
Discussion started by: Diya123
3 Replies

8. UNIX for Advanced & Expert Users

Help with credentials when using Grep across multiple servers

Hello all, I need some help with a script I have been working on. I was wanting to know if it is possible to add authentication to it for each server it runs across? The credentials are all the same on each server. This is what I am using so far and it seems to work. I am trying to avoid... (3 Replies)
Discussion started by: Smorgen
3 Replies

9. UNIX for Beginners Questions & Answers

ssh multiple servers

Hi folks. I'm pretty new to unix, while I'm learning a lot I'm finding bash scripting quite confusing. Im sure it's not really, my head just hasn't clicked with it. Anyway, I need a script to loop the ip addresses stored in a file and run a "pgrep <process>" and return the pid or some... (2 Replies)
Discussion started by: MuntyScrunt
2 Replies

10. UNIX for Beginners Questions & Answers

How to apply the update statement in multiple servers on multiple dbs at a time .?

Hi , Can any please help the below requirement on all multiple servers and multiple dbs. update configuration set value='yes' ;1) the above statement apply on 31 Databases at a time on different Ip address eg : 10.104.1.12 (unix ip address ) the above ip box contains 4 db's eg : db... (2 Replies)
Discussion started by: venkat918
2 Replies
SSH-ADD(1)						    BSD General Commands Manual 						SSH-ADD(1)

NAME
ssh-add -- adds private key identities to the authentication agent SYNOPSIS
ssh-add [-cDdkLlXx] [-t life] [file ...] ssh-add -s pkcs11 ssh-add -e pkcs11 DESCRIPTION
ssh-add adds private key identities to the authentication agent, ssh-agent(1). When run without arguments, it adds the files ~/.ssh/id_rsa, ~/.ssh/id_dsa, ~/.ssh/id_ecdsa, ~/.ssh/id_ed25519 and ~/.ssh/identity. After loading a private key, ssh-add will try to load corresponding certificate information from the filename obtained by appending -cert.pub to the name of the private key file. Alternative file names can be given on the command line. If any file requires a passphrase, ssh-add asks for the passphrase from the user. The passphrase is read from the user's tty. ssh-add retries the last passphrase if multiple identity files are given. The authentication agent must be running and the SSH_AUTH_SOCK environment variable must contain the name of its socket for ssh-add to work. The options are as follows: -c Indicates that added identities should be subject to confirmation before being used for authentication. Confirmation is performed by the SSH_ASKPASS program mentioned below. Successful confirmation is signaled by a zero exit status from the SSH_ASKPASS program, rather than text entered into the requester. -D Deletes all identities from the agent. -d Instead of adding identities, removes identities from the agent. If ssh-add has been run without arguments, the keys for the default identities and their corresponding certificates will be removed. Otherwise, the argument list will be interpreted as a list of paths to public key files to specify keys and certificates to be removed from the agent. If no public key is found at a given path, ssh-add will append .pub and retry. -e pkcs11 Remove keys provided by the PKCS#11 shared library pkcs11. -k When loading keys into or deleting keys from the agent, process plain private keys only and skip certificates. -L Lists public key parameters of all identities currently represented by the agent. -l Lists fingerprints of all identities currently represented by the agent. -s pkcs11 Add keys provided by the PKCS#11 shared library pkcs11. -t life Set a maximum lifetime when adding identities to an agent. The lifetime may be specified in seconds or in a time format specified in sshd_config(5). -X Unlock the agent. -x Lock the agent with a password. ENVIRONMENT
DISPLAY and SSH_ASKPASS If ssh-add needs a passphrase, it will read the passphrase from the current terminal if it was run from a terminal. If ssh-add does not have a terminal associated with it but DISPLAY and SSH_ASKPASS are set, it will execute the program specified by SSH_ASKPASS and open an X11 window to read the passphrase. This is particularly useful when calling ssh-add from a .xsession or related script. (Note that on some machines it may be necessary to redirect the input from /dev/null to make this work.) SSH_AUTH_SOCK Identifies the path of a UNIX-domain socket used to communicate with the agent. FILES
~/.ssh/identity Contains the protocol version 1 RSA authentication identity of the user. ~/.ssh/id_dsa Contains the protocol version 2 DSA authentication identity of the user. ~/.ssh/id_ecdsa Contains the protocol version 2 ECDSA authentication identity of the user. ~/.ssh/id_ed25519 Contains the protocol version 2 ED25519 authentication identity of the user. ~/.ssh/id_rsa Contains the protocol version 2 RSA authentication identity of the user. Identity files should not be readable by anyone but the user. Note that ssh-add ignores identity files if they are accessible by others. EXIT STATUS
Exit status is 0 on success, 1 if the specified command fails, and 2 if ssh-add is unable to contact the authentication agent. SEE ALSO
ssh(1), ssh-agent(1), ssh-keygen(1), sshd(8) AUTHORS
OpenSSH is a derivative of the original and free ssh 1.2.12 release by Tatu Ylonen. Aaron Campbell, Bob Beck, Markus Friedl, Niels Provos, Theo de Raadt and Dug Song removed many bugs, re-added newer features and created OpenSSH. Markus Friedl contributed the support for SSH protocol versions 1.5 and 2.0. BSD
December 7, 2013 BSD
All times are GMT -4. The time now is 12:33 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy