Sponsored Content
Top Forums Shell Programming and Scripting Using xargs for multiple functions Post 302436227 by tange on Friday 9th of July 2010 06:16:02 PM
Old 07-09-2010
Quote:
Originally Posted by krypton

Code:
cat syslogs | nawk '/From/ { print $3 }' | uniq | xargs -I {} grep {} syslogs >> {}

GNU Parallel is your friend:

cat syslogs | nawk '/From/ { print $3 }' | uniq | parallel "grep {} syslogs >> {}"

This even does the grepping in parallel, so if syslogs is bigger than your disk cache this will go considerably faster than reading the file from disk over and over again. The option -j0 may be useful to you as well.

Watch the intro video for GNU Parallel: YouTube - Part 1: GNU Parallel script processing and execution
This User Gave Thanks to tange For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Can Xargs execute multiple commands of evry input file

Hello , I am trying to print the footer of evry file in the given directory with xargs command like follows ls -1 | xargs -I {} gzcat {} | tail -1 now problem with this is only last file foooter is getting printed as " | tail -1 " is getting executed for the last file. I know this can... (4 Replies)
Discussion started by: nilesrex
4 Replies

2. Shell Programming and Scripting

Multiple functions

Hi, which of the shell support multiple functions in a shell script? Thanks Amit (1 Reply)
Discussion started by: amitrajvarma
1 Replies

3. Shell Programming and Scripting

Run Multiple Functions over SSH (BASH)

I am trying to write a script that will ssh into a remote machine and recurse through a specified directory, find mp3 files which may be two or three directories deep (think iTunes: music/artist/album/song.mp3), and scp them back to the machine running the script. The script should also maintain... (3 Replies)
Discussion started by: johnnybg00de
3 Replies

4. Shell Programming and Scripting

Calling multiple functions in parallel

Hello, I have multiple functions within a shell script. eg. function_database_backup, unix_tar_creation, etc. I would like to run these functions in parallel, as each is independent of the other. If these were simple commands, I could have probably run each of the commands in background. ... (1 Reply)
Discussion started by: neil.k
1 Replies

5. Shell Programming and Scripting

How to run multiple functions in Background in UNIX Shell Scripting?

Hi, I am using ksh , i have requirement to run 4 functions in background , 4 functions call are available in a case that case is also in function, i need to execute 1st function it should run in background and return to case and next i will call 2nd function it should run in background and... (8 Replies)
Discussion started by: karthikram
8 Replies

6. Shell Programming and Scripting

Xargs: multiple commands to each argument

Hello. There is my one-liner to get subjects of potential spam mails sudo exiqgrep -bf "spamer@example.com" |cut -d' ' -f1 |xargs -I ~ sudo /usr/sbin/exim -Mvh ~ |grep 'Subject: ' I want to insert blank line after each iteration to make output more readable. I tried sudo exiqgrep -bf... (1 Reply)
Discussion started by: urello
1 Replies

7. Shell Programming and Scripting

Parallel processing for functions in xargs

I have a script (ksh) which tries to run a function in parallel for performance gains. I am also trying to limit the number of parallel child processes to avoid overloading the system by using a variable to count triggered processes and waiting for completion e.g. do_something () { ... } ... (9 Replies)
Discussion started by: jawsnnn
9 Replies

8. Shell Programming and Scripting

Python passing multiple parameters to functions

Hi, I am a beginner in python programming. In my python script have a main function which calls several other functions. The main function gets its input by reading lines from a input text file. I call the main function for every line in input text file through a loop. def main(line): var1... (6 Replies)
Discussion started by: ctrld
6 Replies

9. Shell Programming and Scripting

Shell script to apply functions to multiple columns dynamically

Hello, I have a requirement to apply hashing algorithm on flat file on one or more columns dynamically based on header sample input file ID|NAME|AGE|GENDER 10|ABC|30|M 20|DEF|20|F say if i want multiple columns based on the header example id,name or id,age or name,gender and hash and... (13 Replies)
Discussion started by: mkathi
13 Replies

10. Shell Programming and Scripting

Xargs to call python executable to process multiple bam files

I am running the below loop that to process the 3 bam files (which isn't always the case). A .py executable is then called using | xargs sh to further process. If I just run it with echo the output is fine and expected, however when | xargs sh is added I get the error. I tried adding | xargs... (4 Replies)
Discussion started by: cmccabe
4 Replies
syslog.auth(4)						     Kernel Interfaces Manual						    syslog.auth(4)

NAME
syslog.auth - authorization file for accepting remote syslog messages SYNOPSIS
# format: Each fully qualified host name on a separate line hostname.domain_name DESCRIPTION
The /etc/syslog.auth file specifies which remote hosts are allowed to forward syslog messages to the local host. For the sake of security, only messages coming from remote hosts listed in the local /etc/syslog.auth file will be logged by the syslogd daemon. If the /etc/syslog.auth file does not exist, then messages coming from any host will be accepted. Each remote host name should appear in a separate line in /etc/syslog.auth. A line started with the # character is considered as a comment and is thus ignored. A host name must be a complete domain name such as trout.zk3.dec.com. If a domain host name is given, it must either appear in the local /etc/hosts file or be able to be resolved by the local name server (BIND). Note that a host name can have at most as many characters as defined by the MAXHOSTNAMELEN constant in <sys/param.h>, although each line in the /etc/syslog.auth file can have up to 512 characters. The /etc/syslog.auth file must be owned by root and has a permission of 0600. To invoke a new version of the /etc/syslog.auth file, run the following command (as the super user) to initialize the syslogd daemon: kill -HUP `cat /var/run/syslog.pid` EXAMPLES
The following example provides a typical authorization file: # format: Each fully qualified host name on a separate line c3poid.rvo.dec.com r2d2id.ckt.dec.com FILES
Location of the authorization file. RELATED INFORMATION
Commands: syslogd(8), syslog(1) System Administration delim off syslog.auth(4)
All times are GMT -4. The time now is 08:52 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy