Pipe filtering


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pipe filtering
# 1  
Old 08-02-2008
Pipe filtering

Hi,

The thing is that sometimes I need to filter an arbitrary output given a condition (think on the "filter" function present on some programming languages) from the shell (I use sh/bash). I tried this:

Code:
#!/bin/sh
# Usage: filter command [options_for_xargs]
COMMAND=$1
shift 1
xargs "$@" -I "{}" sh -c "if $COMMAND; then echo {}; fi"

And now, if I want to get all the non-executable files under /etc, I can run this:

Code:
$ find /etc -type f -print0 | filter "! test -x {}" -0
/etc/fam.conf
/etc/cups/cupsd.conf
/etc/cups/snmp.conf
/etc/cups/lpoptions
/etc/cups/mime.types
....

So the question is... is there any better/faster way of doing this generic filter with sh/bash?

thanks!
# 2  
Old 08-02-2008
FWIW: find supports the -mode option - you use this to test permissions bits.

Your generic filter invokes a whole new process, which may not be all that efficient when compared to other options available to a command. It is an interesting idea, though. Have you considered the concept of a coprocess using named pipes?

See the bash FAQ here:
ftp://ftp.cwru.edu/pub/bash/FAQ
# 3  
Old 08-03-2008
Quote:
Originally Posted by jim mcnamara
FWIW: find supports the -mode option - you use this to test permissions bits.
Yes, I'm aware of this, it was just an example.

Quote:
Originally Posted by jim mcnamara
Your generic filter invokes a whole new process, which may not be all that efficient when compared to other options available to a command.
Certaintly. A pure bash solution (less versatile as it does not use xargs) I though was as simple as:

Code:
#!/bin/sh
COMMAND=$1
while read LINE; do
     eval "${COMMAND/\{\}/$LINE}" && echo $LINE
done

Quote:
Originally Posted by jim mcnamara
Have you considered the concept of a coprocess using named pipes?
No, I didn't, how could it be written? anyway you gave me an idea: taking the 'sh' out of xargs and run it on a pipe:

Code:
xargs "$@" -I "{}" echo "$COMMAND && echo {}" | sh

It's not as fast as the pure bash solution, but is cheaper to run "echo"s than "sh"s processes. If it was possible to for xargs to output the command without actually running it, it should really fast (there is no such option AFAIK)

thanks
# 4  
Old 08-03-2008
The FAQ shows you how to write coprocesses with named pipes.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to ignore Pipe in Pipe delimited file?

Hi guys, I need to know how i can ignore Pipe '|' if Pipe is coming as a column in Pipe delimited file for eg: file 1: xx|yy|"xyz|zzz"|zzz|12... using below awk command awk 'BEGIN {FS=OFS="|" } print $3 i would get xyz But i want as : xyz|zzz to consider as whole column... (13 Replies)
Discussion started by: rohit_shinez
13 Replies

2. Shell Programming and Scripting

Need help on filtering

Hi experts, I have a file image.csv as below: COMPUTERNAME,23/07/2013,22/07/2013,21/07/2013,20/07/2013,19/07/2013,18/07/2013,17/07/2013 AED03852180,3,3,3,3,3,3,3 AED03852181,3,3,3,3,3,3,1 AED09020382,3,0,3,0,3,3,3 AED09020383,1,3,3,3,2,1,3 AED09020386,3,3,0,3,3,0,3 ... (4 Replies)
Discussion started by: zaq1xsw2
4 Replies

3. Shell Programming and Scripting

Filtering

Hi I am interested in DNS resolving a set of sites and each time the output is different- $ host www.yahoo.com www.yahoo.com is an alias for fd-fp3.wg1.b.yahoo.com. fd-fp3.wg1.b.yahoo.com is an alias for ds-fp3.wg1.b.yahoo.com. ds-fp3.wg1.b.yahoo.com is an alias for... (1 Reply)
Discussion started by: jamie_123
1 Replies

4. AIX

Need help with filtering

Hi!! I have a bit of a task here and filtering/scripting not my strongest. I have to collect info of approx 1100 hdiskpower.so i have appended all the hdisk into a text file and i need it to run the command lscfg -vl to confirm if the drive is symmetrix. here's what i have so far at... (3 Replies)
Discussion started by: vpundit
3 Replies

5. UNIX for Dummies Questions & Answers

Filtering F-Dupes

Is there an easy way to tell FDupes what filetypes to look at or ignore? (0 Replies)
Discussion started by: furashgf
0 Replies

6. Shell Programming and Scripting

Filtering files

Hi all, I have some files with different extensions. I want to list the files that doesnt end with particular extension for eg .txt. I want to list all files except .txt. How can I do the same? Thanks Ananth (2 Replies)
Discussion started by: Ananthdoss
2 Replies

7. Shell Programming and Scripting

Replace pipe with Broken Pipe

Hi All , Is there any way to replace the pipe ( | ) with the broken pipe (0xA6) in unix (1 Reply)
Discussion started by: saj
1 Replies

8. Shell Programming and Scripting

Please help me to do some filtering

I have to grep a pattern. scenario is like :- Suppose "/etc/sec/one" is a string, i need to check if this string contains "one" using any utility something like if /etc/sec/one | grep ; then Thanks in advance Renjesh Raju (3 Replies)
Discussion started by: Renjesh
3 Replies

9. UNIX for Dummies Questions & Answers

Filtering mail into a named pipe

Hello, On my machine, all mail is stored in my /var/spool/mail. IS there a way to direct all mail that goes there into a namep pipe? Thank you, Dado (4 Replies)
Discussion started by: dadoprso
4 Replies

10. Shell Programming and Scripting

process filtering

;) Hello, i have written a mail previously but now i have written a script to monitor the status of the unix system process,in which i redirect the out put to file now i have a problem filtering the process that are running and stopped. in fact i want to filter for the processes that have... (6 Replies)
Discussion started by: pradeepmacha
6 Replies
Login or Register to Ask a Question