Getting piped input from a program that's buffering it's stdout


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Getting piped input from a program that's buffering it's stdout
# 1  
Old 10-04-2016
Getting piped input from a program that's buffering it's stdout

The old buffering problem again, in a very specific case. On FreeBSD this time, but it's the generic line-buffered vs fully-buffered problem.

I'm trying to pick an available bluetooth speaker (all named audio_N), by pinging all of them en mass and taking the first to respond.
The pinger-utility holds it's output, when it is going into a pipe, so there's a delay before I get the first response.

Can anyone think of a generic solution to make things snappy? I'm not so irritated by the delay, that it would warrant a custom C-program or a patch into l2ping. I have heard of something called expect, but have not used it ever.

Juha

Edit: I didn't think it through. I'll just launch a player to each of them, and one or zero of those succeeds. Meh...


Code:
#!/bin/sh
#
# send 2 pings, 1 second interval, to each given device.
# print out the quickest to respond, if any.
#
# processes might be left running in the background for a while,
# if there are devices that don't respond.
#
# all output from pings goes into a single pipe, it's okay
# as/while l2ping writes line-by-line or everythings at once during exit,
# less than PIPE_BUF per write.
# l2ping does not fflush stdout. takes 1 second always, or more :(

for i in `sed -n "s/.* \($1\).*/\1/p" /etc/bluetooth/hosts`
do
        l2ping -c2 -i1 -a"$i" &
done | sed -n '/.* bytes from \(.*\) seq_no=.* time=.* ms result=0 *$/ {
        s//\1/p
        q
}
'

$1 for the script is audio_. in my case.

Last edited by Juha Nurmela; 10-04-2016 at 01:43 PM..
# 2  
Old 10-04-2016
Nothing outside the program can induce it to print its output early. Some languages like python respond to a particular environment variable which forces them to not print buffered.
# 3  
Old 10-04-2016
I had a peek into the expect, and it does look like there would a mechanism to run the children in a pty, to seduce line-buffering. Very long manpage though.

Juha
# 4  
Old 10-04-2016
Quote:
Originally Posted by Juha Nurmela
I had a peek into the expect, and it does look like there would a mechanism to run the children in a pty, to reduce line-buffering.
Some programs are smart enough to turn off buffering when talking to a terminal or pipe but nothing forces them to. Its a choice the program makes - "I am talking to a terminal, therefore, turn off the internal buffer". It's not something chosen from the outside. It's not even anything system at all. It's a choice between calling write() once, to write blocks of several thousand bytes at a time, or calling write() many times, one for each line. It's a difference in a program's logic.

In short, still entirely up to the program and can't be forced from the outside by any means if the program doesn't already support it somehow.
# 5  
Old 10-04-2016
Yes, that's just what I was after, to run the pings inside a pty, so that they would choose line-buffering.

---------- Post updated at 01:02 AM ---------- Previous update was at 12:57 AM ----------

Hmm.. one-shots and explicit sleeps
Code:
ping -c1 ... &
(sleep 1; ping -c1 ...) &

Gets ugly quickly...
# 6  
Old 10-04-2016
expect won't help. I've looked through l2ping.c and it prints line-buffered anyway, it's not what's holding you up.

I don't think this construct:

Code:
for X in whatever
do
   something &
done | command

...necessarily does what you think. Background commands are liable to freeze themselves when they try to write to the terminal. Make sure they have absolutely no connection to the terminal:

Code:
for X in whatever
do
   something 2>/dev/null </dev/null &
done | command

In particular, sed may be making you wait, too. Make sure you run sed -u.
# 7  
Old 10-04-2016
They write to the pipe, no?

Doing the explicit sleep actually works really snappy. Immediate response!
Argument has to be -fc1, else the program makes an extra sleep after each ping.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Seeing input that you redirect to a program on the shell

Suppose I have a program that I've written that accepts input, ie this C++ program: #include <iostream> using namespace std; int main() { cout << "Enter something:" << endl; int x; cin >> x; cout << "You entered data" << endl; } Suppose that I have a text file,... (5 Replies)
Discussion started by: Chris J
5 Replies

2. Shell Programming and Scripting

Write a shell program with input

Hi, Here is my question: I want a shell script which I name as 'del', and can be used as del(string). when run del(string), it will delete several directories at different locations in my system,like: rm -fr /lustre/fs/scratch/user/$string rm -fr /home/user/$string rm -fr... (4 Replies)
Discussion started by: 1988PF
4 Replies

3. Shell Programming and Scripting

Help with Bash piped while-read and a read user input at the same time

Hi I am new to writing script and want to use a Bash Piped while-read and read from user input. if something happens on server.log then do while loop or if something happend on user input then do while loop. Pseudocode something like: tail -n 3 -f server.log | while read serverline || read... (8 Replies)
Discussion started by: MyMorris
8 Replies

4. Programming

How to use a debugger a piped program ?

Hi, I have 1 program that writes in to the STDIN of another program as shown below. Both programs contain 4 or 5 lines & would terminate in under a second. $ driver.exe | program.exe How is that I can attach the debugger (gdb) to program.exe ? so that I can step through and see what all... (0 Replies)
Discussion started by: RipClaw
0 Replies

5. Shell Programming and Scripting

Using piped input to gzip

Can anyone tell me why does'nt the following command work? find /gfp1/home/arijit -name "*.sas7bdat" | gzip I am trying to compress all files with extension sas7bdat with gzip. It gives error message gzip: compressed data not written to a terminal. Use -f to force compression.... (5 Replies)
Discussion started by: bs0409
5 Replies

6. Shell Programming and Scripting

Shell variable to c++ program as input

I have an Shell script which has few global variables eg : range=100; echo "$range" I want to use the same variable in my C++ program for example int main() { cout << range << "\n"; } i tried using this int main(int argc, char *argv) { cout << range << "\n"; } but... (5 Replies)
Discussion started by: shashi792
5 Replies

7. Shell Programming and Scripting

Piped input to sed 's/\n/ /' doesn't remove the line breaks..

Using ls input as example.. ls | sed 's/\n/ /'outputs with line breaks, where I was expecting the \n to disappear. I've tried \r as well wondering if terminal output used different breaks. Is there a way to remove the line breaks without saving to file and then working from there? ----------... (2 Replies)
Discussion started by: davidpbrown
2 Replies

8. Programming

C++ How to use pipe() & fork() with stdin and stdout to another program

Hi, Program A: uses pipe() I am able to read the stdout of PROGAM B (stdout got through system() command) into PROGRAM A using: * child -> dup2(fd, STDOUT_FILENO); -> execl("/path/PROGRAM B", "PROGRAM B", NULL); * parent -> char line; -> read(fd, line, 100); Question:... (2 Replies)
Discussion started by: vvaidyan
2 Replies

9. Windows & DOS: Issues & Discussions

Windows Buffering during playing

Hello! Will someone out there pls help in clearifying what is really wrong with my system. I use window 98 as my operating system. I am connected to a proxy server for browsing the net. Whenever l am listening to music online l have the problem of intermitted break in which the playing will... (10 Replies)
Discussion started by: kayode
10 Replies

10. Programming

C program help please! input from pipeline

I have a project where I have to use bzcat to uncompress a file and use that output as the data to run another program on. I understand that you would do (bzcat filename.bz2 ! program name) but then how do you access that data in the c program??? Please help thanks (2 Replies)
Discussion started by: kinggizmo
2 Replies
Login or Register to Ask a Question