Grab exactly one byte from a FIFO, at random intervals


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grab exactly one byte from a FIFO, at random intervals
# 1  
Old 12-23-2012
Grab exactly one byte from a FIFO, at random intervals

I want to develop a script of the following form:

Code:
#!/bin/bash

# Function 'listen' opens a data stream
# which stores all incoming bytes in
# a buffer, preparing them to be
# grabbed by a following function
# which appears at random
# intervals during the execution of
# the script

listen &
INPID=${!}

# To test if the script is really capable
# of grabbing bytes randomly, the
# following code is looped over:

PROMPT=n
while test ${PROMPT} != 'q' ; do
        printf 'y: print next byte\nn: don'"'"'t print next byte\nq: quit\n'
        read -n 1 PROMPT
        printf '\n'
        if test ${PROMPT} = 'y' ; then

# The function 'grab1byte' extracts
# ONE byte from the buffer being fed
# bytes by 'listen', and outputs it
# to stdout

                INBYTE=`grab1byte`
                echo The input byte is:
                printf "${INBYTE}" | xxd -cols 1 | sed 's/^.*: //'
        fi
done

kill ${INPID}

The way I've tried to implement this, is by using FIFOs. In one directory, I have 2 FIFOs, which I created with mkfifo; these are named 'INPUT' and 'FIFO'. In fluxbox, I open 2 instances of xterm; in one I run the following script:

Code:
#!/bin/bash

MY_INPUT=INPUT
MY_FIFO=FIFO

# This is 'listen':
(
        IFS=
        tail -f ${MY_INPUT} | while read -N 1 CHAR ; do
                printf "${CHAR}" > ${MY_FIFO}
        done
) &
READPID=${!}

PROMPT=n
while test ${PROMPT} != 'q' ; do
        printf 'y: print next byte\nn: don'"'"'t print next byte\nq: quit\n'
        read -n 1 PROMPT
        printf '\n'
        if test ${PROMPT} = 'y' ; then
# This is 'grab1byte':
                INBYTE=`cat ${MY_FIFO}`
                echo The input byte is:
                printf "${INBYTE}" | xxd -cols 1 | sed 's/^.*: //'
        fi
done

kill ${READPID}

On the other one, I run:

Code:
printf 'Hello, world!' > INPUT

Then, back on the first terminal, I type 'y' to the prompt, to test the bytegrabbing. The problem is: instead of getting only 1 byte, I sometimes get 1, 2, 3, 4, 5 bytes. A typical session looks something like:

Code:
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
48  H
65  e
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
6c  l
6c  l
6f  o
2c  ,
20   
77  w
y: print next byte
n: don't print next byte
q: quit

But, what I want is something like:

Code:
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
48  H
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
 65  e
y: print next byte
n: don't print next byte
q: quit

The mystery is: Why does FIFO spit out 2 or 4 bytes at a time, if I am only writing ONE byte at each iteration of the loop??:

Code:
        IFS=
        tail -n 1 -f ${MY_INPUT} | while read -N 1 CHAR ; do
                printf "${CHAR}" > ${MY_FIFO}
        done

The code seems to work with a 'sleep' delay of 0.2 right after 'printf "${CHAR}" > ${MY_FIFO}'. But... why?

In order for this script to be perfect, I would require it to ONLY use FIFOs: No ugly and slow hard-drive file buffers, please. And also, NO ugly time delays.

Another funny thing is how, when I run in one terminal:

Code:
( IFS= ; tail -f FIFO | while read -N 1 CHAR ; do printf "${CHAR}" | xxd -cols 1 ; printf '..\n' ; done )

And, from another, I do

Code:
printf 'Hello, world!' > FIFO

I get:

Code:
0000000: 48  H
..
0000000: 65  e
..
0000000: 6c  l
..
0000000: 6c  l
..
0000000: 6f  o
..
0000000: 2c  ,
..
0000000: 20   
..
0000000: 77  w
..
0000000: 6f  o
..
0000000: 72  r
..
0000000: 6c  l
..
0000000: 64  d
..
0000000: 21  !
..

Which goes to show that ${CHAR} never stores more than 1 byte at any given time. If it did, the output would look more like:

Code:
0000000: 48  H
0000001: 65  e
0000002: 6c  l
0000003: 6c  l
..
0000000: 6f  o
0000001: 2c  ,
0000002: 20   
..
0000000: 77  w
0000001: 6f  o
0000002: 72  r
0000003: 6c  l
0000004: 64  d
0000005: 21  !
..

So... my question is... basically: What is the deal with this FIFO glitch? If the problem is not in the loop, then: Where is it?

Last edited by vomv1988; 12-23-2012 at 01:52 AM..
# 2  
Old 12-23-2012
This might sound stupid, but can you replace this section:
Code:
# This is 'grab1byte':
                INBYTE=`cat ${MY_FIFO}`
                echo The input byte is:
                printf "${INBYTE}" | xxd -cols 1 | sed 's/^.*: //'

with
Code:
# This is 'grab1byte':
                echo The input byte is:
                xxd -cols 1 ${MY_FIFO} | sed 's/^.*: //'

and give it another try? Let us know the result please.
# 3  
Old 12-23-2012
Quote:
Originally Posted by bipinajith
This might sound stupid, but can you replace this section:
Code:
# This is 'grab1byte':
                INBYTE=`cat ${MY_FIFO}`
                echo The input byte is:
                printf "${INBYTE}" | xxd -cols 1 | sed 's/^.*: //'

with
Code:
# This is 'grab1byte':
                echo The input byte is:
                xxd -cols 1 ${MY_FIFO} | sed 's/^.*: //'

and give it another try? Let us know the result please.
The output is the same with that change, but thanks for trying anyway. After removing the sed filter from the original script I posted, the output looks something like:

Code:
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
0000000: 48  H
0000001: 65  e
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
0000000: 6c  l
0000001: 6c  l
0000002: 6f  o
0000003: 2c  ,
0000004: 20   
0000005: 77  w
y: print next byte
n: don't print next byte
q: quit

So this means that the FIFO buffer contains several bytes instead of just one... Which is weird, because, I thought 'printf "${CHAR}" > ${MY_FIFO}"' was supposed to pause the loop execution, UNTIL FIFO was emptied by something like 'cat ${MY_FIFO}'. After cat-ing FIFO, I thought the loop would freeze again at 'printf "${CHAR}" > ${MY_FIFO}"', until another instance of 'cat ${MY_FIFO}', but, apparently it doesn't. Apparently, it just feeds FIFO a random amount of bytes... WHY???

---------- Post updated at 04:52 PM ---------- Previous update was at 04:44 PM ----------

I'm thinking, maybe, cat opens up the FIFO for a longer time than it takes the loop to iterate, so the loop iterates several times, spitting several bytes into FIFO, until cat stops reading from FIFO (that is, until FIFO's output is closed)... Does this make any sense to you? And, if that is the case: How would I prevent that from happening?
# 4  
Old 12-23-2012
I don't know what magic it is but I'm getting it right!!
Code:
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
48  H
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
65  e
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
6c  l
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
6c  l
y: print next byte
n: don't print next byte
q: quit
y
The input byte is:
6f  o
y: print next byte
n: don't print next byte
q: quit
q

I tested it on GNU/Linux and shell /bin/bash

I recommend you to run ps and verify if you have any previous instance of this script & tail command running because I see the listening section is not getting gracefully terminated after selecting option q and I see this is causing the behavior that you mentioned.
Code:
# ps
  PID TTY          TIME CMD
 4787 pts/0    00:00:00 tail
 4788 pts/0    00:00:00 tfifo.sh
26804 pts/0    00:00:00 ps
29197 pts/0    00:00:00 bash

This User Gave Thanks to Yoda For This Post:
# 5  
Old 12-23-2012
I'm currently using bash in a mac, but I was testing the code from the original post also in GNU/Linux, in bash too.

I see how several unterminated instances of the 'listen' section would cause the behavior. I used ps to check, and killed all undesired instances of the listen section, but that didn't seem to solve it. So I split the script into two scripts:

listen.sh
Code:
#!/bin/bash

# Listen

	IFS=
	tail -f INPUT | while read -n 1 CHAR ; do
		printf "${CHAR}" > FIFO
#		sleep 0.2
	done

EDIT: The behavior only happens without the sleep delay. But I would like to have a working script that does not require the sleep delay.

prompt.sh
Code:
#!/bin/bash

PROMPT=n
while test ${PROMPT} != 'q' ; do
	printf 'y: print next byte\nn: don'"'"'t print next byte\nq: quit\n'
	read -n 1 PROMPT
	printf '\n'
	if test "${PROMPT}" = 'y' ; then

# Grab 1 byte

		BYTE=`cat FIFO`
		printf 'The input byte is:\n'
		printf "${BYTE}" | xxd -cols 1
	fi
done

So, on one terminal, I run ./listen.sh, and I leave it there, on a second one, I run ./prompt.sh, and leave it there also, and finally, on a third terminal, I run "printf 'Hello, world!' > INPUT".

I go back to the second terminal, and type 'y', and get the same undesired behavior. This time, I made sure there were no undesired unterminated instances of the listen section. bipinajith: Are you using the exact same code from my original post, or did you add any changes to yours?
# 6  
Old 12-23-2012
Here is the code:
Code:
#!/bin/bash

MY_INPUT=INPUT
MY_FIFO=FIFO

(
tail -f ${MY_INPUT} | while IFS= read -n 1 CHAR ; do
        printf "${CHAR}" > ${MY_FIFO}
done
) &
READPID=$!

PROMPT=n
while [ "${PROMPT}" != "q" ]
do
        printf 'y: print next byte\nn: don'"'"'t print next byte\nq: quit\n'
        read -n 1 PROMPT
        printf '\n'
        if [ "${PROMPT}" = "y" ]
        then
# This is 'grab1byte':
                echo The input byte is:
                xxd -cols 1 ${MY_FIFO} | sed 's/^.*: //'
        fi
done

kill $READPID

# 7  
Old 12-25-2012
Quote:
Originally Posted by bipinajith
Here is the code:
Code:
#!/bin/bash

MY_INPUT=INPUT
MY_FIFO=FIFO

(
tail -f ${MY_INPUT} | while IFS= read -n 1 CHAR ; do
        printf "${CHAR}" > ${MY_FIFO}
done
) &
READPID=$!

PROMPT=n
while [ "${PROMPT}" != "q" ]
do
        printf 'y: print next byte\nn: don'"'"'t print next byte\nq: quit\n'
        read -n 1 PROMPT
        printf '\n'
        if [ "${PROMPT}" = "y" ]
        then
# This is 'grab1byte':
                echo The input byte is:
                xxd -cols 1 ${MY_FIFO} | sed 's/^.*: //'
        fi
done

kill $READPID

I tried this very code and I am still getting the undesired behavior from my original post. It's funny how the amount of bytes I get from FIFO seems to be system-dependent; more specifically, it seems hardware-dependent. When I tried the code in a very fast mac, with bash, cat-ing FIFO dumped the whole string 'Hello, world!' to stdout. On my (slower) yeeloong notebook, cat-ing FIFO only dumps the first few bytes of the string, and then a few more bytes, etc. Maybe the software in a macbook is designed differently, although the shell is the same (bash), I don't know. I have no clue as to why this happens. This is just so strange.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Gap length between intervals

hi all, I wish to calculate the length between intervals whose are defined by a starting and an end possition. The data looks like this: 1 10 23 30 45 60 70 100... The desired output should be: 13 # (23-10) 15 # (45-30) 10 # (70-60)... I donīt know how to operate with different... (2 Replies)
Discussion started by: lsantome
2 Replies

2. UNIX for Dummies Questions & Answers

Changing a special line and Byte in a random file

Hello I created 3 files by: dd if=/dev/urandom bs=1024 count=1000000 of=./testfile1 dd if=/dev/urandom bs=1024 count=5000000 of=./testfile2 dd if=/dev/urandom bs=1024 count=10000000 of=./testfile3 Now I want to know how to make a change in a specific byte and/or line of theses files? (2 Replies)
Discussion started by: frhling
2 Replies

3. UNIX for Dummies Questions & Answers

Building intervals

Hi all, I hope you can help me with the following question: I have multiple tables like this: Chr Start End Zygosity Gene chr1 153233510 153233510 het LOR chr1 153233615 153233615 hom LOR chr1 153233701 153233701 hom LOR chr1 ... (5 Replies)
Discussion started by: lsantome
5 Replies

4. UNIX for Dummies Questions & Answers

A crude random byte generator...

There was an upload recently on generating a pseudo-random file when /dev/random does NOT exist. This does not need /dev/random, /dev/urandom or $RANDOM either... (I assume $RANDOM relies on the /dev/random device in some way.) This code uses hexdump just because I like hexdump for ease of... (2 Replies)
Discussion started by: wisecracker
2 Replies

5. Shell Programming and Scripting

Need to generate a file with random data. /dev/[u]random doesn't exist.

Need to use dd to generate a large file from a sample file of random data. This is because I don't have /dev/urandom. I create a named pipe then: dd if=mynamed.fifo do=myfile.fifo bs=1024 count=1024 but when I cat a file to the fifo that's 1024 random bytes: cat randomfile.txt >... (7 Replies)
Discussion started by: Devyn
7 Replies

6. Ubuntu

expect script for random password and random commands

Hi I am new to expect. Please if any one can help on my issue its really appreciable. here is my issue: I want expect script for random passwords and random commands generation. please can anyone help me? Many Thanks in advance (0 Replies)
Discussion started by: vanid
0 Replies

7. Shell Programming and Scripting

Divide numbers into intervals

divide input values into specified number (-100 or -200) according to the key (a1 or a2 ....) For ex: if we give -100 in the command line it would create 100 number intervals (1-100, 100-200, 200-300) untill it covers the value 300 in a1. Note: It should work the same even with huge numbers... (3 Replies)
Discussion started by: ruby_sgp
3 Replies

8. Shell Programming and Scripting

Remove a byte(Last byte from the last line)

Hi All Can anyone please suggest me how to remove the last byte from a falt file .This is from the last line's last BYTE. Please suggest me something. Thank's and regards Vinay (1 Reply)
Discussion started by: vinayrao
1 Replies

9. Shell Programming and Scripting

Check if 2 files are identical byte-to-byte?

In my server migration requirement, I need to compare if one file on old server is exactly the same as the corresponding file on the new server. For diff and comm, the inputs need to be sorted. But I do not want to disturb the content of the file and need to find byte-to-byte match. Please... (4 Replies)
Discussion started by: krishmaths
4 Replies

10. Shell Programming and Scripting

Date Intervals

I posted a question on date intervals about a month back asking about how I could be able to go about a user entering the starting year/month/day and an ending year/month/day and then the script automatically cycling through each day of each month of each year that the user has specified. I... (7 Replies)
Discussion started by: yongho
7 Replies
Login or Register to Ask a Question