Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grab exactly one byte from a FIFO, at random intervals

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grab exactly one byte from a FIFO, at random intervals
# 8  
Old 12-25-2012
What about FIFOs being buffered?
# 9  
Old 12-30-2012
Ok, I rewrote the whole script, and came up with this:

Code:
#!/bin/bash

#    Functions:

function makedir() {
    DIR_NAME=${1}
    if test -e "${DIR_NAME}" -a ! -d "${DIR_NAME}" ; then
        printf 'File "'"${DIR_NAME}"'" exists, but is not a directory.\n'
        while test -e "${DIR_NAME}" -a ! -d "${DIR_NAME}" ; do
            printf '\nChoose a different name: '
            read DIR_NAME
        done
        mkdir "${DIR_NAME}"
        printf 'New directory "'"${DIR_NAME}"'" created.\n'
    elif test -d "${DIR_NAME}" ; then
        printf 'Directory "'"${DIR_NAME}"'" already exists.\n[u] Use it\n[c] Create a new one'
        CHOICE='x'
        while test "${CHOICE}" != 'c' -a "${CHOICE}" != 'u' ; do
            printf '\n\tType "c" or "u": '
            read -n 1 CHOICE
        done
        if test "${CHOICE}" = 'c' ; then
            while test -d "${DIR_NAME}" -o -z "${DIR_NAME}" ; do
                printf '\n\tChoose a different name: '
                read DIR_NAME
            done
            mkdir "${DIR_NAME}"
            printf 'New directory "'"${DIR_NAME}"'" created.\n'
        else
            printf '\nUsing directory "'"${DIR_NAME}"'".\n'
        fi
    else
        mkdir "${DIR_NAME}"
        printf 'New directory "'"${DIR_NAME}"'" created.\n'
    fi
}

function makefifo() {
        FIFO_DIR=${1}
        FIFO_NAME='./'"${FIFO_DIR}"'/'"${2}"
        if test ! -p "${FIFO_NAME}" -a -e "${FIFO_NAME}" ; then
                printf 'File "'"${FIFO_NAME}"'" exists, but is not a named pipe.\n'
                while test \( ! -p "${FIFO_NAME}" -a -e "${FIFO_NAME}" \) -o -z "${FIFO_NAME}" ; do
                        printf 'Choose a different name: '
                        read FIFO_NAME
                        FIFO_NAME='./'"${FIFO_DIR}"'/'"${FIFO_NAME}"
                done
                if test -p "${FIFO_NAME}" ; then
                        printf 'Using named pipe "'"${FIFO_NAME}"'".\n'
                else
                        mkfifo "${FIFO_NAME}"
                        printf 'New named pipe "'"${FIFO_NAME}"'" created.\n'
                fi
        elif test -p "${FIFO_NAME}" ; then
                printf 'Using named pipe "'"${FIFO_NAME}"'".\n'
        else
                mkfifo "${FIFO_NAME}"
                printf 'New named pipe "'"${FIFO_NAME}"'" created.\n'
        fi
}

function keep_fifo_open() {
    INPUT=${1}
    KEEP_PAUSE=${2}
    while : ; do
        cat "${KEEP_PAUSE}"
        break
    done > "${INPUT}"
}

function listen() {
    INPUT=${1}
    FIFO=${2}
    LISTEN_PAUSE=${3}
    cat "${INPUT}" | while IFS= read -n 1 BYTE ; do
        printf "${BYTE}" > "${FIFO}"
        EXIT=`cat "${LISTEN_PAUSE}"`
        if test "${EXIT}" = '1' ; then
            break
        fi
    done
}

function yap_randomly() {
    INPUT=${1}
    SLEEP_RANGE=${2}
    while : ; do
        BYTE=`echo "ibase=10;obase=16;(${RANDOM}%256)" | bc`
        printf "\x${BYTE}"
        sleep `echo "(${RANDOM}%${SLEEP_RANGE})" | bc`
    done > "${INPUT}"
}

function grab_bytes() {
    FIFO=${1}
    LISTEN_PAUSE=${2}
    KEEP_PAUSE=${3}
    printf '[p] Print next byte\n[q] Quit\n'
    PROMPT='n'
    while test "${PROMPT}" != 'q' ; do
        printf 'Type "p" or "q": '
        read -n 1 PROMPT
        printf '\n'
        if test "${PROMPT}" = 'p' ; then
            printf 'Next byte is:\n'
            cat "${FIFO}" | xxd -cols 1
            printf '0' > "${LISTEN_PAUSE}"
        fi
    done
    cat "${FIFO}" > /dev/null
    printf '1' > "${LISTEN_PAUSE}"
    printf '1' > "${KEEP_PAUSE}"
}

#    Directories:

PIPES_DIR='pipes'

makedir "${PIPES_DIR}"

PIPES_DIR="${DIR_NAME}"

#    Named pipes:

INPUT='INPUT'
FIFO='FIFO'
KEEP_PAUSE='KEEP_PAUSE'
LISTEN_PAUSE='LISTEN_PAUSE'

makefifo "${PIPES_DIR}" "${INPUT}"
INPUT="${FIFO_NAME}"
makefifo "${PIPES_DIR}" "${FIFO}"
FIFO="${FIFO_NAME}"
makefifo "${PIPES_DIR}" "${KEEP_PAUSE}"
KEEP_PAUSE="${FIFO_NAME}"
makefifo "${PIPES_DIR}" "${LISTEN_PAUSE}"
LISTEN_PAUSE="${FIFO_NAME}"

#    Variables:

MAXDELAY=3

#    Main script:

keep_fifo_open "${INPUT}" "${KEEP_PAUSE}" &
KEEPPID="${!}"
printf '\nBackgrounded keep_fifo_open process...\n'
printf '\tkeep_fifo_open '"${INPUT}"' '"${KEEP_PAUSE}"' &\n'
printf 'PID is: '"${KEEPPID}"'\n'

listen "${INPUT}" "${FIFO}" "${LISTEN_PAUSE}" &
LISTENPID="${!}"
printf '\nBackgrounded listen process...\n'
printf '\tlisten '"${INPUT}"' '"${FIFO}"' '"${LISTEN_PAUSE}"' &\n'
printf 'PID is: '"${LISTENPID}"'\n'

yap_randomly "${INPUT}" "${MAXDELAY}" &
YAPPID="${!}"
printf '\nBackgrounded yap_randomly process...\n'
printf '\tyap_randomly '"${INPUT}"' '"${MAXDELAY}"' &\n'
printf 'PID is: '"${YAPPID}"'\n'

printf '\nBeginning byte-grabbing loop!\n'
grab_bytes "${FIFO}" "${LISTEN_PAUSE}" "${KEEP_PAUSE}"

printf '\nKilling all backgrounded processes...\n'
printf '\tkill -9 '"${KEEPPID}"' '"${LISTENPID}"' '"${YAPPID}"'\n'
kill -9 "${KEEPPID}" "${LISTENPID}" "${YAPPID}"

sleep 1

printf '\nCheck for any unterminated processes...\n'
printf '\tps -e | grep -e '"${KEEPPID}"' -e '"${LISTENPID}"' -e '"${YAPPID}"'\n'
ps -e | grep -e "${KEEPPID}" -e "${LISTENPID}" -e "${YAPPID}"

This exits gracefully, but the problem is, I still get more than 1 byte when requesting 1 byte:

Code:
Beginning byte-grabbing loop!
[p] Print next byte
[q] Quit
Type "p" or "q": p
Next byte is:
0000000: 97  .
Type "p" or "q": p
Next byte is:
0000000: df  .
0000001: 58  X
Type "p" or "q":

I just can't figure out why that happens... it makes no sense to me. Isn't LISTEN_PAUSE supposed to be pausing the listen loop? I don't know...
Last edited by vomv1988; 12-30-2012 at 06:19 PM..
# 10  
Old 12-30-2012
You seemed to have a misunderstanding about how pipe works. A pipe has a typical 4k buffer so you can write more than one byte to it. You tried hard to write one byte a time to the pipe (which is not necessary), but in your grab_a_byte function, you used "cat $INPUT" to drain everything which should be just a byte at a time. Either use "read -n 1" as you did in other places or use "dd bs=1 count=1" to grab one byte from the pipe.
# 11  
Old 12-31-2012
Quote:
Originally Posted by binlib
You seemed to have a misunderstanding about how pipe works. A pipe has a typical 4k buffer so you can write more than one byte to it. You tried hard to write one byte a time to the pipe (which is not necessary), but in your grab_a_byte function, you used "cat $INPUT" to drain everything which should be just a byte at a time. Either use "read -n 1" as you did in other places or use "dd bs=1 count=1" to grab one byte from the pipe.
No, you see: I haven't tried to "write one byte at a time", what I'm trying to do is READ one byte at a time. cat ${INPUT} is not "in the grab_a_byte" function, it's in the listen function. cat ${INPUT} does dump the contents of INPUT, but does not close INPUT afterwards (because of keep_fifo_open), and I did pipe it into while IFS= read -n 1 in the listen function.

---------- Post updated at 05:12 PM ---------- Previous update was at 05:07 PM ----------

In case you didn't notice: the listen function is supposed to dump one byte from ${INPUT} at a time into ${FIFO}; therefor, in grab_bytes, cat ${FIFO} SHOULD output only ONE byte.

---------- Post updated at 05:16 PM ---------- Previous update was at 05:12 PM ----------

The idea is basically to have a data stream, a named pipe in this case, constantly open for input. This data stream has to be available at all times, to be able to receive input. And the script is supposed to be able to grab one byte at a time from this stream, without ever closing it. Those are the conditions I'm trying to satisfy with this script: all of them.

---------- Post updated at 07:02 PM ---------- Previous update was at 05:16 PM ----------

Hmm... interestingly, when I limit yap_randomly to [a-z] bytes, thus:

Code:
function yap_randomly() {
        INPUT=${1}
        SLEEP_RANGE=${2}
        while : ; do
#               BYTE=`echo "ibase=10;obase=16;(${RANDOM}%256)" | bc`
                BYTE=`echo "ibase=10;obase=16;(${RANDOM}%25)+97" | bc`
                printf "\x${BYTE}"
                sleep `echo "(${RANDOM}%${SLEEP_RANGE})" | bc`
        done > "${INPUT}"
}

...the script works as it should. Maybe there is a set of specific bytes (from 0x00 to 0xFF) which cause the script to malfunction.

---------- Post updated 12-31-12 at 03:10 AM ---------- Previous update was 12-30-12 at 07:02 PM ----------

Ok. This pretty much solves the "forbidden bytes" problem:

Code:
function listen() {
        INPUT=${1}
        FIFO=${2}
        LISTEN_PAUSE=${3}
        cat "${INPUT}" | while IFS= read -d '' -n 1 BYTE ; do
                printf "%c" "${BYTE}" > "${FIFO}"
                EXIT=`cat "${LISTEN_PAUSE}"`
                if test "${EXIT}" = '1' ; then
                        break
                fi
        done
}

The script is now perfect. Thanks everyone for your support and constructive criticism.

---------- Post updated at 03:24 AM ---------- Previous update was at 03:10 AM ----------

Here's a final, fully functional, version of the script, in case anybody wants to put it to some good use, or to modify it:

EDIT: The script isn't perfect... the character '\' (0x5C) doesn't get printed for some reason... it's probably getting parsed. Well, I'll look into that some other time. I guess this is enough for now.

EDIT2: read just needs the -r option. NOW it's ready.

EDIT3: nah, not perfect. byte 0xC5 also fails...

Code:
#!/bin/bash

#    Functions:

function makedir() {
    DIR_NAME=${1}
    if test -e "${DIR_NAME}" -a ! -d "${DIR_NAME}" ; then
        printf 'File "'"${DIR_NAME}"'" exists, but is not a directory.\n'
        while test -e "${DIR_NAME}" -a ! -d "${DIR_NAME}" ; do
            printf '\nChoose a different name: '
            read DIR_NAME
        done
        mkdir "${DIR_NAME}"
        printf 'New directory "'"${DIR_NAME}"'" created.\n'
    elif test -d "${DIR_NAME}" ; then
        printf 'Directory "'"${DIR_NAME}"'" already exists.\n[u] Use it\n[c] Create a new one'
        CHOICE='x'
        while test "${CHOICE}" != 'c' -a "${CHOICE}" != 'u' ; do
            printf '\n\tType "c" or "u": '
            read -n 1 CHOICE
        done
        if test "${CHOICE}" = 'c' ; then
            while test -d "${DIR_NAME}" -o -z "${DIR_NAME}" ; do
                printf '\n\tChoose a different name: '
                read DIR_NAME
            done
            mkdir "${DIR_NAME}"
            printf 'New directory "'"${DIR_NAME}"'" created.\n'
        else
            printf '\nUsing directory "'"${DIR_NAME}"'".\n'
        fi
    else
        mkdir "${DIR_NAME}"
        printf 'New directory "'"${DIR_NAME}"'" created.\n'
    fi
}

function makefifo() {
    FIFO_DIR=${1}
    FIFO_NAME='./'"${FIFO_DIR}"'/'"${2}"
    if test ! -p "${FIFO_NAME}" -a -e "${FIFO_NAME}" ; then
        printf 'File "'"${FIFO_NAME}"'" exists, but is not a named pipe.\n'
        while test \( ! -p "${FIFO_NAME}" -a -e "${FIFO_NAME}" \) -o -z "${FIFO_NAME}" ; do
            printf 'Choose a different name: '
            read FIFO_NAME
            FIFO_NAME='./'"${FIFO_DIR}"'/'"${FIFO_NAME}"
        done
        if test -p "${FIFO_NAME}" ; then
            printf 'Using named pipe "'"${FIFO_NAME}"'".\n'
        else
            mkfifo "${FIFO_NAME}"
            printf 'New named pipe "'"${FIFO_NAME}"'" created.\n'
        fi
    elif test -p "${FIFO_NAME}" ; then
        printf 'Using named pipe "'"${FIFO_NAME}"'".\n'
    else
        mkfifo "${FIFO_NAME}"
        printf 'New named pipe "'"${FIFO_NAME}"'" created.\n'
    fi
}

function keep_fifo_open() {
    INPUT=${1}
    KEEP_PAUSE=${2}
    while : ; do
        cat "${KEEP_PAUSE}"
        break
    done > "${INPUT}"
}

function listen() {
# Some bytes, such as 0xC5 don't get stored in BYTE with this
# Use od instead (scroll further down in this post to see how)
    INPUT=${1}
    FIFO=${2}
    LISTEN_PAUSE=${3}
    cat "${INPUT}" | while IFS= read -r -d '' -n 1 BYTE ; do
        printf '%c' "${BYTE}" > "${FIFO}"
        EXIT=`cat "${LISTEN_PAUSE}"`
        if test "${EXIT}" = '1' ; then
            break
        fi
    done
}

function yap_randomly() {
    INPUT=${1}
    SLEEP_RANGE=${2}
    while : ; do
        BYTE=`echo "ibase=10;obase=16;(${RANDOM}%256)" | bc`
        printf "\x${BYTE}" > "${INPUT}"
        sleep `echo "(${RANDOM}%${SLEEP_RANGE})" | bc`
    done
}

function yap_in_sequence() {
    INPUT=${1}
    COUNT=0
    while test "${COUNT}" -lt 256 ; do
        BYTE=`echo "obase=16;ibase=10;${COUNT}" | bc`
        printf "\x${BYTE}" > "${INPUT}"
        COUNT=`expr "${COUNT}" + 1`
    done
}

function grab_bytes() {
    FIFO=${1}
    LISTEN_PAUSE=${2}
    KEEP_PAUSE=${3}
    printf '[p] Print next byte\n[q] Quit\n'
    PROMPT='n'
    while test "${PROMPT}" != 'q' ; do
        printf 'Type "p" or "q": '
        read -n 1 PROMPT
        printf '\n'
        if test "${PROMPT}" = 'p' ; then
            printf 'Next byte is:\n'
            cat "${FIFO}" | xxd -cols 1
            printf '0' > "${LISTEN_PAUSE}"
        fi
    done
    cat "${FIFO}" > /dev/null
    printf '1' > "${LISTEN_PAUSE}"
    printf '1' > "${KEEP_PAUSE}"
}

#    Directories:

PIPES_DIR='pipes'

makedir "${PIPES_DIR}"
PIPES_DIR="${DIR_NAME}"

#    Named pipes:

INPUT='INPUT'
FIFO='FIFO'
KEEP_PAUSE='KEEP_PAUSE'
LISTEN_PAUSE='LISTEN_PAUSE'

makefifo "${PIPES_DIR}" "${INPUT}"
INPUT="${FIFO_NAME}"
makefifo "${PIPES_DIR}" "${FIFO}"
FIFO="${FIFO_NAME}"
makefifo "${PIPES_DIR}" "${KEEP_PAUSE}"
KEEP_PAUSE="${FIFO_NAME}"
makefifo "${PIPES_DIR}" "${LISTEN_PAUSE}"
LISTEN_PAUSE="${FIFO_NAME}"

#    Variables:

MAXDELAY=3

#    Main script:

keep_fifo_open "${INPUT}" "${KEEP_PAUSE}" &
KEEPPID="${!}"
printf '\nBackgrounded keep_fifo_open process...\n'
printf '\tkeep_fifo_open '"${INPUT}"' '"${KEEP_PAUSE}"' &\n'
printf 'PID is: '"${KEEPPID}"'\n'

listen "${INPUT}" "${FIFO}" "${LISTEN_PAUSE}" &
LISTENPID="${!}"
printf '\nBackgrounded listen process...\n'
printf '\tlisten '"${INPUT}"' '"${FIFO}"' '"${LISTEN_PAUSE}"' &\n'
printf 'PID is: '"${LISTENPID}"'\n'

yap_randomly "${INPUT}" "${MAXDELAY}" &
YAPPID="${!}"
printf '\nBackgrounded yap_randomly process...\n'
printf '\tyap_randomly '"${INPUT}"' '"${MAXDELAY}"' &\n'
printf 'PID is: '"${YAPPID}"'\n'

printf '\nBeginning byte-grabbing loop!\n'
grab_bytes "${FIFO}" "${LISTEN_PAUSE}" "${KEEP_PAUSE}"

printf '\nKilling all backgrounded processes...\n'
printf '\tkill -9 '"${KEEPPID}"' '"${LISTENPID}"' '"${YAPPID}"'\n'
kill -9 "${KEEPPID}" "${LISTENPID}" "${YAPPID}"

sleep 1

printf '\nCheck for any unterminated processes...\n'
printf '\tps -e | grep -e '"${KEEPPID}"' -e '"${LISTENPID}"' -e '"${YAPPID}"'\n'
ps -e | grep -e "${KEEPPID}" -e "${LISTENPID}" -e "${YAPPID}"

---------- Post updated at 03:55 PM ---------- Previous update was at 03:24 AM ----------

Well, I wasn't aware of this, but apparenty read forbids a certain set of bytes from being stored in a variable, and dd has trouble receiving newline characters when used thus:

Code:
cat FIFO | while IFS= BYTE="`dd bs=1 count=1`" ; do
       printf "${BYTE}" | xxd -cols 1
done

So, I opted for od, and now the script takes in ANY byte (from 0x00 to 0xFF):

Code:
function listen() {
        INPUT=${1}
        FIFO=${2}
        LISTEN_PAUSE=${3}
        cat "${INPUT}" | while BYTE="`od -An -tx1 -N1 | sed 's/ //g'`" ; do
                printf "\x${BYTE}" > "${FIFO}"
                if test "`cat \"${LISTEN_PAUSE}\"`" = '1' ; then
                        break
                fi
        done
}

Yup... it's pretty much perfect now. Perhaps with a few boundary issues, such as: what happens if someone requesets more bytes than have been inputted into the INPUT named pipe (i.e., pressing "p" one too many times)? Currently, the script just gets stuck indefinitely waiting for more input to be dumped into INPUT; you can "un-stuck" it by shooting more bytes into INPUT from another process, but, well, maybe I should add a timeout or something like that. Anyway: it's finished. You can try it if you want, just remember to substitute the listen function with the one that uses od.
Last edited by vomv1988; 01-01-2013 at 02:59 AM..
# 12  
Old 01-10-2013
here's what I meant
https://github.com/vomv1988/bash_IPC
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Gap length between intervals

hi all, I wish to calculate the length between intervals whose are defined by a starting and an end possition. The data looks like this: 1 10 23 30 45 60 70 100... The desired output should be: 13 # (23-10) 15 # (45-30) 10 # (70-60)... I donīt know how to operate with different... (2 Replies)
Discussion started by: lsantome
2 Replies

2. UNIX for Dummies Questions & Answers

Changing a special line and Byte in a random file

Hello I created 3 files by: dd if=/dev/urandom bs=1024 count=1000000 of=./testfile1 dd if=/dev/urandom bs=1024 count=5000000 of=./testfile2 dd if=/dev/urandom bs=1024 count=10000000 of=./testfile3 Now I want to know how to make a change in a specific byte and/or line of theses files? (2 Replies)
Discussion started by: frhling
2 Replies

3. UNIX for Dummies Questions & Answers

Building intervals

Hi all, I hope you can help me with the following question: I have multiple tables like this: Chr Start End Zygosity Gene chr1 153233510 153233510 het LOR chr1 153233615 153233615 hom LOR chr1 153233701 153233701 hom LOR chr1 ... (5 Replies)
Discussion started by: lsantome
5 Replies

4. UNIX for Dummies Questions & Answers

A crude random byte generator...

There was an upload recently on generating a pseudo-random file when /dev/random does NOT exist. This does not need /dev/random, /dev/urandom or $RANDOM either... (I assume $RANDOM relies on the /dev/random device in some way.) This code uses hexdump just because I like hexdump for ease of... (2 Replies)
Discussion started by: wisecracker
2 Replies

5. Shell Programming and Scripting

Need to generate a file with random data. /dev/[u]random doesn't exist.

Need to use dd to generate a large file from a sample file of random data. This is because I don't have /dev/urandom. I create a named pipe then: dd if=mynamed.fifo do=myfile.fifo bs=1024 count=1024 but when I cat a file to the fifo that's 1024 random bytes: cat randomfile.txt >... (7 Replies)
Discussion started by: Devyn
7 Replies

6. Ubuntu

expect script for random password and random commands

Hi I am new to expect. Please if any one can help on my issue its really appreciable. here is my issue: I want expect script for random passwords and random commands generation. please can anyone help me? Many Thanks in advance (0 Replies)
Discussion started by: vanid
0 Replies

7. Shell Programming and Scripting

Divide numbers into intervals

divide input values into specified number (-100 or -200) according to the key (a1 or a2 ....) For ex: if we give -100 in the command line it would create 100 number intervals (1-100, 100-200, 200-300) untill it covers the value 300 in a1. Note: It should work the same even with huge numbers... (3 Replies)
Discussion started by: ruby_sgp
3 Replies

8. Shell Programming and Scripting

Remove a byte(Last byte from the last line)

Hi All Can anyone please suggest me how to remove the last byte from a falt file .This is from the last line's last BYTE. Please suggest me something. Thank's and regards Vinay (1 Reply)
Discussion started by: vinayrao
1 Replies

9. Shell Programming and Scripting

Check if 2 files are identical byte-to-byte?

In my server migration requirement, I need to compare if one file on old server is exactly the same as the corresponding file on the new server. For diff and comm, the inputs need to be sorted. But I do not want to disturb the content of the file and need to find byte-to-byte match. Please... (4 Replies)
Discussion started by: krishmaths
4 Replies

10. Shell Programming and Scripting

Date Intervals

I posted a question on date intervals about a month back asking about how I could be able to go about a user entering the starting year/month/day and an ending year/month/day and then the script automatically cycling through each day of each month of each year that the user has specified. I... (7 Replies)
Discussion started by: yongho
7 Replies
Login or Register to Ask a Question