Sponsored Content
Top Forums Shell Programming and Scripting Incredibly inefficient cat | grep script Post 302913203 by Corona688 on Thursday 14th of August 2014 12:34:30 PM
Old 08-14-2014
This replaces the eval with a safer text replacement loop. I've also turned fname into FNAME since I'm exporting it to the environment as an afterthought. Otherwise your code looks terrific.

Code:
#!/bin/bash
lines=1000
ccount=0

# Convert "command" "-flag" "@FNAME@" into "command" "-flag" "000001"
# uses $FNAME external.  Result is in ARGS array.
parsecmd() {
        local N=0
        ARGS=()
        while [ "$#" -gt 0 ]
        do
                ARGS[$((N++))]="${1//@FNAME@/${FNAME}}"
                shift
        done
}

while getopts l: opt 2> /dev/null
do
    case $opt in
        l) lines=$OPTARG ;;
        *) echo "Illegal option -$opt" >&2
           exit 1
        ;;
    esac
done
shift $((OPTIND-1))

if [ $# -le 0 ]
then
    cat >&2 <<EOF
linesplit:  Designed to read from standard input
            and split into multiple streams, running one command per loop
            and writing into its STDIN.  @FNAME@ is available as an
            sequence number if needed.

syntax:  linesplit [-l lines] command arguments @FNAME@ ...
EOF
    exit 1
fi

while [[ ${PIPESTATUS[0]} != 33 ]]
do
    ((ccount++))
    printf -v FNAME "%08d" $ccount
    export FNAME
    parsecmd "$@" # Convert @FNAME@ into $FNAME, put in ${ARGS[@]}

    for((n=0; n<lines; n++))
    do
        read line && echo "$line" || exit 33
    done | "${ARGS[@]}"
done

printf "Wrote %'d full chunks of %'d lines\n" $((ccount-1)) $lines >&2

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

cat and grep not working

I am trying to cat a file and then grep that file for a number. I can do it fine on other files but this particular file will not do anything. I tried running it on an older file from the same device but it is just not working. The file is nothing more than a flat file on a unix box. Here is just a... (3 Replies)
Discussion started by: jphess
3 Replies

2. Shell Programming and Scripting

cat in the command line doesn't match cat in the script

Hello, So I sorted my file as I was supposed to: sort -n -r -k 2 -k 1 file1 | uniq > file2 and when I wrote > cat file2 in the command line, I got what I was expecting, but in the script itself ... sort -n -r -k 2 -k 1 averages | uniq > temp cat file2 It wrote a whole... (21 Replies)
Discussion started by: shira
21 Replies

3. Shell Programming and Scripting

Perl sum really inefficient!!

Hi all, I have a file like the following: ID, 2,Andrew,0,1,2,3,4,2,5,6,7,7,9,3,4,5,34,3,2,1,5,6,78,89,8,7,6...................... 4,James,0,6,7,0,5,6,4,7,8,9,6,46,6,3,2,5,6,87,0,341,0,5,2,5,6.................... END, (there are more entires on each line but to keep it simple I've left... (10 Replies)
Discussion started by: Donkey25
10 Replies

4. Shell Programming and Scripting

Problem with IF - CAT - GREP in simple shell script

Hi all, Here is my requirement I have to search 'ORA' word in out.log file,if it is present then i need to send that file (out.log) content to some mail id.If 'ORA' word is not in that file then i need to send 'load succesful' message to some mail id. The below the shell script is not... (5 Replies)
Discussion started by: mak_boop
5 Replies

5. Shell Programming and Scripting

cat /etc/passwd and grep -v on /etc/shells

Hi All, I'd like to do this cat /etc/passwd and grep -v on the /etc/shells list I'd like to find all shell that doesn't exist on the /etc/passwd. Is there an easy way without doing a egrep -v "/bin/sh|/bin/bash................"? How do I use a file /etc/shells as my list for... (4 Replies)
Discussion started by: itik
4 Replies

6. Shell Programming and Scripting

cat -n and grep

I am not sure if using cat -n is the most efficient way to split a file into multiple files, one file per line in the source file. I thought using cat -n would make it easy to process the file because it produces an output that numbers each line that I could then grep for with the regex "^ *$i".... (3 Replies)
Discussion started by: kapu
3 Replies

7. Shell Programming and Scripting

grep or cat using sed

Is there a way using grep or cat a file to create a new file based on whether the first 9 positions of each record is less than 399999999? This is a fixed file format. (3 Replies)
Discussion started by: ski
3 Replies

8. UNIX for Dummies Questions & Answers

Grep and cat combined

Hello, i need to search one word (snp1) from many files and copy the content of the columns of this word in new file. example: file 1: SNP BP CHR P snp1 1 3 0.01 snp2 2 2 0.05 . . file 2: SNP BP CHR P snp1 1 3 0.06 snp2 2 2 0.3 output... (6 Replies)
Discussion started by: biopsy
6 Replies

9. Shell Programming and Scripting

Replace cat and grep with <

Hello someone told me to use OS=`awk '{print int($3)}' < /etc/redhat-release` instead of OS=cat /etc/redhat-release | `awk '{print int($3)}'` any idea for the reason ? (5 Replies)
Discussion started by: nimafire
5 Replies

10. UNIX for Dummies Questions & Answers

Grep or cat The Whole Directory PROBLEMS :(

Hi Guys This is my first post so I am not sure how things go here. I'm sorry if I'm breaking the rule or something. Feel free to correct me about that :) So as I was saying... I'd been trying to grep this folder containing 900,000 txt files but seems no luck. I get either "No such file... (6 Replies)
Discussion started by: Nexeu
6 Replies
getoptcvt(1)                                                       User Commands                                                      getoptcvt(1)

NAME
getoptcvt - convert to getopts to parse command options SYNOPSIS
/usr/lib/getoptcvt [-b] filename /usr/lib/getoptcvt DESCRIPTION
/usr/lib/getoptcvt reads the shell script in filename, converts it to use getopts instead of getopt, and writes the results on the standard output. getopts is a built-in Bourne shell command used to parse positional parameters and to check for valid options. See sh(1). It supports all applicable rules of the command syntax standard (see Rules 3-10, intro(1)). It should be used in place of the getopt command. (See the NOTES section below.) The syntax for the shell's built-in getopts command is: getopts optstring name [ argument...] optstring must contain the option letters the command using getopts will recognize; if a letter is followed by a colon (:), the option is expected to have an argument, or group of arguments, which must be separated from it by white space. Each time it is invoked, getopts places the next option in the shell variable name and the index of the next argument to be processed in the shell variable OPTIND. Whenever the shell or a shell script is invoked, OPTIND is initialized to 1. When an option requires an option-argument, getopts places it in the shell variable OPTARG. If an illegal option is encountered, ? will be placed in name. When the end of options is encountered, getopts exits with a non-zero exit status. The special option -- may be used to delimit the end of the options. By default, getopts parses the positional parameters. If extra arguments (argument ...) are given on the getopts command line, getopts parses them instead. So that all new commands will adhere to the command syntax standard described in intro(1), they should use getopts or getopt to parse posi- tional parameters and check for options that are valid for that command (see the NOTES section below). OPTIONS
The following option is supported: -b Makes the converted script portable to earlier releases of the UNIX system. /usr/lib/getoptcvt modifies the shell script in file- name so that when the resulting shell script is executed, it determines at run time whether to invoke getopts or getopt. EXAMPLES
Example 1: Processing the arguments for a command The following fragment of a shell program shows how one might process the arguments for a command that can take the options -a or -b, as well as the option -o, which requires an option-argument: while getopts abo: c do case $c in a | b) FLAG=$c;; o) OARG=$OPTARG;; ?) echo $USAGE exit 2;; esac done shift `expr $OPTIND - 1` Example 2: Equivalent code expressions This code accepts any of the following as equivalent: cmd -a -b -o "xxx z yy" filename cmd -a -b -o "xxx z yy" -filename cmd -ab -o xxx,z,yy filename cmd -ab -o "xxx z yy" filename cmd -o xxx,z,yy b a filename ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of getopts: LC_CTYPE, LC_MESSAGES, and NLSPATH. OPTIND This variable is used by getoptcvt as the index of the next argument to be processed. OPTARG This variable is used by getoptcvt to store the argument if an option is using arguments. EXIT STATUS
The following exit values are returned: 0 An option, specified or unspecified by optstring, was found. >0 The end of options was encountered or an error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |CSI |enabled | +-----------------------------+-----------------------------+ SEE ALSO
intro(1), getopts(1), sh(1), shell_builtins(1), getopt(3C), attributes(5) DIAGNOSTICS
getopts prints an error message on the standard error when it encounters an option letter not included in optstring. NOTES
Although the following command syntax rule (see intro(1)) relaxations are permitted under the current implementation, they should not be used because they may not be supported in future releases of the system. As in the EXAMPLES section above, -a and -b are options, and the option -o requires an option-argument. The following example violates Rule 5: options with option-arguments must not be grouped with other options: example% cmd -aboxxx filename The following example violates Rule 6: there must be white space after an option that takes an option-argument: example% cmd -ab oxxx filename Changing the value of the shell variable OPTIND or parsing different sets of arguments may lead to unexpected results. SunOS 5.10 7 Jan 2000 getoptcvt(1)
All times are GMT -4. The time now is 02:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy