Sponsored Content
Top Forums Shell Programming and Scripting Using awk instead of while loop to read file Post 302742973 by Scrutinizer on Wednesday 12th of December 2012 12:59:53 AM
Old 12-12-2012
Hi, it is mainly so slow, because of the way of coding. awk is faster than shell, but what is goiing to make a much bigger difference is that there are many external program calls within the loop, that are called for every line in input2 and also for every line, every CONFLOG*.txt in the directory is grepped.

And all this using intermediate files, pipelines and subshells, so for every iteration of the loop, many subshells are being started...

Could you provide some samples/structure of your input files and desired output? Please use code tags for code, samples and output
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read from a file and use the strings in a loop

Hello all, I need some help to create a script which contain a few strings on every line, and use those strings in a loop to fire some commands. for exmaple the file looks like tom dave bill andy paul I want to read one line at a time and use it in loop like command tom command dave... (3 Replies)
Discussion started by: xboxer21
3 Replies

2. UNIX for Dummies Questions & Answers

How to read a file in unix using do....done loop

Hi , can some give me idea about how to use do...done while loop in UNIX to read the contents of a file.. (2 Replies)
Discussion started by: sreenusola
2 Replies

3. Shell Programming and Scripting

How to Read the entire file using while loop

Guys, I am trying to read the whole file using while loop but when i run the ssh part of the script it reads only the first line and exit after that. There are in total 134 lines in the file, but when the output is redirected, it does only for one line and comes to command prompt. pls help..... (11 Replies)
Discussion started by: sdosanjh
11 Replies

4. Shell Programming and Scripting

Problem reading file in while/read loop

I know I should be able to see a way of doing this easily, but my brain just won't engage. I have a script working on an embedded device that checks to see if an item is in a blacklist before performing some actions. At the moment the code reads thus.... while read BLACKLIST ; do ... (7 Replies)
Discussion started by: Bashingaway
7 Replies

5. Shell Programming and Scripting

Read file using while loop not reading last line

I have written a script to read the file line by line. It is reading and printing the lines. But it is coming out of loop before reading last line. So I am not able to print last line. How do I solve it. (6 Replies)
Discussion started by: dgmm
6 Replies

6. Shell Programming and Scripting

IF awk in a while read line-loop

Hi As a newbe in scripting, i struggle hard with my first script. What i want to do is, bringing data of two files together. file1: .... 05/14/12-04:00:00 41253 4259 5135 5604 5812 5372 05/14/12-04:10:00 53408 5501 6592 7402 7354 6639 05/14/12-04:20:00 58748 6037 7292 8223... (13 Replies)
Discussion started by: IMPe
13 Replies

7. Shell Programming and Scripting

For loop inside awk to read and print contents of files

Hello, I have a set of files Xfile0001 - Xfile0021, and the content of this files (one at a time) needs to be printed between some line (lines start with word "Generated") that I am extracting from another file called file7.txt and all the output goes into output.txt. First I tried creating a for... (5 Replies)
Discussion started by: jaldo0805
5 Replies

8. Shell Programming and Scripting

Do While Loop + Read From File + assign line to a variable

Hello, I am using below code for reading from a file and assigning the values to a variable , but it is loosing the value after the loop , please suggest to retain the value of the variable after the loop , while IFS=: read -r line do set $dsc=$line echo 'printing line variable ' $line... (1 Reply)
Discussion started by: ParthThakkar
1 Replies

9. Shell Programming and Scripting

Failure using regex with awk in 'while read file' loop

I have a file1.txt with several 100k lines, each of which has a column 9 containing one of 60 "label" identifiers. Using an labels.txt file containing a list of labels, I'd like to extract 200 random lines from file1.txt for each of the labels in index.txt. Using a contrived mini-example: $ cat... (8 Replies)
Discussion started by: pathunkathunk
8 Replies

10. Shell Programming and Scripting

Use while loop to read file and use ${file} for both filename input into awk and as string to print

I have files named with different prefixes. From each I want to extract the first line containing a specific string, and then print that line along with the prefix. I've tried to do this with a while loop, but instead of printing the prefix I print the first line of the file twice. Files:... (3 Replies)
Discussion started by: pathunkathunk
3 Replies
DPKG-AWK(1)						      General Commands Manual						       DPKG-AWK(1)

NAME
dpkg-awk - Utility to read a dpkg style db file SYNOPSIS
dpkg-awk [(-f|--file) filename] [(-d|--debug) ##] [(-s|--sort) list] [(-rs|--rec_sep) ??] '<fieldname>:<regex>' ... -- <out_fieldname> .. DESCRIPTION
dpkg-awk Parses a dpkg status file (or other similarly formatted file) and outputs the resulting records. It can use regex on the field values to limit the returned records, it can also be told which fields to output, and it can sort the matched fields. OPTIONS
-f filename --file filename The file to parse. The default is /var/lib/dpkg/status. -d [#] --debug [#] Each time this is specified, it increased the debug level. -s field(s) --sort field(s) A space or comma separated list of fields to sort on. -n field(s) --numeric field(s) A space or comma separated list of fields that should be interpreted as numeric in value. -rs ?? --rec_sep ?? Output this string at the end of each output paragraph. -h --help Display some help. fieldname The fields from the file, that are matched with the regex given. The fieldnames are case insensitive. out_fieldname The fields from the file, that are output for each record. If the first field listed begins with ^, then the list of fields that follows will NOT be output. BUGS
Be warned that the author has only a shallow understanding of the dpkg packaging system, so there are probably tons of bugs in this pro- gram. This program comes with no warranties. If running this program causes fire and brimstone to rain down upon the earth, you will be on your own. This program accesses the dpkg database directly in places, querying for data that cannot be gotten via dpkg. AUTHOR
Adam Heath <doogie@debian.org> DEBIAN
Debian Utilities DPKG-AWK(1)
All times are GMT -4. The time now is 05:17 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy