Just getting rid of the grep should save you some cycles. Also reading the result into backticks and then looping over the result is wasteful (although some shells probably optimize that into a loop internally).
Code:
awk -F'|' "/^\[$STARTHOUR/"'{print $4}' ANYOLDFILE |
while read results; do
stuff
done
If you can implement "stuff" in awk you are probably as fast as this is going to get, unless of course you are in a position to sit down and write your own little C program. Anyway, the "stuff" part is probably responsible for most of the processing time, not the loop scaffolding around it.
I need an assistance in file generation using awk, sed or anything...
I have a big file that i need to filter desired parts only. The objective is to select (and print) the report # having the string "apple" on 2 consecutive lines in every report. Please note that the "apple" line has a HEX... (1 Reply)
Hi,
This is my first time post a new thread. I have been trying to work on this for the past 2 days and could not find any good solution.
I have 1 long long line ( EDI wrapped file) like below:
NEW*SR*04411763447*279*278*Q~*ZR*AAV*SR*04511763460*SQ*21B37F04~HL*305*304*Q~K~SN1*1*1*SR*05511763461*... (6 Replies)
Hi, I want to get data from Xml file by using sed or awk command. I want to get the following result :
mon titre 1;Createur1;Dossier1
mon titre 1;Createur1;Dossier1
and save it in cvs file (fichier.cvs).
FROM this Xml file (test.xml):
<playlist version="1">
<trackList>
<track>... (1 Reply)
How to cut data from big file
my file around 30 gb
I tried "head -50022172 filename > newfile.txt ,and tail -5454283 newfile.txt. It's slowy.
afer that I tried sed -n '46467831,50022172p' filename > newfile.txt ,also slow
Please recommend me , faster command to cut some data from... (4 Replies)
Hi everyone,
I cannot figure out how I can do a search in a file that has Names, Surnames, Addresses and telephone number of a number of people.
Here is an example of the data file
Daisy:Hunter:490 London Road:07313196347
Richard:Murphy:983 Main Road:07002625997
Isobel:Magnusson:133 London... (1 Reply)
Hello,
I have a big data file (160 MB) full of records with pipe(|) delimited those fields. I`m sorting the file on the first field.
I'm trying to sort with "sort" command and it brings me 6 minutes.
I have tried with some transformation methods in perl but it results "Out of memory". I was... (2 Replies)
Hi,
I am new to this world..
Using expect i loging to router and checking ping response to my links.
I need to genarate report using this output and that report contains only three file link name, packet loss, latency.
my output of script is like below:
-bash-3.00$ monmw/mwbkp... (2 Replies)
Hi everyone,
I am trying to write a bash script which reads a data file and does some algebraic operations.
here is the structure of data.xml file that I have;
1 <data>
2 .
3 .
4 .
5 </data>
6 <data>
7 .
8 .
9 .
10</data>
etc.
Each data block contains same number of lines (say... (4 Replies)
Below is the output of a DB2 command. Now I have 2 requirements...
Database Partition 0 -- Database TESTDB1 -- Active Standby -- Up 213 days 02:33:07 -- Date 02/22/2016 17:04:50
HADR Information:
Role State SyncMode HeartBeatsMissed LogGapRunAvg (bytes)
Standby ... (2 Replies)
Discussion started by: rlokesh27
2 Replies
LEARN ABOUT CENTOS
igawk
IGAWK(1) Utility Commands IGAWK(1)NAME
igawk - gawk with include files
SYNOPSIS
igawk [ all gawk options ] -f program-file [ -- ] file ...
igawk [ all gawk options ] [ -- ] program-text file ...
DESCRIPTION
Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).
AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like
@include getopt.awk
in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.
OPTIONS
See gawk(1) for a full description of the AWK language and the options that gawk supports.
EXAMPLES
cat << EOF > test.awk
@include getopt.awk
BEGIN {
while (getopt(ARGC, ARGV, "am:q") != -1)
...
}
EOF
igawk -f test.awk
SEE ALSO gawk(1)
Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.
AUTHOR
Arnold Robbins (arnold@skeeve.com).
Free Software Foundation Nov 3 1999 IGAWK(1)