Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

awk script performance issue

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers awk script performance issue
# 1  
Old 01-28-2014
awk script performance issue
Hello All,
I have the below excerpt of code in my shell script and it taking long time to complete, though it prints the output quickly. Is there a way to make it come out once it finds the first instance as the file size of 4.7 GB it could be going through all lines of the data file to find for any more instances of what it is searching for.


Code:
tr -d '\000' < file_2013-12-31_5_v2.5.xml | awk -v RS="<" -F'"' '/[sS][cC][hH][eE][mM][aA]_[vV][eE][rR][sS][iI][oO][nN]=/ {for(i=1;i<=NF;i++)if($i~/[sS][cC][hH][eE][mM][aA]_[vV][eE][rR][sS][iI][oO][nN]=/) print $(i+1) }'
2.5


Thank you.
# 2  
Old 01-28-2014
Quit after the first instance?
Code:
tr -d '\000' < file_2013-12-31_5_v2.5.xml | awk -v RS="<" -F'"' '/[sS][cC][hH][eE][mM][aA]_[vV][eE][rR][sS][iI][oO][nN]=/ {for(i=1;i<=NF;i++)if($i~/[sS][cC][hH][eE][mM][aA]_[vV][eE][rR][sS][iI][oO][nN]=/) print $(i+1) ; exit }'

This User Gave Thanks to Corona688 For This Post:
Ariean
# 3  
Old 01-29-2014
Quote:
Originally Posted by Corona688
Quit after the first instance?
Code:
tr -d '\000' < file_2013-12-31_5_v2.5.xml | awk -v RS="<" -F'"' '/[sS][cC][hH][eE][mM][aA]_[vV][eE][rR][sS][iI][oO][nN]=/ {for(i=1;i<=NF;i++)if($i~/[sS][cC][hH][eE][mM][aA]_[vV][eE][rR][sS][iI][oO][nN]=/) print $(i+1) ; exit }'

It worked, yes the SCHEMA_VERSION & PROVIDER attributes in XML file will appear at the top of XML only once in the XML data file they sent today, so there is no point in searching further for now. Appreciate your help, thank you.
# 4  
Old 01-29-2014
Corona I think if he is using gawk then this might work

Code:
$ awk 'BEGIN{IGNORECASE=1}/ABCD/' <<EOF 
> abcd
> acdb
> ABCD
> EOF

abcd
ABCD

$ awk 'BEGIN{IGNORECASE=0}/ABCD/' <<EOF 
abcd
acdb
ABCD
EOF

ABCD

This User Gave Thanks to Akshay Hegde For This Post:
Corona688
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Performance Issue - Shell Script

Hi, I am beginner in shell scripting. I have written a script to parse file(s) having large number of lines each having multiple comma separated strings. But it seems like script is very slow. It took more than 30mins to parse a file with size 120MB (523564 lines), below is the script code ... (4 Replies)
Discussion started by: imrandec85
4 Replies

2. Shell Programming and Scripting

Performance issue in shell script

Hi All, I am facing performance issue while rinning the LINUX shell script. I have file1 and file 2. File one is the source file and file 2 is lookup file. Need to replace if the pattern is matching in file1 with file2. The order of lookup file is important as if any match then exit... (8 Replies)
Discussion started by: ureddy
8 Replies

3. Shell Programming and Scripting

awk performance issue

Hi, I have the code below as cat <filename> | tr '~' '\n' | sed '/^$/ d' | sed "s/*/|/g" > <filename> awk -F\| -vt=`date +%m%d%y%H%M%S%s` '$1=="ST",$1=="SE"{if($1=="ST"){close(f);f="214_edifile_"t"" ++i} ; $1=$1; print>f}' OFS=\| <filename> This script replaces some characters and... (4 Replies)
Discussion started by: atlantis_yy
4 Replies

4. UNIX for Dummies Questions & Answers

Performance issue

hi I am having a performance issue with the following requirement i have to create a permutation and combination on a set of three files such that each record in each file is picked and the output is redirected in a specific format but it is taking around 70 odd hours to prepare a combination... (7 Replies)
Discussion started by: mad_man12
7 Replies

5. Shell Programming and Scripting

Script performance issue

hi i have written a shell script which comapare a text file data with files within number of different directories. example. Text File: i have a file /u02/abc.txt which have almost 20000 file names Directories: i have a path /u03 which have some subdirectories like a,b,c which have almost... (2 Replies)
Discussion started by: malikshahid85
2 Replies

6. Solaris

Performance issue

Hi Gurus, I am beginner in solaris and want to know what are the things we need to check for performance monitoring on our solairs OS. for DISK,CPU and MEMORY. Also how we do ipforwarding in slaris Many thanks for your help Pradeep P (4 Replies)
Discussion started by: ppandey21
4 Replies

7. UNIX for Advanced & Expert Users

FTP-Shell Script-Performance issue

Hello All, Request any one of Unix/Linux masters to clarify on the below. How far it is feasible to open a new ftp connection for transferring each file when there are multiple files to be sent. I have developed shell script to send all files at single stretch but some how it doesnt suit to... (3 Replies)
Discussion started by: RSC1985
3 Replies

8. Shell Programming and Scripting

Performance issue with ftp script.

Hi All, I have written a script to FTP files from local server to remote server. When i try it for few number of files the scripts runs successfully. But the same script when i run for 200-300 files it gives me performanace issue by aborting the connection. Please help me out to improve the... (7 Replies)
Discussion started by: Shiv@jad
7 Replies

9. Shell Programming and Scripting

performance issue using gzcat, awk and sort

hi all, I was able to do a script to gather a few files and sort them. here it is: #!/usr/bin/ksh ls *mainFile* |cut -c20-21 | sort > temp set -A line_array i=0 file_name='temp' while read file_line do line_array=${file_line} let i=${i}+1 (5 Replies)
Discussion started by: naoseionome
5 Replies

10. Shell Programming and Scripting

Performance issue with awk script.

Hi, The below awk script is taking about 1 hour to fetch just 11 records(columns). There are about 48000 records. The script file name is take_first_uniq.sh #!/bin/ksh if then while read line do first=`echo $line | awk -F"|" '{print $1$2$3}'` while read line2 do... (4 Replies)
Discussion started by: RRVARMA
4 Replies
Login or Register to Ask a Question