awk field separator or print command


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk field separator or print command
# 1  
Old 03-19-2009
awk field separator or print command

Hello Experts,

I am back, with another doubt. I am not sure what it relates to this time - awk or the print command actually. I'll explain the scenario:

I have a huge file, and it has some traces(logs). In between those logs, there are statements with some SQL queries. All I want to do is fetch those SQL Queries alone and forget the rest of the part. The file is like:
Code:
blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah
blah blah blah SQL_SUCCESS(0)  blah blah
blah blah blah SQL_SUCCESS(0)  ALTER ................
blah blah blah SQL_SUCCESS(0)  INSERT ................
blah blah blah SQL_SUCCESS(0)  SELECT ................
blah blah blah SQL_SUCCESS(0)  blah blah
blah blah blah SQL_SUCCESS(0)  blah blah
blah blah blah SQL_SUCCESS(0)  ALTER ................
blah blah blah SQL_SUCCESS(0)  blah blah
blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah

1. I am not sure what all SQL queries it might have. I have to fetch all.
2. The file is huge so I am assuming that the queries are always after SQL_SUCCESS(0). But always a query does not follow it as shown.
3. I ve tried something as follows:
Code:
 
#!/bin/sh
cat 20090304_160659.tr1 | grep "SQL_SUCCESS(0)" | awk -F")" '{print $2}' > hTmp.txt
sed "s/^\ *//g" hTmp.txt > hTmp2.txt
cat hTmp2.txt | grep "^[^0-9]" > hTmp.txt

Here the problem is
Code:
awk -F")" '{print $2}'

Suppose the query is like
Code:
SELECT ... FROM (SELECT..... ) WHERE ....

only the following would be fetched:
Code:
SELECT ... FROM (SELECT.....

I would like to know how I shall edit the print in awk to print all that is found after first ")".

Also please tell me if my choice of awk is wrong, if I should ve used cut or something else and if there is a better way to do this.

PS: If you want to see the actual file, I ll post a sample. But I am more interested in the logic here.

Thank You.
Regards,

HKansal
# 2  
Old 03-19-2009
Code:
grep "SQL_SUCCESS(0)" 20090304_160659.tr1 |
 cut =d ')' -f2- |
  sed -n -e 's/^\ *//g' -e '/^[^0-9]/p' > hTmp.txt

# 3  
Old 03-19-2009
Hello,

Thank you Cfajohnson, I ll try that and get back to you.

Still that might be a correct answer to my prob, but please help me learn.

I understand my choice of awk was not good. However, is there a method by which we can get the complete string after the first occurence of the separator?

OR can I do something like awk -F"(0)" '{print....

Thank You.

Regards,
HKansal
# 4  
Old 03-19-2009
Quote:
Originally Posted by hkansal
I understand my choice of awk was not good.
Try this with awk:

Code:
awk '/SQL_SUCCESS\(0\)/ && (/ALTER/||/INSERT/||/SELECT/) {
  sub(".*SQL_SUCCESS\\(0\\)  ",""); print
}' 20090304_160659.tr1

This is the output I get:

Code:
$ cat file
blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah
blah blah blah SQL_SUCCESS(0)  blah blah
blah blah blah SQL_SUCCESS(0)  ALTER ................
blah blah blah SQL_SUCCESS(0)  INSERT ................
blah blah blah SQL_SUCCESS(0)  SELECT ................
blah blah blah SQL_SUCCESS(0)  blah blah
blah blah blah SQL_SUCCESS(0)  blah blah
blah blah blah SQL_SUCCESS(0)  ALTER ................
blah blah blah SQL_SUCCESS(0)  blah blah
blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah
$ awk '/SQL_SUCCESS\(0\)/ && (/ALTER/||/INSERT/||/SELECT/){
  sub(".*SQL_SUCCESS\\(0\\)  ","");print
}' file
ALTER ................
INSERT ................
SELECT ................
ALTER ................
$

Regards
# 5  
Old 04-10-2009
My result

Hello,

Kindly excuse me for the late resonse... I had gone a long vacation Smilie

This is what I ve come up with:
Code:
####################################################################
## 
## Author  : Built with help from Unix.com Forums
## Date    : March 20, 2009
## Description  : Takes a filename as input and extracts all the SQL queries present.
##
#################################################################### 
 
#!/bin/sh
 
# Check for filename
if [ ! $# = 1 ]; then
 echo "USAGE:"
 echo "./getSQL <file_name>\nOR\nsh getSQL <file_name>"
 echo "The <file_name> must be fully qualified, absolutely or relatively,\nwith the location of the file if not in the same folder as shell"
 exit 0
fi
 
srcFile=$1
 
# Check if file exists and has some content
if [ ! -s ${srcFile} ]; then
 echo "The file \"${srcFile}\" does not exist or is empty."
 exit 0
fi 
 
# Parse Filename to remove extension
# Choose a fileName type
#destFile=${srcFile%\.*}   # Does not append file extension to name
destFile=`echo ${srcFile} | sed 's/\.//g'` # Appends file extension to name
destFile="${destFile}_sql.txt"
 
# File has been specified and has content, begin search
# We are assuming that every SQL query is preceded by "SQL_SUCCESS(0)".
# Search each line with SQL_SUCCESS(0) and get evrything after the first ")".
#result=`grep "SQL_SUCCESS(0)" ${srcFile} | cut -d ')' -f2- | sed -n -e 's/^\ *//g' -e '/^[^0-9]/p'`
grep "SQL_SUCCESS(0)" ${srcFile} | cut -d ')' -f2- | sed -n -e 's/^\ *//g' -e '/^[^0-9]/p' > "${destFile}"
 
# Check length of result, give message and exit if 0, else save to file
#if [ ${#result} = 0 ]; then
# echo "No SQL queries found."
# exit 0
#else
# echo ${result}  > "${destFile}_sql.txt"# 
#fi
 
# Get queries 
# Insert a linefeed after every line for clarity
sed '/\.*/G' ${destFile} > hTmp.txt
 
# Move output to required file
mv hTmp.txt ${destFile}
 
echo "Saved to ${destFile}"
 
######################################################################
## End of GET SQL Script
######################################################################

I would really appreciate if somwbody could review the code and comment on it. I would like to learn every bit I can.
Please feel free to scold me if I have committed a silly mistakeSmilie.

After this is done I would like to ask some doubts based on the script.

Thank You

Regards,
HKansal
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk field separator not working

Hi, can some some help to get me the right results, I have few text files, need to grep few columns from each file and get the results in one row with comma separated. my code is #folder=/nz/kit/log/backupsvr folder=/export/home/nz/valai/tmpfiles/ echo $folder for entry in `ls... (4 Replies)
Discussion started by: ValaiG
4 Replies

2. Shell Programming and Scripting

Use two field separator in the same line and print them

Hi Guys, I have the file --- HOST_NAME,data_coleta,data_carga,CPU_util,CPU_idle,run_queue,memory,MEMORY_SYSTEM,MEMORY_TOTAL,MEMORY_SWAPIN,MEMORY_SWAPOUT,DISK_READ,DISK_WRITE,DISK_IO,NET_IN_PACKET, NET_OUT_PACKET... (4 Replies)
Discussion started by: antoniorajr
4 Replies

3. Shell Programming and Scripting

Field Separator in printf (awk)

I can not figure out how to set the Output filed separator in awk when using printf. Example: cat file some data here_is_more information Requested output some------------data her_is_more-----information Here are some that does not work: awk '{printf "%-15s %s\n",$1,$2}' OFS="-" file... (9 Replies)
Discussion started by: Jotne
9 Replies

4. Shell Programming and Scripting

awk field separator help -

Hi Experts , file : - How to construct the awk filed separator so that $1, $2 $3 , can be assigned to the each "" range. I am trying : awk -F"]" '{print $1}' but it is printing the entire file. Not first field. The desired output needed for first field... (9 Replies)
Discussion started by: rveri
9 Replies

5. Shell Programming and Scripting

awk field separator

I need to set awk field separator to ";", but I need to avoid ";EXT". so that echo a;b;c;EXTd;e;f | awk -F";" '{print $3}' would give "c;EXTd" (2 Replies)
Discussion started by: locoroco
2 Replies

6. Shell Programming and Scripting

awk - show field separator

I am using this code to insert something into a csv file: awk -F";" -v url=$url -v nr=$nr 'NR==nr{$2=url$2}1' file Why do I get the output field1 field2 instead of field1;field2 I have given -F";", so the field separator should surely be ";". (1 Reply)
Discussion started by: locoroco
1 Replies

7. Shell Programming and Scripting

awk, comma as field separator and text inside double quotes as a field.

Hi, all I need to get fields in a line that are separated by commas, some of the fields are enclosed with double quotes, and they are supposed to be treated as a single field even if there are commas inside the quotes. sample input: for this line, 5 fields are supposed to be extracted, they... (8 Replies)
Discussion started by: kevintse
8 Replies

8. Shell Programming and Scripting

awk (nawk) field separator

Hi; i have a file and i want to get; - If the last word in line 14 is NOT equal to "Set."; then print 2nd, 3rd, 4th and 5th values of 3rd line. and my code is: nawk 'NR==14 {if ($NF!="Set.") (NR==3{print $2,$3,$4,$5}) }' file.txt but no result?? :confused::(:confused::( (4 Replies)
Discussion started by: gc_sw
4 Replies

9. Shell Programming and Scripting

Field separator in awk

Hi I need to check if field separator I am using in awk statement is " : ", for example: TIME=12:59 HOUR=`echo "$TIME" | awk '{FS=":"; print $1}'` MINUTES=`echo "$TIME" | awk '{FS=":"; print $2}'` Is there a way to check within the above awk statement ? Thanks for help -A (2 Replies)
Discussion started by: aoussenko
2 Replies

10. UNIX for Advanced & Expert Users

Awk command to print the field

894344202808090;11122;040320 075858 166;101;0;0;10u;0;NA;65;221890;2;101973;185059;568674;Y; PRE;0;0;NA;NA;0;NA;0;NA;textmsg;textmsg_snd1;telusmob;TEXTMSG1;0.15000000;126037;2010/03/04 12:58:57gmt;0;70532192; plz tell me any awk command which on the basis of the yellow field which is... (1 Reply)
Discussion started by: madfox
1 Replies
Login or Register to Ask a Question