Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Select a portion of file based on query

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Select a portion of file based on query
# 8  
Old 10-26-2006
Quote:
Originally Posted by ghostdog74
Sample input:
Code:
header
xxyy
lmno
xxyy
wxyz
footer

header
abcd
pqrs
footer

header
fklsdjsdfalgj
xy
sd;lfks;lkfsd
footer


Python alternative:

Code:
import re
data = open("input.txt").read()
pattern = re.compile(r"header(.*?)footer",re.M|re.DOTALL)
for i in pattern.findall(data):
 	if i.find("xy") != -1:
 		print i

output:
Code:
xxyy
lmno
xxyy
wxyz


fklsdjsdfalgj
xy
sd;lfks;lkfsd


Thanx Friend
But I have to execute the program at unix system..
some script in unix shell scripting r invited awk etc...
Please help me out
thanx in advance
# 9  
Old 10-26-2006
hi there
if in awk, etc have you tried the awk solution posted by anbu23?
# 10  
Old 10-26-2006
Quote:
Originally Posted by vanand420
Thanks for the reply friend Smilie
But where to specify my fixed header and fixed footer in the script
In the file there are several such header and footer...as in log files...
How to select one or more such block based on search query...
You don't need to specify header and footer in script. Above code will work if each record is separated by blank line.

To select one or more such blocks

Code:
awk -v var="xy pc" ' BEGIN { RS = "" ; FS = "\n"
n=split(var,arr," ")}
{ flag=0;
for( i = 1 ; i <= NF; ++i )
{
for( j = 1 ; j <= n ; ++j )
if(match ( $i , "^"arr[j]"$" ))
        {  flag=1;break;  }
}
if( flag == 1 ) print
printf("\n")
} ' file

In above code it will print blocks with xy or pc. But if you have spaces in the fields which you specify to select then you can use some other separator like this
Code:
var="xy:pc"

If you change the separator then you have to make same changes in code
Code:
n=split(var,arr,":")

# 11  
Old 10-30-2006
Quote:
Originally Posted by anbu23
You don't need to specify header and footer in script. Above code will work if each record is separated by blank line.

To select one or more such blocks

Code:
awk -v var="xy pc" ' BEGIN { RS = "" ; FS = "\n"
n=split(var,arr," ")}
{ flag=0;
for( i = 1 ; i <= NF; ++i )
{
for( j = 1 ; j <= n ; ++j )
if(match ( $i , "^"arr[j]"$" ))
        {  flag=1;break;  }
}
if( flag == 1 ) print
printf("\n")
} ' file

In above code it will print blocks with xy or pc. But if you have spaces in the fields which you specify to select then you can use some other separator like this
Code:
var="xy:pc"

If you change the separator then you have to make same changes in code
Code:
n=split(var,arr,":")


Thanks
But the script giving
syntax error at line1
bailing out at line 1
what modifications required in script as blank line seperation can't be the criteria in my case as each block internally contains blank lines so it has to be header and footer which are common in all blocks.
# 12  
Old 10-30-2006
You might have missed some quotes.

Try this to remove blank lines between header and footer
Code:
sed "/header/,/footer/{/^ *$/d;}" file | 
awk -v var="xy pc" ' BEGIN { RS = "" ; FS = "\n"
n=split(var,arr," ")}
{ flag=0;
for( i = 1 ; i <= NF; ++i )
{
for( j = 1 ; j <= n ; ++j )
if(match ( $i , "^"arr[j]"$" ))
        {  flag=1;break;  }
}
if( flag == 1 ) print
printf("\n")
} '

# 13  
Old 10-30-2006
Another way with awk :
Code:
awk -v search='xy' '
    /header/,/footer/ { 
       block = (block ? block ORS : "") $0;
    }
    /footer/ { 
       if (block ~ search) 
          print block;
       block = "";
    } ' inputfile

Jean-Pierre.
# 14  
Old 10-31-2006
Quote:
Originally Posted by aigles
Another way with awk :
Code:
awk -v search='xy' '
    /header/,/footer/ { 
       block = (block ? block ORS : "") $0;
    }
    /footer/ { 
       if (block ~ search) 
          print block;
       block = "";
    } ' inputfile

Jean-Pierre.

Thanx very much friend Smilie
It worked perfectly with nawk instead of awk..
Can u please explain the script specially the portion
block = (block ? block ORS : "") $0;
What does it means.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Select last update data based on file name

Hi All, I need to remove all files except the most update data based on date on filename Input data_AIDS_20150312.txt data_AIDS_20150311.txt data_AIDS_20150411.txt data_AIDS_20140312.txt the most updated data is data_AIDS_20150411.txt, so I'll remove other files. My expected output... (3 Replies)
Discussion started by: radius
3 Replies

2. Shell Programming and Scripting

Select lines from a file based on a criteria

Hi I need to select lines from a txt file, I have got a line starting with ZMIO:MSISDN= and after a few line I have another line starting with 'MOBILE STATION ISDN NUMBER' and another one starting with 'VLR-ADDRESS' I need to copy these three lines as three different columns in a separate... (3 Replies)
Discussion started by: Tlcm sam
3 Replies

3. Shell Programming and Scripting

Rename portion of file based on another file

Hello, I've been searching and reading, but I can't figure out how to solve this problem with my newbie skills. In my directory, I have a list of files (see dirlist.txt attachment) that I need to merge and rename. I have part of the code of the code figured out (see below). However, I... (3 Replies)
Discussion started by: anjulka
3 Replies

4. Shell Programming and Scripting

redirecting oracle sqlplus select query into file

So, I would like to run differen select queries on multiple databases.. I made a script wich I thought to be called something like.. ./script.sh sql_file_name out.log or to enter select statement in a command line.. (aix) and I did created some shell script wich is not working.. it... (6 Replies)
Discussion started by: bongo
6 Replies

5. UNIX for Advanced & Expert Users

need to get a portion of entries in file based on a criteria --- Help please

All, Below is the file, what i need to do is take the text in between the /*-- and --*/ , i mean the jobs. Then i have grep for system name . If the job is there in system 1 i have to print to a file. Basically i want to take all the jobs that are in system1 to another file . because... (7 Replies)
Discussion started by: arunkumar_mca
7 Replies

6. Shell Programming and Scripting

Redirecting sql select query result to txt file

Hi Yogesh, Lucky that i caught you online. Yeah i read about DBI and the WriteExcel module. But the server is not supporting these modules. It said..."Cannot locate DBI"..."Cannot locate Spreadsheet::WriteExcel" I tried creating a simple text file to get the query output, but the... (1 Reply)
Discussion started by: dolphin123
1 Replies

7. Shell Programming and Scripting

Redirecting sql select query result to txt file

Hi , I just found you while surfing for the string 'Redirecting sql select query output from within a shell script to txt file/excel file' Could you find time sending me the code for the above question? It'll be great help for me. I have a perl file that calls the sql file... (1 Reply)
Discussion started by: dolphin123
1 Replies

8. UNIX for Dummies Questions & Answers

Reading from a file and passing the value to a select query

Hi all, Here is my problem. I want to read data from a file and pass the variable to a select query. I tried but it doesn't seem to work. Please advise. Example below. FileName='filekey.txt' while read LINE do var=$LINE print "For File key $var" ${ORACLE_HOME}/bin/sqlplus -s... (1 Reply)
Discussion started by: er_ashu
1 Replies

9. Shell Programming and Scripting

select a portion of a file into a CSV

How will i convert a file <LDATE>10-12-07</LDATE><LTIME>13:47:48.553</LTIME><LTEXT>name:anju;city:blore;ph:123</LTEXT> <LDATE>10-12-07</LDATE><LTIME>13:47:48.553</LTIME><LTEXT>name:anju;city:blore;ph:123</LTEXT>... (8 Replies)
Discussion started by: anju
8 Replies

10. Shell Programming and Scripting

awk program to select a portion of a line

Hi all, I am new to awk programs.I have a file like this vjfavhjlaf<LTEXT>aabcdfffvvbbxbcddjbv</LTEXT>fAFdfdADfd vjfavhjlaf<LTEXT>aabcdfffvvbbxbcddjbv</LTEXT>fAFdfdADfd vjfavhjlaf<LTEXT>aabcdfffvvbbxbcddjbv</LTEXT>fAFdfdADfd vjfavhjlaf<LTEXT>aabcdfffvvbbxbcddjbv</LTEXT>fAFdfdADfd... (3 Replies)
Discussion started by: anju
3 Replies
Login or Register to Ask a Question