Parsing file and extracting the useful data block


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing file and extracting the useful data block
# 1  
Old 03-07-2005
Parsing file and extracting the useful data block

Greetings All!!

I have a very peculiar problem where I have to parse a big text file and extract useful data out of it with starting and ending block pattern matching.

e.g. I have a input file like this:

sample data
block1
sample data
start
useful data
end
sample data
block2
sample data
start
useful data
end
sample data

My output shall be like following:

block1
useful data
block2
useful data

Any help will be appreciable.

Cheers

Arminder
# 2  
Old 03-07-2005
Assuming that your input file is always of the same format as specified in your example, the following code will work. Save this as block.awk
Code:
#!/usr/bin/awk -f

/^block/ {
  print $0
  next
}
/^start/ {
  flag = 1
  next
}
flag == 1 {
  print $0
  flag = 0
}

Make it executable, run it over your input file, and redirect output if desired...
Code:
$ chmod +x ./block.awk
$ ./block.awk my_file > my_output
$ more my_output
block1
useful data
block2
useful data

Cheers
ZB
# 3  
Old 03-07-2005
USe this,

Code:
 awk '{ if ($0 ~ "block") { print $0;getline;print $0 } }' testfile

hth.
# 4  
Old 03-07-2005
Quote:
Originally Posted by muthukumar
USe this,

Code:
 awk '{ if ($0 ~ "block") { print $0;getline;print $0 } }' testfile

hth.
No, that won't work. The OP wants "useful data" not "sample data". You'd need three "getline"s to get this to work....
Code:
awk '{ if ($0 ~ "block") { print $0;getline;getline;getline;print $0 } }' testfile

Cheers
ZB
# 5  
Old 03-07-2005
oops.. ZB you are 100% correct.
# 6  
Old 03-07-2005
Sorry Guys, maybe I didn't put it correctly. The amount of "useful data" isn't known i.e. it may be 2 lines long or 10/20/100 i.e. no same occurences. Same goes for the "sample data" too. Its only that "start" and "end" is that can tell me where to start from and where to end.

Later I figured a way to print the block and so currently I am using following in awk file:

/^start/./^end/

Thereafter I grep out (grep -v) the start and end text.

Cheers

Arminder
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing file data

Hey Guys, I'm a novice at shell scripts and i need some help parsing file data. Basically, I want to write a script that retrieves URLs. Here is what I have so far. #!/bin/bash echo "Please enter start date (format: yyyy-mm-dd):\c" read STARTDATE echo "Please enter end date... (7 Replies)
Discussion started by: silverdust
7 Replies

2. Shell Programming and Scripting

extracting block of lines from a file

consider the input file which i am dealing with looks like this.. #cat 11.sql create table abc ( . . . ) engine=Innodb ; . . etc . . . create table UsM ( blah blah blah ) engine=Innodb ; (5 Replies)
Discussion started by: vivek d r
5 Replies

3. UNIX for Dummies Questions & Answers

Extracting data from file

I am trying to compare the data in lines 3 & 5 to see if they match up to the '-S570' (see first code set, all proprietary information has been removed from code set) spawn telnet Trying ... Connected to CA-LOS1234-ASE-S570.cl . Escape character is '^]'. CA-LOS1234-ASE-S570 Username: ... (1 Reply)
Discussion started by: slipshft
1 Replies

4. Shell Programming and Scripting

Need help in extracting data from xml file

Hello, This is my first post in here, so excuse me if I sound too noob here! I need to extract the path "/apps/mp/installedApps/V61/HRO/hrms_01698_A_qa.ear" from the below xml extract. The path will always appear with the key "binariesURL" <deployedObject... (6 Replies)
Discussion started by: abhishek2386
6 Replies

5. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

6. UNIX for Dummies Questions & Answers

Extracting data from an xml file

Hello, Please can someone assist. I have the following xml file: <?xml version="1.0" encoding="utf-8" ?> - <PUTTRIGGER xmlns:xsd="http://www.test.org/2001/XMLSchema" xmlns:xsi="http://www.test.org/2001/XMLSchema-instance" APPLICATIONNUMBER="0501160" ACCOUNTNAME="Mrs S Test"... (15 Replies)
Discussion started by: Dolph
15 Replies

7. Shell Programming and Scripting

Parsing file, yaml file? Extracting specific sections

Here is a data file, which I believe is in YAML. I am trying to retrieve just the 'addon_domains" section, which doesnt seem to be as easy as I had originally thought. Any help on this would be greatly appreciated!! I have been trying to do this in awk and mostly bash scripting instead of perl... (3 Replies)
Discussion started by: Rhije
3 Replies

8. Shell Programming and Scripting

urgent-extracting block data from flat file using shell script

Hi, I want to extract block of data from flat file. the data will be like this start of log One two three end of log i want all data between start of log to end of log i.e One two three to be copied to another file. This particular block may appear multiple times in same file. I... (4 Replies)
Discussion started by: shirish_cd
4 Replies

9. UNIX for Dummies Questions & Answers

Extracting Data from a File

Hi I need to calculate the number of occurrences of a item in a number of files using Perl. The item appears continually throughout the files but in each case I only want to calculate it in certain blocks of the file. Example - Calculalte the number of occurrences of a 'pass' in a block of... (0 Replies)
Discussion started by: oop
0 Replies

10. Shell Programming and Scripting

Parsing the data in a file

Hi, I have file (FILE.tmp) having contents, FILE.tmp ======== filename=menudata records=0000000000037 ldbname=pinsys timestamp=2005/05/14-18:32:33 I want to parse it bring a new file which will look like, filename records ldbname timestamp... (2 Replies)
Discussion started by: Omkumar
2 Replies
Login or Register to Ask a Question