Change Parse Script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Change Parse Script
# 1  
Old 08-30-2011
Question Change Parse Script

I have a question about changing how parsing occurs currently for us:

input FILE123
Code:
TAGA01: 01
TAG02: daadsf
TAG03: adfasdf
TAGBBB04: 35
TAG05: asdfa
TAG07: adfd
TAG07: adfa3
TAG07: 234234
TAGCC08: 3525df
TAG09: adsfa
TAG10: 245
TAG11: nnnn
EOR:
TAGA01: 02
TAG02: abas
TAG03: asdfasd
TAGBBB04: E
TAG05: asdfasd
TAG07: acvasc
TAG07: czcvc
TAG07: 22
TAGCC08: adsfasd
TAG09: Y
TAG11: yyyy
EOR:
.
.
.

Note that some tags may not be in a record, and some tags may repeat in the same record.

I need to covert to the following inline format (limiter doesn't matter, and I can change it should the data include the limiter in other files) and trim it so the tag doesn't appear:

Code:
Format:
TAGA01 TAGCC08 TAGBBB04 TAG09 TAG11

output.file:
01 3535df 35 adsfa nnnn
02 adsfasd E Y yyyy
.
.
.

Here is what is used currently (from memory, so the syntax isn't correct but the idea is):
Code:
cat FILE123 | egrep "^TAGA01 ^TAGBBB04 ^TAGCC08 ^TAG09 ^TAG11" | awk -F. -f awkfile.awk > output.file

where awkfile.awk contains if statements and a printf output statement (again, syntax along with substring numbers are not correct - but the idea is there):
Code:
if ($1==TAGA01) {pTAGA01=substr($1,3)}
.
.
.
if ($1==TAG11) {
   pTAG11=substr($1,4)
   printf pTAGA01 ... pTAG11
}



I wanted to see different ideas for two reasons: one to see if this could be more efficient since every tag gets multiple ifs every time, and just to straight up learn something new.

Thanks for your time!
# 2  
Old 08-30-2011
If you change EOR: to the blank line you can use perl in its paragraph mode:
Code:
sed 's/^EOR:$//' INPUTFILE | 
perl -00 -ne '/
TAGA01:\s+(.*?)\n
.*
TAGCC08:\s+(.*?)\n
# and so on
/xs && print "$1 $2\n"'

# 3  
Old 08-31-2011
Quote:
Originally Posted by yazu
If you change EOR: to the blank line you can use perl in its paragraph mode:
Code:
sed 's/^EOR:$//' INPUTFILE | 
perl -00 -ne '/
TAGA01:\s+(.*?)\n
.*
TAGCC08:\s+(.*?)\n
# and so on
/xs && print "$1 $2\n"'

Thanks yazu...if you get a moment, could you explain a few things to me:
I understand what the sed command does, though not all of the syntax. From the man page: s/regexp/replacement. Why the ^ (was wondering about this character in the egrep command in the original script) and the $/ (is that the new line character)? And the third / indicates replace with a blank line since no other character is shown?

All of this is piped into the perl script.

Again, from the man page -00 will 'slurp' the stream in paragraph mode. -ne starts a while <> loop command. Not sure about perl command itself. Why the first /? The command searches for a tag:, but not sure what \s+(.*?) does, along with .* in the between tag searches. Not sure what /ns does, along with what the $1 and $2 refer to in the print line.

And I assume all of this can just be > into an output file, or is there a better way to do that with perl?

Thanks again!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Homework & Coursework Questions

Script parse file

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Hi all, I need help for a script that pulls out a series of numbers from a file (attached file) Basically I... (1 Reply)
Discussion started by: gianvitolinuxs
1 Replies

2. Shell Programming and Scripting

Script parse file Linux

Hi all, I need help for a script that pulls out a series of numbers from a file (attached file) Basically I need a parse to write me in a variable: 9d424312 Can someone help me? Thank you (2 Replies)
Discussion started by: gianvitolinuxs
2 Replies

3. Shell Programming and Scripting

Script to parse bookmarks file

I am using Internet Explorer v10 at work and regularly need to import my personal Firefox bookmarks over. Long story short, I have found the import falling over on any bookmark elements which are over 256 characters. The bookmark file contains bookmarks of this format: <DT><A... (4 Replies)
Discussion started by: ozgadgetguy
4 Replies

4. Shell Programming and Scripting

awk :help to parse a file to change to separated by colon ":"

Hi experts , I am trying to get the below output: file : 0/6/4/1 0x0019503C2E26 5 UP lan5 snap5 1 ETHER Yes 224 0/6/4/0 0x0019503C2E25 6 UP lan6 snap6 2 ETHER Yes 224 0/2/1/0 0x0019503E6900 0 UP lan0 snap0 3 ETHER Yes 224... (8 Replies)
Discussion started by: rveri
8 Replies

5. Shell Programming and Scripting

script to parse the properties file

Hi Friends, I have a requirement to parse a properties file having a key=value pairs. i need to count the number of key value pairs in the properties file and iterate through each key-value pair. I have written the script to read the number of lines from the property file, but cannot... (2 Replies)
Discussion started by: prashdeep
2 Replies

6. UNIX for Dummies Questions & Answers

bash script to parse sequence...

Hi, I have 4000 list files and 4000 sequence data files. Each list file contains a number of 'headers' and data file contains 'header and data'. I would like to extract data from the data file using the list file and write into a new file. As each of the files are quite large, an efficient piece... (6 Replies)
Discussion started by: Fahmida
6 Replies

7. Shell Programming and Scripting

Parse for errors shell script

All, I have a shell script which parses the /var/adm/messages file for errors every 15 minutes as a cron job. The script runs at 01, 16, 31, and 46 minutes every hour. The problem is if the error is encountered any time during the beginning of hour I can get paged three times. I would like to... (2 Replies)
Discussion started by: bubba112557
2 Replies

8. Shell Programming and Scripting

parse data using sh script

Hi, I am a newbie to unix/shell scripting and i have a question on how to parse a txt file using perl in a sh script. I have a txt file that contains hundreds of lines with data like this.... X, Y, Latitude, Longitude 1, 142, -38.000000, -91.000000, 26.348 2, 142, 60.000000, -90.000000,... (2 Replies)
Discussion started by: moonbaby
2 Replies

9. Shell Programming and Scripting

Script to parse an access-list

Folks, I have a textfile with the config of my router. Part of that config looks like this: router config ... ip access-list extended MyAccessList remark this is my security rule permit host 10.0.0.1 any deny host 10.0.0.2 any ... ip access-list extended YourAccessList ... (25 Replies)
Discussion started by: philipz
25 Replies

10. Shell Programming and Scripting

How do you parse a variable in a bash script?

I have a script I use on my web server (Apache2). I am changing to Lighttpd and need to make a few changes. This is what I use on my apache server #!/bin/bash # accepts 3 parameters: <domain name> <user name> <XXXXXXXX> # domain name is without www (just domain.com) # username would be... (3 Replies)
Discussion started by: vertical98
3 Replies
Login or Register to Ask a Question