Sponsored Content
Top Forums Shell Programming and Scripting how to get one particular section (using awk)? Post 302467213 by DGPickett on Thursday 28th of October 2010 03:13:48 PM
Old 10-28-2010
I think sed is a more regular and friendly tool that is usually faster and easy to use. Awk has more strength in delimited many-field records. I am ot much of an awk user, and I don't think either is great about multi-line blocks, but sed is pretty good, as you can pile up lines in the buffer.

Code:
sed '
  :loop
  /^>ref|'$input'| *$/!d
  :loop2
  $b
  N
  P
  s/.*\n//
  /^>...|.*| *$/b loop
  b loop2
 ' $in_file

Narrative: sed opens $in_file and examines every line.
  1. Set a branch tag called 'loop'
  2. If we do not find the desired $input in a header line, delete this line (which starts the script over on the next line).
  3. Set a second branch tag 'loop2',
  4. if at EOF, branch (to end of script = print and exit),
  5. get the next line into the buffer after the first and \n,
  6. print the first line,
  7. delete the first line and \n,
  8. if there is a header in the buffer, loop way back to :loop,
  9. branch to loop2 to continue printing the block.
You can see the simple commands in sed allow easy reuse. I call this sort of sed a looper script. Some sed scripts just work on one line, so I call them filter-transform scripts. If you need both at once, you can pipe one sed to the next. Sed is fast and has no size limits.

I see an awk guy bartus11 has given us a one line awk script. I could one line, I guess, but I like to make it easy to read and maintain. The '.' in your $input is the sed metachar for any char, but that does not seem a very big weakness. I used more definition on the header line, so you can have <>| in the text without side effects. Is every '^>[^|]\{3\}|[^\]\{1,99\}| *$' line a header? I may have used too much definition, with /^>ref| so I will revise -- poof! Always >...|?

Last edited by DGPickett; 10-28-2010 at 04:23 PM..
 

10 More Discussions You Might Find Interesting

1. Post Here to Contact Site Administrators and Moderators

New Section

Just like we have a section "Unix for dummies..." , why not have a section on UNIX BACKUP AND RECOVERY Thanks :) (3 Replies)
Discussion started by: kapilv
3 Replies

2. Shell Programming and Scripting

sed & awk--get section of file based 2 params

I need to get a section of a file based on 2 params. I want the part of the file between param 1 & 2. I have tried a bunch of ways and just can't seem to get it right. Can someone please help me out.....its much appreciated. Here is what I have found that looks like what I want....but doesn't... (12 Replies)
Discussion started by: Andy Cook
12 Replies

3. Post Here to Contact Site Administrators and Moderators

New section

Hi Just a thought if it already hasn't been suggested. While looking at the forums I thought it might be a good idea under somewhere like 'special forums' add a section called 'projects'. I think this would be good for people to be able to post projects they have created. For example I am... (3 Replies)
Discussion started by: woofie
3 Replies

4. Shell Programming and Scripting

Extract section of file based on word in section

I have a list of Servers in no particular order as follows: virtualMachines="IIBSBS IIBVICDMS01 IIBVICMA01"And I am generating some output from a pre-existing script that gives me the following (this is a sample output selection). 9/17/2010 8:00:05 PM: Normal backup using VDRBACKUPS... (2 Replies)
Discussion started by: jelloir
2 Replies

5. Shell Programming and Scripting

Prepend first line of section to each line until the next section header

I have searched in a variety of ways in a variety of places but have come up empty. I would like to prepend a portion of a section header to each following line until the next section header. I have been using sed for most things up until now but I'd go for a solution in just about anything--... (7 Replies)
Discussion started by: pagrus
7 Replies

6. Shell Programming and Scripting

using awk to get specific section of lines in logs

i have a log file that has the date and time that looks like this: Wed Jun 28 15:46:21 2012 test failed tailed passed passed not error panic what we want to focus on is the first 5 columns because they contain the date and time. the date and time can be anywhere on the line. in this... (6 Replies)
Discussion started by: SkySmart
6 Replies

7. UNIX for Dummies Questions & Answers

Sorting arrays horizontally without END section, awk

input: ref001, Europe, Belgium, 1001 ref001, Europe, Spain, 203 ref001, Europe, Germany, 457 ref002, America, Canada, 234 ref002, America, US, 87 ref002, America, Alaska, 652 Without using an END section, I need to write all the info related to the same ref number ($1)and continent ($2) on... (9 Replies)
Discussion started by: lucasvs
9 Replies

8. Shell Programming and Scripting

Extracting text from within a section of text using AWK

I have a command which returns the below output. How can I write a script to extract mainhost and secondhost from this output and put it into an array? I may sometimes have more hosts like thirdhost. I am redirecting this output to a variable. So I guess there should be a awk or sed command to... (7 Replies)
Discussion started by: heykiran
7 Replies

9. Shell Programming and Scripting

awk to lookup section of file in a range of another file

In the below, I am trying to lookup $1 and $2 from file1, in a range search using $1 $2 $3 of file2. If the search key from file1 is found in file2, then the word low is printed in the last field of that line in the updated file1. Only the last section of file1 needs to be searched, but I am not... (6 Replies)
Discussion started by: cmccabe
6 Replies

10. Shell Programming and Scripting

awk to parse section of csv into array

In the awk below I am trying to parse the Sample Name below the section. The values that are extracted are read into array s(each value in a row seperated by a space) which will be used later in a bash script. The awk does execute but no values are printed. I am also not sure how to print in a row... (1 Reply)
Discussion started by: cmccabe
1 Replies
SED(1)								   User Commands							    SED(1)

NAME
sed - stream editor for filtering and transforming text SYNOPSIS
sed [OPTION]... {script-only-if-no-other-script} [input-file]... DESCRIPTION
Sed is a stream editor. A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipe- line). While in some ways similar to an editor which permits scripted edits (such as ed), sed works by making only one pass over the input(s), and is consequently more efficient. But it is sed's ability to filter text in a pipeline which particularly distinguishes it from other types of editors. -n, --quiet, --silent suppress automatic printing of pattern space -e script, --expression=script add the script to the commands to be executed -f script-file, --file=script-file add the contents of script-file to the commands to be executed --follow-symlinks follow symlinks when processing in place -i[SUFFIX], --in-place[=SUFFIX] edit files in place (makes backup if extension supplied) -l N, --line-length=N specify the desired line-wrap length for the `l' command --posix disable all GNU extensions. -r, --regexp-extended use extended regular expressions in the script. -s, --separate consider files as separate rather than as a single continuous long stream. -u, --unbuffered load minimal amounts of data from the input files and flush the output buffers more often --help display this help and exit --version output version information and exit If no -e, --expression, -f, or --file option is given, then the first non-option argument is taken as the sed script to interpret. All remaining arguments are names of input files; if no input files are specified, then the standard input is read. E-mail bug reports to: bonzini@gnu.org . Be sure to include the word ``sed'' somewhere in the ``Subject:'' field. COMMAND SYNOPSIS
This is just a brief synopsis of sed commands to serve as a reminder to those who already know sed; other documentation (such as the tex- info document) must be consulted for fuller descriptions. Zero-address ``commands'' : label Label for b and t commands. #comment The comment extends until the next newline (or the end of a -e script fragment). } The closing bracket of a { } block. Zero- or One- address commands = Print the current line number. a text Append text, which has each embedded newline preceded by a backslash. i text Insert text, which has each embedded newline preceded by a backslash. q Immediately quit the sed script without processing any more input, except that if auto-print is not disabled the current pattern space will be printed. Q Immediately quit the sed script without processing any more input. r filename Append text read from filename. R filename Append a line read from filename. Commands which accept address ranges { Begin a block of commands (end with a }). b label Branch to label; if label is omitted, branch to end of script. t label If a s/// has done a successful substitution since the last input line was read and since the last t or T command, then branch to label; if label is omitted, branch to end of script. T label If no s/// has done a successful substitution since the last input line was read and since the last t or T command, then branch to label; if label is omitted, branch to end of script. c text Replace the selected lines with text, which has each embedded newline preceded by a backslash. d Delete pattern space. Start next cycle. D Delete up to the first embedded newline in the pattern space. Start next cycle, but skip reading from the input if there is still data in the pattern space. h H Copy/append pattern space to hold space. g G Copy/append hold space to pattern space. x Exchange the contents of the hold and pattern spaces. l List out the current line in a ``visually unambiguous'' form. n N Read/append the next line of input into the pattern space. p Print the current pattern space. P Print up to the first embedded newline of the current pattern space. s/regexp/replacement/ Attempt to match regexp against the pattern space. If successful, replace that portion matched with replacement. The replacement may contain the special character & to refer to that portion of the pattern space which matched, and the special escapes 1 through 9 to refer to the corresponding matching sub-expressions in the regexp. w filename Write the current pattern space to filename. W filename Write the first line of the current pattern space to filename. y/source/dest/ Transliterate the characters in the pattern space which appear in source to the corresponding character in dest. Addresses Sed commands can be given with no addresses, in which case the command will be executed for all input lines; with one address, in which case the command will only be executed for input lines which match that address; or with two addresses, in which case the command will be executed for all input lines which match the inclusive range of lines starting from the first address and continuing to the second address. Three things to note about address ranges: the syntax is addr1,addr2 (i.e., the addresses are separated by a comma); the line which addr1 matched will always be accepted, even if addr2 selects an earlier line; and if addr2 is a regexp, it will not be tested against the line that addr1 matched. After the address (or address-range), and before the command, a ! may be inserted, which specifies that the command shall only be executed if the address (or address-range) does not match. The following address types are supported: number Match only the specified line number. first~step Match every step'th line starting with line first. For example, ``sed -n 1~2p'' will print all the odd-numbered lines in the input stream, and the address 2~5 will match every fifth line, starting with the second. (This is an extension.) $ Match the last line. /regexp/ Match lines matching the regular expression regexp. cregexpc Match lines matching the regular expression regexp. The c may be any character. GNU sed also supports some special 2-address forms: 0,addr2 Start out in "matched first address" state, until addr2 is found. This is similar to 1,addr2, except that if addr2 matches the very first line of input the 0,addr2 form will be at the end of its range, whereas the 1,addr2 form will still be at the beginning of its range. addr1,+N Will match addr1 and the N lines following addr1. addr1,~N Will match addr1 and the lines following addr1 until the next line whose input line number is a multiple of N. REGULAR EXPRESSIONS
POSIX.2 BREs should be supported, but they aren't completely because of performance problems. The sequence in a regular expression matches the newline character, and similarly for a, , and other sequences. BUGS
E-mail bug reports to bonzini@gnu.org. Be sure to include the word ``sed'' somewhere in the ``Subject:'' field. Also, please include the output of ``sed --version'' in the body of your report if at all possible. COPYRIGHT
Copyright (C) 2003 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU- LAR PURPOSE, to the extent permitted by law. SEE ALSO
awk(1), ed(1), grep(1), tr(1), perlre(1), sed.info, any of various books on sed, the sed FAQ (http://sed.sf.net/grabbag/tutorials/sed- faq.txt), http://sed.sf.net/grabbag/. The full documentation for sed is maintained as a Texinfo manual. If the info and sed programs are properly installed at your site, the command info sed should give you access to the complete manual. sed version 4.1.5 July 2010 SED(1)
All times are GMT -4. The time now is 10:40 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy