Sponsored Content
Top Forums Shell Programming and Scripting awk removing data before or after a pattern Post 302443394 by agama on Sunday 8th of August 2010 03:42:12 PM
Old 08-08-2010
This is one way to accomplish what you need; may not be the most efficient, but is easy to understand. It will do both 'drop before' and 'drop after' functions. I've set it up to match strings, not patterns, if you truly need to match patterns, use the match() function in awk rather than index().

Code:
#!/usr/bin/env ksh

# parms:        $1 -- before string; all records before matching this string are dropped
#                       If this is "none" then all records until the after string is matched are kept
#               $2 -- after string; all records after this string are dropped.

awk -v toss_before="${1:-none}" -v toss_after="$2" '
        BEGIN {
                if( toss_before == "none" )     # keep everything from the start
                        snarf = 1;
                else
                        snarf = 0;              # must wait until we see toss_before to start keeping data
        }

        {
                if( snarf ) 
                {
                        printf( "%s\n", $0 );           # print if snarfing 

                        if( toss_after && index( $0, toss_after ) )     # check to see if this has the end string
                                exit( 0 );
                }
                else                                    # not snarfing, see if this is the start string
                {
                        if( index( $0, toss_before ) )
                        {
                                printf( "%s\n", $0 );
                                snarf = 1;
                        }
                }
        }
'

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

removing a line containing a pattern in sed

i need to use sed to remove an entire line containing a pattern stored in a variable say $var1 this var1 will be a URL and will therefore contain slashes any help would be greatly appreciated (1 Reply)
Discussion started by: Fire_Storm
1 Replies

2. Shell Programming and Scripting

Replacing or removing a long list of pattern by using awk or sed

Input: >abc|123456|def|EXIT| >abc|203456|def|EXIT2| >abc|234056|def|EXIT3| >abc|340056|def|EXIT4| >abc|456000|def|EXIT5| . . . Output: def|EXIT| def|EXIT2| def|EXIT3| def|EXIT4| def|EXIT5| . . My try code: (9 Replies)
Discussion started by: patrick87
9 Replies

3. Shell Programming and Scripting

SFTP to server, pulling data and removing the data

Hi all, I have the following script, but are not too sure about the syntax to complete the script. In essence, the script must connect to a SFTP server at a client site with username and password located in a file on my server. Then change to the appropriate directory. Pull the data to the... (1 Reply)
Discussion started by: codenjanod
1 Replies

4. Shell Programming and Scripting

Removing data with pattern matching

I have the following: HH:MM:SS I want to use either % or # sign to remove :SS can somebody please provide me an example. I know how to do this in awk, but awk is too much overhead for something this simple since I will be doing this in a loop a lot of times. Thanks in advance to all... (2 Replies)
Discussion started by: BeefStu
2 Replies

5. Shell Programming and Scripting

Removing repeating lines from a data frame (AWK)

Hey Guys! I have written a code which combines lots of files into one big file(.csv). However, each of the original files had headers on the first line, and now that I've combined the files the headers are interspersed throughout the new combined data frame. For example, throughout the data... (21 Replies)
Discussion started by: gd9629
21 Replies

6. Shell Programming and Scripting

how to get data from hex file using SED or AWK based on pattern sign

I have a binary (hex) file I need to parse to get some data which are encoded this way: .* b4 . . . 01 12 .* af .* 83 L1 x1 x2 xL 84 L2 y1 y2 yL By another words there is a stream of hexadecimal bytes (in my example separated by space for better readability). I need to get value stored in... (3 Replies)
Discussion started by: sameucho
3 Replies

7. Shell Programming and Scripting

Removing files matching a pattern

I am on ubuntu 11.10 using bash scripts I want to remove all files matching a string pattern and I am using the following code find . -name "*$pattern*" -exec rm -f {} \;I have encountered a problem when $pattern is empty. In this case all my files in my current directory were deleted. This... (3 Replies)
Discussion started by: kristinu
3 Replies

8. Shell Programming and Scripting

Removing a pattern in a line

Dear team, I have a file curve.csv which is generated from oracle and each line has a comment associated with it, I want to get rid of this comment, can you please suggest me a command as how to do it Eg, cat curve.csv /*data for today curve*/ /*data for text1*/ this is the header /*data... (6 Replies)
Discussion started by: infyanurag
6 Replies

9. UNIX for Dummies Questions & Answers

Removing PATTERN from txt without removing lines and general text formatting

Hi Everybody! First post! Totally noobie. I'm using the terminal to read a poorly formatted book. The text file contains, in the middle of paragraphs, hyphenation to split words that are supposed to be on multiple pages. It looks ve -- ry much like this. I was hoping to use grep -v " -- "... (5 Replies)
Discussion started by: AxeHandle
5 Replies

10. Shell Programming and Scripting

awk to grab data in range then search for pattern

im using the following code to grab data, but after the data in the range im specifying has been grabbed, i want to count how many instances of a particular pattern is found? awk 'BEGIN{count=0} /parmlib.*RSP/,/seqfiles.*SSD/ {print; count++ } /103 error in ata file/ END { print count }'... (3 Replies)
Discussion started by: SkySmart
3 Replies
switch(n)						       Tcl Built-In Commands							 switch(n)

__________________________________________________________________________________________________________________________________________________

NAME
switch - Evaluate one of several scripts, depending on a given value SYNOPSIS
switch ?options? string pattern body ?pattern body ...? switch ?options? string {pattern body ?pattern body ...?} _________________________________________________________________ DESCRIPTION
The switch command matches its string argument against each of the pattern arguments in order. As soon as it finds a pattern that matches string it evaluates the following body argument by passing it recursively to the Tcl interpreter and returns the result of that evaluation. If the last pattern argument is default then it matches anything. If no pattern argument matches string and no default is given, then the switch command returns an empty string. If the initial arguments to switch start with - then they are treated as options unless there are exactly two arguments to switch (in which | case the first must the string and the second must be the pattern/body list). The following options are currently supported: -exact Use exact matching when comparing string to a pattern. This is the default. -glob When matching string to the patterns, use glob-style matching (i.e. the same as implemented by the string match command). -regexp When matching string to the patterns, use regular expression matching (as described in the re_syntax reference page). | -nocase | Causes comparisons to be handled in a case-insensitive manner. | -matchvar varName | This option (only legal when -regexp is also specified) specifies the name of a variable into which the list of matches found by | the regular expression engine will be written. The first element of the list written will be the overall substring of the input | string (i.e. the string argument to switch) matched, the second element of the list will be the substring matched by the first | capturing parenthesis in the regular expression that matched, and so on. When a default branch is taken, the variable will have | the empty list written to it. This option may be specified at the same time as the -indexvar option. | -indexvar varName | This option (only legal when -regexp is also specified) specifies the name of a variable into which the list of indices referring | to matching substrings found by the regular expression engine will be written. The first element of the list written will be a | two-element list specifying the index of the start and index of the first character after the end of the overall substring of the | input string (i.e. the string argument to switch) matched, in a similar way to the -indices option to the regexp can obtain. | Similarly, the second element of the list refers to the first capturing parenthesis in the regular expression that matched, and | so on. When a default branch is taken, the variable will have the empty list written to it. This option may be specified at the | same time as the -matchvar option. -- Marks the end of options. The argument following this one will be treated as string even if it starts with a -. This is not | required when the matching patterns and bodies are grouped together in a single argument. Two syntaxes are provided for the pattern and body arguments. The first uses a separate argument for each of the patterns and commands; this form is convenient if substitutions are desired on some of the patterns or commands. The second form places all of the patterns and commands together into a single argument; the argument must have proper list structure, with the elements of the list being the patterns and commands. The second form makes it easy to construct multi-line switch commands, since the braces around the whole list make it unnec- essary to include a backslash at the end of each line. Since the pattern arguments are in braces in the second form, no command or vari- able substitutions are performed on them; this makes the behavior of the second form different than the first form in some cases. If a body is specified as "-" it means that the body for the next pattern should also be used as the body for this pattern (if the next pattern also has a body of "-" then the body after that is used, and so on). This feature makes it possible to share a single body among several patterns. Beware of how you place comments in switch commands. Comments should only be placed inside the execution body of one of the patterns, and not intermingled with the patterns. EXAMPLES
The switch command can match against variables and not just literals, as shown here (the result is 2): set foo "abc" switch abc a - b {expr {1}} $foo {expr {2}} default {expr {3}} Using glob matching and the fall-through body is an alternative to writing regular expressions with alternations, as can be seen here (this returns 1): switch -glob aaab { a*b - b {expr {1}} a* {expr {2}} default {expr {3}} } Whenever nothing matches, the default clause (which must be last) is taken. This example has a result of 3: switch xyz { a - b { # Correct Comment Placement expr {1} } c { expr {2} } default { expr {3} } } When matching against regular expressions, information about what exactly matched is easily obtained using the -matchvar option: | switch -regexp -matchvar foo -- $bar { | a(b*)c { | puts "Found [string length [lindex $foo 1]] 'b's" | } | d(e*)f(g*)h { | puts "Found [string length [lindex $foo 1]] 'e's and | [string length [lindex $foo 2]] 'g's" | } | } | SEE ALSO
for(n), if(n), regexp(n) KEYWORDS
switch, match, regular expression Tcl 8.5 switch(n)
All times are GMT -4. The time now is 10:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy