awk removing data before or after a pattern | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

awk removing data before or after a pattern

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 08-08-2010
BeefStu BeefStu is offline
Registered User
 
Join Date: Aug 2010
Last Activity: 18 October 2013, 1:22 PM EDT
Posts: 135
Thanks: 24
Thanked 14 Times in 14 Posts
awk removing data before or after a pattern

I have the following data:

01:00:00 29 10 20 41
01:20:00 18 6 34 42
01:40:00 28 5 24 43
02:00:01 11 7 8 74
02:20:01 19 15 12 54
02:40:01 1 4 0 95
03:00:01 1 3 0 96
03:20:01 0 0 0 99
03:40:01 1 2 0 97

First I want to remove all data after a certain pattern, Ie 03:00 sp this
should leave me with the following results

01:00:00 29 10 20 41
01:20:00 18 6 34 42
01:40:00 28 5 24 43
02:00:01 11 7 8 74
02:20:01 19 15 12 54
02:40:01 1 4 0 95
03:00:01 1 3 0 96


Secondly I would also like to know how to remove all before the pattern
so this should leave me with the following results.

03:00:01 1 3 0 96
03:20:01 0 0 0 99
03:40:01 1 2 0 97

Note: Both these statements not be run at the same time.

Thanks to all who answer.
Sponsored Links
    #2  
Old 08-08-2010
bartus11's Avatar
bartus11 bartus11 is online now Forum Staff  
Moderator
 
Join Date: Apr 2009
Last Activity: 16 April 2014, 2:49 PM EDT
Posts: 3,679
Thanks: 6
Thanked 1,131 Times in 1,104 Posts
Removing after pattern:
Code:
awk '!p;/^03:00/{p=1}' file

Removing before pattern:
Code:
awk '/^03:00/{p=1}p' file

Sponsored Links
    #3  
Old 08-08-2010
agama agama is offline Forum Advisor  
Always Learning
 
Join Date: Jul 2010
Last Activity: 7 April 2014, 3:02 PM EDT
Location: earth>US>UTC-5
Posts: 1,466
Thanks: 110
Thanked 502 Times in 481 Posts
This is one way to accomplish what you need; may not be the most efficient, but is easy to understand. It will do both 'drop before' and 'drop after' functions. I've set it up to match strings, not patterns, if you truly need to match patterns, use the match() function in awk rather than index().


Code:
#!/usr/bin/env ksh

# parms:        $1 -- before string; all records before matching this string are dropped
#                       If this is "none" then all records until the after string is matched are kept
#               $2 -- after string; all records after this string are dropped.

awk -v toss_before="${1:-none}" -v toss_after="$2" '
        BEGIN {
                if( toss_before == "none" )     # keep everything from the start
                        snarf = 1;
                else
                        snarf = 0;              # must wait until we see toss_before to start keeping data
        }

        {
                if( snarf ) 
                {
                        printf( "%s\n", $0 );           # print if snarfing 

                        if( toss_after && index( $0, toss_after ) )     # check to see if this has the end string
                                exit( 0 );
                }
                else                                    # not snarfing, see if this is the start string
                {
                        if( index( $0, toss_before ) )
                        {
                                printf( "%s\n", $0 );
                                snarf = 1;
                        }
                }
        }
'

    #4  
Old 08-08-2010
gvj gvj is offline
Registered User
 
Join Date: Oct 2009
Last Activity: 28 June 2012, 6:11 AM EDT
Posts: 28
Thanks: 2
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by bartus11 View Post
Removing after pattern:
Code:
awk '!p;/^03:00/{p=1}' file

Removing before pattern:
Code:
awk '/^03:00/{p=1}p' file


Could you pls explain why it is working?? its seems interesting. I knw the basis of awk but how this printing is happening by assigning one variable ? is it due to some special variable like $_ in perl??
Sponsored Links
    #5  
Old 08-08-2010
agama agama is offline Forum Advisor  
Always Learning
 
Join Date: Jul 2010
Last Activity: 7 April 2014, 3:02 PM EDT
Location: earth>US>UTC-5
Posts: 1,466
Thanks: 110
Thanked 502 Times in 481 Posts
Quote:
Originally Posted by gvj View Post
Could you pls explain why it is working??
His script is taking advantage of the fact that if the action part of a pattern-action pair is missing, the action {print;} is assumed. Further, !p is the same as saying 'p == 0' in the pattern portion of the statement. The value of an undefined variable is either 0 or "" depending on how it is used. To write the script in 'long form' might make more sense to you:


Code:
awk ' 
BEGIN { p = 0;}
p == 0 { print; }       # pattern not matched; print  (!p;)
/^03:00/ { p = 1; }  # pattern is matched, stop printing (exit(0) would be more efficient)
' file

Similarly, printing everything after the pattern line:

Code:
BEGIN { p = 0;}
/^03:00/ { p = 1; }  # pattern is matched, allow printing
p != 0 { print; }       # p;  in the original script
' file

The Following User Says Thank You to agama For This Useful Post:
gvj (08-09-2010)
Sponsored Links
    #6  
Old 08-08-2010
kurumi kurumi is offline
Resident Ruby Wrangler
 
Join Date: Dec 2009
Last Activity: 12 January 2014, 4:32 AM EST
Posts: 601
Thanks: 2
Thanked 83 Times in 78 Posts

Code:
sed -n '1,/03:00/p' file


Code:
sed -n '/03:00/,$p' file

Sponsored Links
    #7  
Old 08-08-2010
rdcwayx rdcwayx is offline Forum Advisor  
Use nawk in Solaris
 
Join Date: Jun 2006
Last Activity: 22 March 2014, 12:27 PM EDT
Posts: 2,759
Thanks: 44
Thanked 418 Times in 406 Posts

Code:
awk '$1<"03:00:00"' urfile


Code:
awk '$1>="03:00:00"' urfile

Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
SFTP to server, pulling data and removing the data codenjanod Shell Programming and Scripting 1 03-03-2010 07:26 AM
Replacing or removing a long list of pattern by using awk or sed patrick87 Shell Programming and Scripting 9 11-20-2009 02:45 AM
removing duplicates of a pattern from a file ashisharora UNIX for Dummies Questions & Answers 3 09-04-2008 06:25 AM
removing pattern which is spread in multiple lines sabyasm Shell Programming and Scripting 2 05-13-2008 05:19 AM
removing a line containing a pattern in sed Fire_Storm Shell Programming and Scripting 1 04-20-2008 05:13 AM



All times are GMT -4. The time now is 02:57 PM.