Removing all lines prior to the last pattern in a file/stream Post: 302503758

Sponsored Content

Top Forums Shell Programming and Scripting Removing all lines prior to the last pattern in a file/stream Post 302503758 by LivinFree on Friday 11th of March 2011 01:50:02 PM

03-11-2011

Registered User

Removing all lines prior to the last pattern in a file/stream

Hi all,

I didn't find anything that specifically answers this after searching for a bit, so please forgive me if this has been covered before.

I'm looking to delete all lines prior to the last occurrence of a string in a file or stream from within a shell script (bash.)

A bit of background:
I'm running a command to pull the contents of a circular buffer, so I get whatever happens to be there. I'm only interested in the last string and the contents after that for each command I run. If can operate on the stream as it comes out and before it's written to a file, that'd be ideal. I'd also like to use sed if possible, since I use it in other areas of the script and I want to keep in all in-line.

An pseudo-example of the output I might get is below:

Code:

# run_async_command "mark command start test"
# pull_serial_log
^@CLI> [20372:53.558][20372:59.504] command^@ start^@ test^@
command output 1
command output 1
[...]
command output 1
^@CLI> [20372:53.558][20372:59.504] command^@ some^@ other^@ test^@
command output 2
command output 2
[...]
command output 2
^@CLI> [20372:53.558][20372:59.504] command^@ start^@ test^@
command output 3
command output 3
[...]
command output 3
#

In this case, all I'd want to see is the last instance of "command^@start^@ test^@" followed by its output to the end of the stream/file. Note that the "^@" characters here represent NULL characters. The command output is an arbitrary number of lines - it may be 10 lines, or it may be 1000 lines, and has no specific structure. Here's what I would like to see:

Code:

# run_async_command "mark command start test"
# pull_serial_log | sed -n 'MAGIC_GOES_HERE'
^@CLI> [20372:53.558][20372:59.504] command^@ start^@ test^@
command output 3
command output 3
[...]
command output 3
#

What I plan on doing is running the run_async_command, then dumping the buffer and stripping out only the last command, then run_async_command again with a different command and stripping out only the output from that command, and so on.

Please let me know if anyone wants more details.

LivinFree

View Public Profile for LivinFree

Find all posts by LivinFree

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search file for pattern and grab some lines before pattern

I want to search a file for a string and then if the string is found I need the line that the string is on - but also the previous two lines from the file (that the pattern will not be found in) This is on solaris Can you help?

2. Shell Programming and Scripting

removing pattern which is spread in multiple lines

I have several huge files wich contains oracle table creation scripts as follows: I would need to remove the pattern colored in red above. Any sed/awk/pearl code will be of much help. Thanks

3. UNIX for Dummies Questions & Answers

removing duplicates of a pattern from a file

hey all, I need some help. I have a text file with names in it. My target is that if a particular pattern exists in that file more than once..then i want to rename all the occurences of that pattern by alternate patterns.. for e.g if i have PATTERN occuring 5 times then i want to...

4. Shell Programming and Scripting

how to get lines prior to the line being searched

Hi, Can anbody please let me know how i can retrieve lines above the line being searched in a file. I am looking for an error message from a file, if I see that message I want the lines above that message along with this line. how do we do this. Please do let me know An example which i have...

5. Shell Programming and Scripting

Delete lines prior to a specific date in a log file.

Hi all. I have a database log file in which log data get appended to it daily. I want to do a automatic maintainence of this log by going through the log and deleting lines belonging to a certain date. How should i do it? Please help. Thanks. Example. To delete all lines prior to Jun...

6. Shell Programming and Scripting

pattern match in live stream

hi! i have a situation like this where i have to analyse the live log generated from /bin/scp -v you@example.com is if a pattern like say "Too many connections" comes i shud be able to identify it .

7. Shell Programming and Scripting

removing lines around a matched pattern

I have an ugly conf file that has the string I'm interested in searching for in the middle of a block of code that's relevant, and I'm trying to find a way to remove that entire block based on the matched line. I've googled for this problem, and most people helping are only interested in...

8. Shell Programming and Scripting

Deleting lines from a stream after matching a pattern

Hi, I have a requirement to to an ldapsearch and remove the shadow attributes in the output file. What I do is ldapsearch() | operation to remove shadow > FILE The ldapsearch gives output like this(with same line formation): objectClass: FSConfig objectClass: extensibleObject fsCAIP:...

9. UNIX for Dummies Questions & Answers

Removing PATTERN from txt without removing lines and general text formatting

Hi Everybody! First post! Totally noobie. I'm using the terminal to read a poorly formatted book. The text file contains, in the middle of paragraphs, hyphenation to split words that are supposed to be on multiple pages. It looks ve -- ry much like this. I was hoping to use grep -v " -- "...

10. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when...

LEARN ABOUT REDHAT

open

OPEN(1) 							     Linux 1.x								   OPEN(1)

NAME

       open - start a program on a new virtual terminal (VT).

SYNOPSIS

       open [-c vtnumber] [-s] [-u] [-l] [-v] [--] command command_options

DESCRIPTION

       open will find the first available VT, and run on it the given command with the given command options, standard input, output and error are
       directed to that terminal. The current search path ($PATH) is used to find the requested command. If no command is specified then the envi-
       ronment variable $SHELL is used.

   OPTIONS
       -c vtnumber
	      Use the given VT number and not the first available. Note you must have write access to the supplied VT for this to work.

       -s     Switch to the new VT when starting the command. The VT of the new command will be made the new current VT.

       -u     Figure out the owner of the current VT, and run login as that user.  Suitable to be called by init. Shouldn't be used with -c or -l.

       -l     Make the command a login shell. A - is prepended to the name of the command to be executed.

       -v     Be a bit more verbose.

       -w     wait for command to complete. If -w and -s are used together then open will switch back to the controlling terminal when the command
	      completes.

       --     end of options to open.

NOTE

       If open is compiled with a POSIX (Gnu) getopt() and you wish to set options to the command to be run, then  you	must  supply  the  end	of
       options -- flag before the command.

EXAMPLES

       open can be used to start a shell on the next free VT, by using the command:

       open bash

       To start the shell as a login shell, use:

       open -l bash

       To get a long listing you must supply the -- separator:

       open -- ls -l

SEE ALSO

       login(1), doshell(8), switchto(1).

AUTHOR

       Jon Tombs <jon@gtex02.us.es or jon@robots.ox.ac.uk>
       -w idea from "sam".

19 Jul 1996							       V1.4								   OPEN(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search file for pattern and grab some lines before pattern

Discussion started by: frustrated1

2. Shell Programming and Scripting

removing pattern which is spread in multiple lines

Discussion started by: sabyasm

3. UNIX for Dummies Questions & Answers

removing duplicates of a pattern from a file

Discussion started by: ashisharora

4. Shell Programming and Scripting

how to get lines prior to the line being searched

Discussion started by: arunrao_oradba