10-14-2004
grep and sed to find a pattern and add newline
Hello All,
I have log file the result from a multithreaded process. So when a process finishes it will write to this log file as 123 rows merged.
The issue is sometimes the processess finish at the same time or write to the file at the same time as
123 rows merged.145 rows merged.
At the end of the script we do a source count from table and target count from log file. As you can see the above line will fail grep as grep will identify the line and move on while awk will take the first parameter as 123 only.
I have used the following command to correct this
grep -i "^[0-9].* rows merged" ttt | sed 's/merged\./merged\
/g' | grep -i "^[0-9].* rows merged" | sed '/^$/'d | awk '{print $1}' | while read X
do
echo $X
Y=`expr $Y + $X`
done
echo $Y
TCOUNT=${Y}
but there was another instance when this script failed because the log file has something like
Commit complete.150 rows merged.Commit complete.
The above 150 rows that were merged didn't get counted. I would like to find [0-9].* rows merged anywhere on any line and separate this pattern into a new line
EG: the above Commit complete.150 rows merged.Commit complete.
should become
Commit complete.
150 rows merged.
Commit complete.
Any help will be greatly appreciated.
Thanks
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi
I have a problem using grep to find a pattern followed by newline.
Here is my file
xxxxxxxxxxxpattern
patternxxxxxx
pattern
xxxpattern
xxpatternx
xxxxxxyyyyxxxx
I want the result to be like this
xxxxxxxxxxxpattern
pattern
xxxpattern
Please help (2 Replies)
Discussion started by: lalelle
2 Replies
2. Shell Programming and Scripting
Hi,
I have the following XML not well-indented code:
<hallo
>this is a line
</hallo>
So I need to remove the newline.
This syntax finds what I need to correct, but I don't know how to remove the newline after my pattern:
sed 's/<.*$/&/'
How can I subtract the newline after my... (1 Reply)
Discussion started by: nico.ben
1 Replies
3. Shell Programming and Scripting
This is my first post, please be nice. I have tried to google and read different tutorials.
The task at hand is:
Input file input.txt (example)
abc123defhij-E-1234jslo
456ujs-W-abXjklp
From this file the task is to grep the -E- and -W- strings that are unique and write a new file... (5 Replies)
Discussion started by: TestTomas
5 Replies
4. Shell Programming and Scripting
<Update>
I have the solution:
sed 's/\{3\}/&;&;---;4/'
The thread can be marked as solved!
</Update>
Hi There,
I'm working on a script processing some data from a website into cvs format. There is only one final problem left I can't find a solution.
I've processed my file... (0 Replies)
Discussion started by: lolworlds
0 Replies
5. Shell Programming and Scripting
Hi,
I am getting crazy after days on looking at it:
Bash in Ubuntu 12.04.1
I want to do this:
pattern="system /path1/file1 file1"
new_pattern=" data /path2/file2 file2"
file to edit: data.db
- I need to search in the file data.db for the nth occurrence of pattern
- pattern must... (14 Replies)
Discussion started by: Phil3759
14 Replies
6. UNIX for Dummies Questions & Answers
Hi,
I've been trying to work out how to add a new line to a file when the pattern matches .dmg.
I've been searching Google but yet not found a working solution.
Help would be appreciated... (9 Replies)
Discussion started by: pburge
9 Replies
7. Shell Programming and Scripting
Hi ,
I have a file where i have modifed certain things compared to original file . The difference of the original file and modified file is as follows.
# diff mir_lex.c.modified mir_lex.c.orig
3209c3209
< if(yy_current_buffer -> yy_is_our_buffer == 0) {
---
>... (5 Replies)
Discussion started by: breezevinay
5 Replies
8. Shell Programming and Scripting
The intended result should be :
PDF converters
'empty line'
gpdftext and pdftotext?xml version="1.0"?>
xml:space="preserve"><note-content version="0.1" xmlns:/tomboy/link" xmlns:size="http://beatniksoftware.com/tomboy/size">PDF converters
gpdftext and pdftotext</note-content>... (9 Replies)
Discussion started by: Klasform
9 Replies
9. Shell Programming and Scripting
Greetings Experts,
I am in AIX; I have a file generated through awk after processing the input files. Now I need to replace or remove the new-line characters on all lines that doesn't have a ; which is the last character on the line. I tried to use sed 's/\n/ /g' After checking through the... (6 Replies)
Discussion started by: chill3chee
6 Replies
10. Shell Programming and Scripting
Original File
Server1|poweredOn|268401| Server1/Server1.vmx|Red Hat Enterprise Linux 5 (64-bit)
Need Output
Server1|poweredOn|DR|T1|268401| Server1/Server1.vmx|Red Hat Enterprise Linux 5 (64-bit)
Conduction to check find the string "SFCHT1" and "SR" and add prefix has... (4 Replies)
Discussion started by: ranjancom2000
4 Replies
LEARN ABOUT LINUX
unbuffer
UNBUFFER(1) General Commands Manual UNBUFFER(1)
NAME
unbuffer - unbuffer output
SYNOPSIS
unbuffer program [ args ]
INTRODUCTION
unbuffer disables the output buffering that occurs when program output is redirected from non-interactive programs. For example, suppose
you are watching the output from a fifo by running it through od and then more.
od -c /tmp/fifo | more
You will not see anything until a full page of output has been produced.
You can disable this automatic buffering as follows:
unbuffer od -c /tmp/fifo | more
Normally, unbuffer does not read from stdin. This simplifies use of unbuffer in some situations. To use unbuffer in a pipeline, use the
-p flag. Example:
process1 | unbuffer -p process2 | process3
CAVEATS
unbuffer -p may appear to work incorrectly if a process feeding input to unbuffer exits. Consider:
process1 | unbuffer -p process2 | process3
If process1 exits, process2 may not yet have finished. It is impossible for unbuffer to know long to wait for process2 and process2 may
not ever finish, for example, if it is a filter. For expediency, unbuffer simply exits when it encounters an EOF from either its input or
process2.
In order to have a version of unbuffer that worked in all situations, an oracle would be necessary. If you want an application-specific
solution, workarounds or hand-coded Expect may be more suitable. For example, the following example shows how to allow grep to finish pro-
cessing when the cat before it finishes first. Using cat to feed grep would never require unbuffer in real life. It is merely a place-
holder for some imaginary process that may or may not finish. Similarly, the final cat at the end of the pipeline is also a placeholder
for another process.
$ cat /tmp/abcdef.log | grep abc | cat
abcdef
xxxabc defxxx
$ cat /tmp/abcdef.log | unbuffer grep abc | cat
$ (cat /tmp/abcdef.log ; sleep 1) | unbuffer grep abc | cat
abcdef
xxxabc defxxx
$
BUGS
The man page is longer than the program.
SEE ALSO
"Exploring Expect: A Tcl-Based Toolkit for Automating Interactive Programs" by Don Libes, O'Reilly and Associates, January 1995.
AUTHOR
Don Libes, National Institute of Standards and Technology
1 June 1994 UNBUFFER(1)