Sponsored Content
Top Forums Shell Programming and Scripting How to remove repetitive lines in a file with sed? Post 303022121 by baris35 on Saturday 25th of August 2018 04:16:34 PM
Old 08-25-2018
How to remove repetitive lines in a file with sed?

Hello,
My goal is the make all x times repeated lines into a single line.
I need to attain the expected output with sed -i , I need to overwrite the MyFile
MyFile:
Code:
Hello World
Welcome
Hello World
Welcome Back
This is my test

Expected output:
Code:
Hello World
Welcome
Welcome Back
This is my test

I found below code in another forum while searching but it did not solve the issue, even with sed -n:
Code:
sed -n '
:start
/^\[/{
    h
  :loop
    n
    /^\[/b start
    /^$/b loop
    x;p;g
}
p' MyFile

It would be nice if you would help me out on getting rid of this issue.
Many thanks
Boris

Last edited by baris35; 08-26-2018 at 07:56 AM.. Reason: upon more specific description request
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unix Remove repetitive alphabets

Hi, I am trying to write a script that will take 2 or more instances of repetitive alphabets (ZZ) to be removed from a field. This should only happen from beginning and end of a field. For Example : Input File a) ZZZIBM Corporation b) ZZZIBM Corporation ZZZZZ b) IBM ZZZ... (26 Replies)
Discussion started by: msalam65
26 Replies

2. Shell Programming and Scripting

using sed to remove lines

Can somebody explain why my sed command is not working. I do the folloinwg: Generates a binary file to /tmp/x1.out /usr/lib/sa/sa2 -s 4:00 -e 8:00 -i 3600 -A -o /tmp/x1.out decodes the file (no problem so far) sar -f /tmp/x1.out When I do this it does not appear to delete the... (4 Replies)
Discussion started by: BeefStu
4 Replies

3. Shell Programming and Scripting

grep/sed to remove lines in file

Hi, I have a file with values, file1: BELL-1180-1180-81|577:1017| BELL-1180-1180-81|jm10i-auto-stub1/577:102| BELL-1180-1180-81|jm10i-auto-stub1/577:101| BELL-1180-1180-81|jm10i-auto-stub1/577:1700| BELL-1180-1180-81|jm10i-auto-stub1/577:1699| I need to remove the lines which has... (9 Replies)
Discussion started by: giri_luck
9 Replies

4. Shell Programming and Scripting

Print lines between two repetitive patterns

Hi users I have one file which has number of occurrence of one pattern examples Adjustmenttype,11 xyz 10 dwe 9 abd 13 def 14 Adjustmenttype,11 xyz 24 dwe 34 abd 35 def 11 nmb 12 Adjustmenttype, not eleven .... ... ... (2 Replies)
Discussion started by: eranmoh
2 Replies

5. Shell Programming and Scripting

Remove a range of lines from a file using sed

Hi I am having some issue editing a file in sed. What I want to do is, in a loop pass a variable to a sed command. Sed should then search a file for a line that matches that variable, then remove all lines below until it reaches a line starting with a constant. I have managed to write a... (14 Replies)
Discussion started by: Andy82
14 Replies

6. UNIX for Dummies Questions & Answers

How to remove certain lines using sed?

Hi I have the following kind of line sin my file . print ' this is first'. print ' this is firs and next ' ' line continuous '. -- this is entire print line. print ' this is first and next ' ' line continuous and' 'still there now over'. -- this 3lines together a single print line. ... (5 Replies)
Discussion started by: Sivajee
5 Replies

7. Shell Programming and Scripting

How to remove certain lines using sed?

Hi, I am new to unix and i started some scripting recently. Please go through the following script i wrote. #!/bin/sh file='path../tfile' file1='path../tfile1' rmfile='path../test2' C1=1 C2=1 exec 3< $file1 while read LINE1; do read LINE2 <&3 a=$LINE1 b=`expr $LINE2 - 1` ... (1 Reply)
Discussion started by: Subbu123
1 Replies

8. Shell Programming and Scripting

Using sed in a loop/to remove lines contained in variable from file

I've tried numerous commands, but I am not sure how to use sed in a loop. This is what I have: VARZ contains CARD_FILE_LIST and it also contains CARD_FILE_LIST2 so echo "$VARZ" CARD_FILE_LIST CARD_FILE_LIST2 I have a file with 60 lines in /tmp/testfile it and I want those lines deleted... (3 Replies)
Discussion started by: newbie2010
3 Replies

9. Shell Programming and Scripting

I need help to removing repetitive lines

Hello SuperUsers! First i wanna say my english sucks.. Don't hate me for that. :rolleyes: I need to make a Bash for removing smilar lines from an output file. My output file always same. Line 1 & 2 Stays. And others similar to this lines needs to be delete. </UsageData><?xml version="1.0"... (4 Replies)
Discussion started by: morphin
4 Replies

10. Shell Programming and Scripting

sed to remove all lines in file that are not .vcf.gz extention

I am trying to use sed to remove all lines in a file that are nor vcf.gz. The sed below runs but returns all the files with vcf.gz in them, rather then just the ones that end in only that extention. Thank you :). file ... (9 Replies)
Discussion started by: cmccabe
9 Replies
IO::Async::Timer::Absolute(3pm) 			User Contributed Perl Documentation			   IO::Async::Timer::Absolute(3pm)

NAME
"IO::Async::Timer::Absolute" - event callback at a fixed future time SYNOPSIS
use IO::Async::Timer::Absolute; use POSIX qw( mktime ); use IO::Async::Loop; my $loop = IO::Async::Loop->new; my @time = gmtime; my $timer = IO::Async::Timer::Absolute->new( time => mktime( 0, 0, 0, $time[4]+1, $time[5], $time[6] ), on_expire => sub { print "It's midnight "; $loop->stop; }, ); $loop->add( $timer ); $loop->run; DESCRIPTION
This subclass of IO::Async::Timer implements one-shot events at a fixed time in the future. The object waits for a given timestamp, and invokes its callback at that point in the future. For a "Timer" object that waits for a delay relative to the time it is started, see instead IO::Async::Timer::Countdown. EVENTS
The following events are invoked, either using subclass methods or CODE references in parameters: on_expire Invoked when the timer expires. PARAMETERS
The following named parameters may be passed to "new" or "configure": on_expire => CODE CODE reference for the "on_expire" event. time => NUM The epoch time at which the timer will expire. Once constructed, the timer object will need to be added to the "Loop" before it will work. Unlike other timers, it does not make sense to "start" this object, because its expiry time is absolute, and not relative to the time it is started. AUTHOR
Paul Evans <leonerd@leonerd.org.uk> perl v5.14.2 2012-10-24 IO::Async::Timer::Absolute(3pm)
All times are GMT -4. The time now is 06:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy