How to delete a duplicate line and original with sed.
I am completely new to shell scripting but have been assigned the task of creating several batch files to manipulate data. My final task requires me to find lines that have duplicates present then delete not only the duplicate but the original as well. The script will be used in a windows environment so I am using GNU sed. Below is a sample of the data:
In this example the first and last lines are duplicates and I would like to delete them both. I have been searching for several days and have not been able to figure out how to achieve this . Unfortunately I am short on time and would greatly appreciate any help possible. Thanks.
Last edited by Franklin52; 06-20-2011 at 03:32 AM..
Reason: Please use code tags
Hi,
I want to write a sed script which from
batiato:
batiato/giubbe:
pip_b.2.txt
pip_b.3.txt
pip_b.3mmm.txt
bennato:
bennato/peterpan:
123.txt
consoli:
pip_a.12.txt
daniele: (2 Replies)
Please let me know wat would be sed command to delete any partcular line from a file and also moving lines below it to up.
ie wen line #9 is deleted data in line #10 should move to #9 and so on. (2 Replies)
I'm new to using sed and grep commands, but have found them extremely useful. However I am having a hard time figuring this one out:
Delete every line containing the word CEN and the next line as well.
ie. test.txt
blue
324 CEN
green
red
blue
324 CEN
green
red
blue
to produce:... (2 Replies)
Hi ALL
I need a help
I need to retain only the first line of 035 if I have two line before =040 , if only one then need to take that
Eg:
Input
=035 (ABC)12324141241
=035 (XYZPQR)704124
=040 AB$QS$WEWR
=035 (ABC)08080880809
=035 (XYZPQR)9809314
=040 ... (4 Replies)
I need help in figuring out hoe to delete lines in a data file. The data file is huge. I am currently using "vi" to search and delete the lines - which is cumbersome since it takes lots of time to save that file (due to its huge size).
Here is the issue. I have a data file with the following... (4 Replies)
here is what i want to achieve.. i have a file with below contents
cat fileName
blah blah blah
.
.DROP this
REJECT that
.
--sport 7800 -j REJECT --reject-with icmp-port-unreachable
--dport 7800 -j REJECT --reject-with icmp-port-unreachable
.
.
.
more blah blah blah
--dport 3306... (14 Replies)
There are some duplicate field on description column .I want to print duplicate row along with highest version of number and corresponding description column.
file1.txt
number Description
=== ============
34567 nl21a00is-centerdb001:ncdbareq:Error in loading init
34577 ... (7 Replies)
Hi,
I have an input file like this
line1
line2
line3 hello unix how are you
This is what I am expecting my output to be
line1
line2
#line3 hello unix how are you
line3 hello (3 Replies)
Hi,
i need help to remove duplicates in my file. The problem is i need to delete one duplicate for each line only. the input file as follows and it is not tab delimited:-
The output need to remove 2nd word (in red) that duplicate with 1st word (in blue). Other duplicates should remained... (12 Replies)
Hello All !
I need your help on this case,
I have a csv file with this:
ITEM105;ARI FSR;2016-02-01 08:02;243
ITEM101;ARI FSR;2016-02-01 06:02;240
ITEM032;RNO TLE;2016-02-01 11:03;320
ITEM032;RNO TLE;2016-02-02 05:43;320
ITEM032;RNO TLE;2016-02-01 02:03;320
ITEM032;RNO... (2 Replies)
Discussion started by: vadim-bzh
2 Replies
LEARN ABOUT DEBIAN
rra-appointment-to-vevent
synce-partnership(1) http://synce.sourceforge.net/ synce-partnership(1)NAME
rra-appointment-from-vevent, rra-appointment-to-vevent, rra-contact-from-vcard, rra-contact-to-vcard, rra-decode, rra-delete, rra-get-data,
rra-get-ids, rra-get-recurring-appointments, rra-get-types, rra-put-data, rra-subscribe, rra-task-from-vtodo, rra-task-to-vtodo, rra-time-
zone - Tools for manipulating the WinCE Database
SYNOPSIS
rra-appointment-from-vevent [-d LEVEL] [-c CODEPAGE] [-u] [-t TZFILE] VEVENT_FILE APPOINTMENT_FILE
rra-appointment-to-vevent [-d LEVEL] [-c CODEPAGE] [-u] [-t TZFILE] APPOINTMENT_FILE [VEVENT_FILE]
rra-contact-from-vcard [-d LEVEL] [-c CODEPAGE] [-u] VCARD_FILE CONTACT_FILE
rra-contact-to-vcard [-d LEVEL] [-c CODEPAGE] [-u] CONTACT_FILE [VCARD_FILE]
rra-decode FILE
rra-delete [-d LEVEL] TYPE-ID OBJECT-ID ...
rra-get-data [-d LEVEL] TYPE-ID OBJECT-ID FILENAME
rra-get-ids [-d LEVEL] TYPE-ID ...
rra-get-recurring-appointments
rra-get-types [-d LEVEL]
rra-put-data TYPE-ID OBJECT-ID FLAGS FILENAME
rra-subscribe [-d LEVEL] [TYPE-ID] [TYPE-ID] ...
rra-task-from-vtodo [-d LEVEL] [-c CODEPAGE] [-u] [-t TZFILE] VTODO_FILE TASK_FILE
rra-task-to-vtodo [-d LEVEL] [-c CODEPAGE] [-u] [-t TZFILE] TASK_FILE [VTODO_FILE]
rra-timezone [-d LEVEL] [-h] [-i INPUT-FILE] [-o OUTPUT-FILE] [-b] [-r] [-v]
DESCRIPTION
This small tools could be used to manipulate the database of a WinCE device.
AUTHOR
This manual page was written by Jonathan McDowell <noodles@earth.li>
SEE ALSO synce(1)The SynCE project September 2003 synce-partnership(1)
06-14-2011
. Unfortunately I am short on time and would greatly appreciate any help possible. Thanks.