How to delete a duplicate line and original with sed.
I am completely new to shell scripting but have been assigned the task of creating several batch files to manipulate data. My final task requires me to find lines that have duplicates present then delete not only the duplicate but the original as well. The script will be used in a windows environment so I am using GNU sed. Below is a sample of the data:
In this example the first and last lines are duplicates and I would like to delete them both. I have been searching for several days and have not been able to figure out how to achieve this . Unfortunately I am short on time and would greatly appreciate any help possible. Thanks.
Last edited by Franklin52; 06-20-2011 at 03:32 AM..
Reason: Please use code tags
Hi,
I want to write a sed script which from
batiato:
batiato/giubbe:
pip_b.2.txt
pip_b.3.txt
pip_b.3mmm.txt
bennato:
bennato/peterpan:
123.txt
consoli:
pip_a.12.txt
daniele: (2 Replies)
Please let me know wat would be sed command to delete any partcular line from a file and also moving lines below it to up.
ie wen line #9 is deleted data in line #10 should move to #9 and so on. (2 Replies)
I'm new to using sed and grep commands, but have found them extremely useful. However I am having a hard time figuring this one out:
Delete every line containing the word CEN and the next line as well.
ie. test.txt
blue
324 CEN
green
red
blue
324 CEN
green
red
blue
to produce:... (2 Replies)
Hi ALL
I need a help
I need to retain only the first line of 035 if I have two line before =040 , if only one then need to take that
Eg:
Input
=035 (ABC)12324141241
=035 (XYZPQR)704124
=040 AB$QS$WEWR
=035 (ABC)08080880809
=035 (XYZPQR)9809314
=040 ... (4 Replies)
I need help in figuring out hoe to delete lines in a data file. The data file is huge. I am currently using "vi" to search and delete the lines - which is cumbersome since it takes lots of time to save that file (due to its huge size).
Here is the issue. I have a data file with the following... (4 Replies)
here is what i want to achieve.. i have a file with below contents
cat fileName
blah blah blah
.
.DROP this
REJECT that
.
--sport 7800 -j REJECT --reject-with icmp-port-unreachable
--dport 7800 -j REJECT --reject-with icmp-port-unreachable
.
.
.
more blah blah blah
--dport 3306... (14 Replies)
There are some duplicate field on description column .I want to print duplicate row along with highest version of number and corresponding description column.
file1.txt
number Description
=== ============
34567 nl21a00is-centerdb001:ncdbareq:Error in loading init
34577 ... (7 Replies)
Hi,
I have an input file like this
line1
line2
line3 hello unix how are you
This is what I am expecting my output to be
line1
line2
#line3 hello unix how are you
line3 hello (3 Replies)
Hi,
i need help to remove duplicates in my file. The problem is i need to delete one duplicate for each line only. the input file as follows and it is not tab delimited:-
The output need to remove 2nd word (in red) that duplicate with 1st word (in blue). Other duplicates should remained... (12 Replies)
Hello All !
I need your help on this case,
I have a csv file with this:
ITEM105;ARI FSR;2016-02-01 08:02;243
ITEM101;ARI FSR;2016-02-01 06:02;240
ITEM032;RNO TLE;2016-02-01 11:03;320
ITEM032;RNO TLE;2016-02-02 05:43;320
ITEM032;RNO TLE;2016-02-01 02:03;320
ITEM032;RNO... (2 Replies)
Discussion started by: vadim-bzh
2 Replies
LEARN ABOUT OSX
shar
SHAR(1) BSD General Commands Manual SHAR(1)NAME
shar -- create a shell archive of files
SYNOPSIS
shar file ...
DESCRIPTION
shar writes an sh(1) shell script to the standard output which will recreate the file hierarchy specified by the command line operands.
Directories will be recreated and must be specified before the files they contain (the find(1) utility does this correctly).
shar is normally used for distributing files by ftp(1) or mail(1).
SEE ALSO compress(1), mail(1), tar(1), uuencode(1)BUGS
shar makes no provisions for special types of files or files containing magic characters.
EXAMPLES
To create a shell archive of the program ls(1) and mail it to Rick:
cd ls
shar `find . -print` | mail -s "ls source" rick
To recreate the program directory:
mkdir ls
cd ls
...
<delete header lines and examine mailed archive>
...
sh archive
HISTORY
The shar command appears in 4.4BSD.
SECURITY CONSIDERATIONS
It is easy to insert trojan horses into shar files. It is strongly recommended that all shell archive files be examined before running them
through sh(1). Archives produced using this implementation of shar may be easily examined with the command:
egrep -v '^[X#]' shar.file
4.4BSD June 6, 1993 4.4BSD
06-14-2011
. Unfortunately I am short on time and would greatly appreciate any help possible. Thanks.