Sponsored Content
Top Forums Shell Programming and Scripting How to Remove duplicate value from file? Post 302770753 by zazzybob on Monday 18th of February 2013 08:08:25 AM
Old 02-18-2013
As long as you specify the correct path to your input file in the file variable in the script, it should work. In the example below the input file is foo.txt and it exists in the same directory as foo.sh, which is also your present working directory.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Duplicate Lines in File

I am doing KSH script to remove duplicate lines in a file. Let say the file has format below. FileA 1253-6856 3101-4011 1827-1356 1822-1157 1822-1157 1000-1410 1000-1410 1822-1231 1822-1231 3101-4011 1822-1157 1822-1231 and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies

2. UNIX for Dummies Questions & Answers

Remove Duplicate lines from File

I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Discussion started by: Nysif Steve
18 Replies

3. Shell Programming and Scripting

Sort and Remove Duplicate on file

How do we sort and remove duplicate on column 1,2 retaining the record with maximum date (in feild 3) for the file with following format. aaa|1234|2010-12-31 aaa|1234|2010-11-10 bbb|345|2011-01-01 ccc|346|2011-02-01 bbb|345|2011-03-10 aaa|1234|2010-01-01 Required Output ... (5 Replies)
Discussion started by: mabarif16
5 Replies

4. Shell Programming and Scripting

Formatting a file - Remove Duplicate

Hi I have a file in the following format. Basically the file contains tablename and their aliases: TABLE1 TABLE1 A TABLE2 TABLE2 B TABLE3 TABLE4 TABLE4 C TABLE4 Upon formatting an sql statement I am getting such output. Problem: Whenever a tablename appears with alias, it has... (5 Replies)
Discussion started by: freakygs
5 Replies

5. Shell Programming and Scripting

remove of duplicate line from a file

I have a file a.txt having content like deepak ram sham deepram sita kumar I Want to delete the first line containing "deep" ... I tried using... grep -i 'deep' a.txt It gives me 2 rows...I want to delete the first one.. + need to know the command to delete the line from... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

6. Shell Programming and Scripting

How do I remove the duplicate lines in this file?

Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file. I wrote the following script, but does not work. What is the problem? The output file should only contain five lines: Later! (5 Replies)
Discussion started by: Ernst
5 Replies

7. Shell Programming and Scripting

Remove the duplicate content in a file

Here is the contents of test.txt Dependencies Resolved Changes in packages about to be updated: ChangeLog for: 1:perl-Archive-Extract-0.38-131.el6_4.x86_64, - Resolves: #915692 - CVE-2013-1667 (DoS in rehashing code) Dependencies Resolved Changes in packages about to be updated: ... (5 Replies)
Discussion started by: ashokvpp
5 Replies

8. Shell Programming and Scripting

Remove duplicate lines from a file

Hi, I have a csv file which contains some millions of lines in it. The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line). I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies

9. Shell Programming and Scripting

Remove duplicate values in a column(not in the file)

Hi Gurus, I have a file(weblog) as below abc|xyz|123|agentcode=sample code abcdeeess,agentcode=sample code abcdeeess,agentcode=sample code abcdeeess|agentadd=abcd stereet 23343,agentadd=abcd stereet 23343 sss|wwq|999|agentcode=sample1 code wqwdeeess,gentcode=sample1 code... (4 Replies)
Discussion started by: ratheeshjulk
4 Replies
Mojo::Command(3pm)					User Contributed Perl Documentation					Mojo::Command(3pm)

NAME
Mojo::Command - Command base class SYNOPSIS
# Lower case command name package Mojolicious::Command::mycommand; # Subclass use Mojo::Base 'Mojo::Command'; # Take care of command line options use Getopt::Long 'GetOptions'; # Short description has description => "My first Mojo command. "; # Short usage message has usage => <<"EOF"; usage: $0 mycommand [OPTIONS] These options are available: -s, --something Does something. EOF # <suitable Futurama quote here> sub run { my $self = shift; # Handle options local @ARGV = @_; GetOptions('s|something' => sub { $something = 1 }); # Magic here! :) } DESCRIPTION
Mojo::Command is an abstract base class for Mojo commands. See Mojolicious::Commands for a list of commands that are available by default. ATTRIBUTES
Mojo::Command implements the following attributes. "description" my $description = $command->description; $command = $command->description('Foo!'); Short description of command, used for the command list. "quiet" my $quiet = $command->quiet; $command = $command->quiet(1); Limited command output. "usage" my $usage = $command->usage; $command = $command->usage('Foo!'); Usage information for command, used for the help screen. METHODS
Mojo::Command inherits all methods from Mojo::Base and implements the following new ones. "app" my $app = $command->app; Currently active application. # Introspect say "Template path: $_" for @{$command->app->renderer->paths}; "chmod_file" $command = $command->chmod_file('/home/sri/foo.txt', 0644); Portably change mode of a file. "chmod_rel_file" $command = $command->chmod_rel_file('foo/foo.txt', 0644); Portably change mode of a file relative to the current working directory. "class_to_file" my $file = $command->class_to_file('Foo::Bar'); Convert a class name to a file. Foo::Bar -> foo_bar FOO::Bar -> foobar FooBar -> foo_bar FOOBar -> foobar "class_to_path" my $path = $command->class_to_path('Foo::Bar'); Convert class name to path. Foo::Bar -> Foo/Bar.pm "create_dir" $command = $command->create_dir('/home/sri/foo/bar'); Portably create a directory. "create_rel_dir" $command = $command->create_rel_dir('foo/bar/baz'); Portably create a directory relative to the current working directory. "get_all_data" my $all = $command->get_all_data; my $all = $command->get_all_data('Some::Class'); Extract all embedded files from the "DATA" section of a class. "get_data" my $data = $command->get_data('foo_bar'); my $data = $command->get_data('foo_bar', 'Some::Class'); Extract embedded file from the "DATA" section of a class. "help" $command->help; Print usage information for command. "rel_dir" my $path = $command->rel_dir('foo/bar'); Portably generate an absolute path for a directory relative to the current working directory. "rel_file" my $path = $command->rel_file('foo/bar.txt'); Portably generate an absolute path for a file relative to the current working directory. "render_data" my $data = $command->render_data('foo_bar', @args); Render a template from the "DATA" section of the command class. "render_to_file" $command = $command->render_to_file('foo_bar', '/home/sri/foo.txt'); Render a template from the "DATA" section of the command class to a file. "render_to_rel_file" $command = $command->render_to_rel_file('foo_bar', 'foo/bar.txt'); Portably render a template from the "DATA" section of the command class to a file relative to the current working directory. "run" $command->run; $command->run(@ARGV); Run command. Meant to be overloaded in a subclass. "write_file" $command = $command->write_file('/home/sri/foo.txt', 'Hello World!'); Portably write text to a file. "write_rel_file" $command = $command->write_rel_file('foo/bar.txt', 'Hello World!'); Portably write text to a file relative to the current working directory. SEE ALSO
Mojolicious, Mojolicious::Guides, <http://mojolicio.us>. perl v5.14.2 2012-09-05 Mojo::Command(3pm)
All times are GMT -4. The time now is 10:53 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy