How to remove duplicated lines?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicated records and update last line record counts

Hi Gurus, I need to remove duplicate line in file and update TRAILER (last line) record count. the file is comma delimited, field 2 is key to identify duplicated record. I can use below command to remove duplicated. but don't know how to replace last line 2nd field to new count. awk -F","...

2. AIX

Remove duplicated bootlist entries

Hello. I have a server with 2 boot disk but in the bootlist there are 5 paths of one disk but no path of the other. How can I remove paths from one disk to insert paths from the other disk? Thanks in advance. server074:root:/# bootlist -om normal hdisk0 blv=hd5 pathid=0 hdisk0...

3. Shell Programming and Scripting

How to remove duplicated column in a text file?

Dear all, How can I remove duplicated column in a text file? Input: LG10_PM_map_19_LEnd 1000560 G AA AA AA AA AA GG LG10_PM_map_19_LEnd 1005621 G GG GG GG AA AA GG LG10_PM_map_19_LEnd 1011214 A AA AA AA AA GG GG LG10_PM_map_19_LEnd 1011673 T TT TT TT TT CC CC...

4. Shell Programming and Scripting

Merge files and remove duplicated rows

In a folder I'll several times daily receive new files that I want to combine into one big file, without any duplicate rows. The file name in the folder will look like e.q: MissingData_2014-08-25_09-30-18.txt MissingData_2014-08-25_09-30-14.txt MissingData_2014-08-26_09-30-12.txt The content...

5. UNIX for Dummies Questions & Answers

Removing duplicated lines??

Hi Guys.. I have a problem for some reason my database has copied everything 4 times. My Database looks like this: >BAC233456 rhjieaheiohjteo tjtjrj6jkk6k6 j54ju54jh54jh >ANI124365 afrhtjykulilil htrjykuk rtkjryky ukrykyrk >BAC233456 rhjieaheiohjteo tjtjrj6jkk6k6 j54ju54jh54jh...

6. Shell Programming and Scripting

Remove rows with first 4 fields duplicated in awk

Hi, I am trying to use awk to remove all rows where the first 4 fields are duplicates. e.g. in the following data lines 6-9 would be removed, leaving one copy of the duplicated row (row 5) Borgarhraun FH9822 ol24 FH9822_ol24_m20 ol Deformed c Borgarhraun FH9822 ol24 ...

7. Shell Programming and Scripting

Help with remove duplicated content

Input file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hcmv-UL70-3p hsa-4486 hcms-US25 hsa-360-5 hcms-US25 hsa-4 hcms-US25 hsa-458 hcms-US25 hsa-44812 . . Desired Output file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hsa-4486...

8. Shell Programming and Scripting

remove duplicated columns

hi all, i have a file contain multicolumns, this file is sorted by col2 and col3. i want to remove the duplicated columns if the col2 and col3 are the same in another line. example fileA AA BB CC DD CC XX CC DD BB CC ZZ FF DD FF HH HH the output is AA BB CC DD BB CC ZZ FF...

9. Shell Programming and Scripting

remove duplicated lines without sort

Hi Just wondering whether or not I can remove duplicated lines without sort For example, I use the command who, which shows users who are logging on. In some cases, it shows duplicated lines of users who are logging on more than one terminal. Normally, I would do who | cut -d" " -f1 |...

10. Shell Programming and Scripting

remove duplicated xml record in a file under unix

Hi, If i have a file with xml format, i would like to remove duplicated records and save to a new file. Is it possible...to write script to do it?

LEARN ABOUT DEBIAN

class::dbi::query

Class::DBI::Query(3pm)					User Contributed Perl Documentation				    Class::DBI::Query(3pm)

NAME

       Class::DBI::Query - Deprecated SQL manager for Class::DBI

SYNOPSIS

	       my $sth = Class::DBI::Query
		       ->new({
			       owner => $class,
			       sqlname => $type,
			       essential => @columns,
			       where_columns => @where_cols,
		       })
		       ->run($val);

DESCRIPTION

       This abstracts away many of the details of the Class::DBI underlying SQL mechanism. For the most part you probably don't want to be
       interfacing directly with this.

       The underlying mechanisms are not yet stable, and are subject to change at any time.

OPTIONS

       A Query can have many options set before executing. Most can either be passed as an option to new(), or set later if you are building the
       query up dynamically:

   owner
       The Class::DBI subclass that 'owns' this query. In the vast majority of cases a query will return objects - the owner is the class of which
       instances will be returned.

   sqlname
       This should be the name of a query set up using set_sql.

   where_clause
       This is the raw SQL that will substituted into the 'WHERE %s' in your query. If you have multiple %s's in your query then you should supply
       a listref of where_clauses. This SQL can include placeholders, which will be used when you call run().

   essential
       When retrieving rows from the database that match the WHERE clause of the query, these are the columns that we fetch back and pre-load the
       resulting objects with. By default this is the Essential column group of the owner class.

METHODS

   where()
	       $query->where($match, @columns);

       This will extend your 'WHERE' clause by adding a 'AND $column = ?' (or whatever $match is, isntead of "=") for each column passed. If you
       have multiple WHERE clauses this will extend the last one.

perl v5.12.4							    2004-04-25						    Class::DBI::Query(3pm)

Shell Programming and Scripting