Sponsored Content
Top Forums Shell Programming and Scripting Find and replace duplicate column values in a row Post 302413336 by nuthalapati on Thursday 15th of April 2010 09:33:35 AM
Old 04-15-2010
Find and replace duplicate column values in a row

I have file which as 12 columns and values like this

Code:
 
1,2,3,4,5
a,b,c,d,e
b,c,a,e,f
a,b,e,a,h

if you see the first column has duplicate values, I need to identify (print it to console) the duplicate value (which is 'a') and also remove duplicate values like below. I could be in two different scripts that fine.

Code:
 
1,2,3,4,5
a,b,c,d,e
b,c,a,e,f
,,e,a,h

Appreciate your help !

Last edited by zaxxon; 04-15-2010 at 10:47 AM.. Reason: code tags please
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete a row that has a duplicate column

I'm trying to remove lines of data that contain duplicate data in a specific column. For example. apple 12345 apple 54321 apple 14234 orange 55656 orange 88989 orange 99898 I only want to see apple 12345 orange 55656 How would i go about doing this? (5 Replies)
Discussion started by: spartan22
5 Replies

2. Shell Programming and Scripting

Convert column values into row

hi, I have a requirement where in I read the values from a file using awk. The resulting data should be converted into row format from column format. For ex: My log file login.lst contains the following SERVER1 DB1 SERVER2 DB2 SERVER3 DB3 SERVER4 DB4 I use awk to grep only the server... (6 Replies)
Discussion started by: senthil3d
6 Replies

3. Shell Programming and Scripting

Converting values in a ROW to COLUMN

Hi All, I needd to convert values in a row to a column. eg: Input is as: value1,value2,value3,value4,.........,value N Required Output: Value1 Value2 Value3 . . . Value N Please help.... (3 Replies)
Discussion started by: sambaman
3 Replies

4. Shell Programming and Scripting

duplicate row based on single column

I am a newbie to shell scripting .. I have a .csv file. It has 1000 some rows and about 7 columns... but before I insert this data to a table I have to parse it and clean it ..basing on the value of the first column..which a string of phone number type... example below.. column 1 ... (2 Replies)
Discussion started by: mitr
2 Replies

5. Shell Programming and Scripting

Print every 5 4th column values as separate row with different first column

Hi, I have the following file, chr1 100 200 20 chr1 201 300 22 chr1 220 345 23 chr1 230 456 33.5 chr1 243 567 90 chr1 345 600 20 chr1 430 619 21.78 chr1 870 910 112.3 chr1 914 920 12 chr1 930 999 13 My output would be peak1 20 22 23 33.5 90 peak2 20 21.78 112.3 12 13 Here the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

6. UNIX for Dummies Questions & Answers

awk to sum column field from duplicate row/lines

Hello, I am new to Linux environment , I working on Linux script which should send auto email based on the specific condition from log file. Below is the sample log file Name m/c usage abc xxx 10 abc xxx 20 abc xxx 5 xyz ... (6 Replies)
Discussion started by: asjaiswal
6 Replies

7. Programming

How to add one to each row values and keep it after the value in the column?

Dear Folks Hello I have a column of numbers, say: 26 79 68 I want to add one to each row value and get this desire column: 26 27 79 80 68 69 (6 Replies)
Discussion started by: sajmar
6 Replies

8. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

9. Shell Programming and Scripting

Do replace operation and awk to sum multiple columns if another column has duplicate values

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (12 Replies)
Discussion started by: as7951
12 Replies

10. UNIX for Beginners Questions & Answers

Find lines with duplicate values in a particular column

I have a file with 5 columns. I want to pull out all records where the value in column 4 is not unique. For example in the sample below, I would want it to print out all lines except for the last two. 40991764 2419 724 47182 Cand A 40992936 3591 724 47182 Cand B 40993016 3671 724 47182 Cand C... (5 Replies)
Discussion started by: kaktus
5 Replies
MYSQLINDEXCHECK(1)						  MySQL Utilities						MYSQLINDEXCHECK(1)

NAME
mysqlindexcheck - Check for redundant or duplicate indexes on a list tables or databases SYNOPSIS
mysqlindexcheck [options] db[:table] ... DESCRIPTION
This utility reads the indexes for one or more tables and identifies duplicate and potentially redundant indexes. To check all tables in a database, specify only the database name. To check a specific table, name the table in db.table format. It is pos- sible to mix database and table names. You can scan tables in any database except the internal databases mysql, INFORMATION_SCHEMA, and performance_schema. Depending on the index type, the utility applies the following rules to compare indexes (designated as idx_a and idx_b): BTREE idx_b is redundant to idx_a if and only if the first n columns in idx_b also appear in idx_a. Order and uniqueness count. HASH idx_a and idx_b are duplicates if and only if they contain the same columns in the same order. Uniqueness counts. SPATIAL idx_a and idx_b are duplicates if and only if they contain the same column (only one column is permitted). FULLTEXT idx_b is redundant to idx_a if and only if all columns in idx_b are included in idx_a. Order counts. To see DROP statements to drop redundant indexes, specify the --show-drops option. To examine the existing indexes, use the --verbose option, which prints the equivalent CREATE INDEX (or ALTER TABLE for primary keys. To display the best or worst nonprimary key indexes for each table, use the --best or --worst option. This causes the output to show the best or worst indexes from tables with 10 or more rows. By default, each option shows five indexes. To override that, provide an integer value for the option. To change the format of the index lists displayed for the --show-indexes, --best, and --worst options, use one of the following values with the --format option: grid (default) Display output in grid or table format like that of the mysql monitor. csv Display output in comma-separated values format. tab Display output in tab-separated format. sql print SQL statements rather than a list. vertical Display output in single-column format like that of the G command for the mysql monitor. Note: The --best and --worst lists cannot be printed as SQL statements. OPTIONS
mysqlindexcheck accepts the following command-line options: --help Display a help message and exit. --best[=<N>] If --stats is given, limit index statistics to the best N indexes. The default value of N is 5 if omitted. --format=<index_format>, -f<index_format> Specify the index list display format for output produced by --stats. Permitted format values are grid, csv, tab, sql, and vertical. The default is grid. --server=<source> Connection information for the server in <user>[:<passwd>]@<host>[:<port>][:<socket>] format. --show-drops, -d Display DROP statements for dropping indexes. --show-indexes, -i Display indexes for each table. --skip, -s Skip tables that do not exist. --stats Show index performance statistics. --verbose, -v Specify how much information to display. Use this option multiple times to increase the amount of information. For example, -v = verbose, -vv = more verbose, -vvv = debug. --version Display version information and exit. --worst[=<N>] If --stats is given, limit index statistics to the worst N indexes. The default value of N is 5 if omitted. NOTES
You must provide connection parameters (user, host, password, and so forth) for an account that has the appropriate privileges to read all objects accessed during the operation. For the --format option, the permitted values are not case sensitive. In addition, values may be specified as any unambiguous prefix of a valid value. For example, --format=g specifies the grid format. An error occurs if a prefix matches more than one valid value. EXAMPLES
To check all tables in the employees database on the local server to see the possible redundant and duplicate indexes, use this command: $ mysqlindexcheck --server=root@localhost employees # Source on localhost: ... connected. # The following indexes are duplicates or redundant for table employees.dept_emp: # CREATE INDEX emp_no ON employees.dept_emp (emp_no) USING BTREE # may be redundant or duplicate of: ALTER TABLE employees.dept_emp ADD PRIMARY KEY (emp_no, dept_no) # The following indexes are duplicates or redundant for table employees.dept_manager: # CREATE INDEX emp_no ON employees.dept_manager (emp_no) USING BTREE # may be redundant or duplicate of: ALTER TABLE employees.dept_manager ADD PRIMARY KEY (emp_no, dept_no) # The following indexes are duplicates or redundant for table employees.salaries: # CREATE INDEX emp_no ON employees.salaries (emp_no) USING BTREE # may be redundant or duplicate of: ALTER TABLE employees.salaries ADD PRIMARY KEY (emp_no, from_date) # The following indexes are duplicates or redundant for table employees.titles: # CREATE INDEX emp_no ON employees.titles (emp_no) USING BTREE # may be redundant or duplicate of: ALTER TABLE employees.titles ADD PRIMARY KEY (emp_no, title, from_date) COPYRIGHT
Copyright (c) 2010, 2012, Oracle and/or its affiliates. All rights reserved. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER- CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA AUTHOR
MySQL Utilities Team COPYRIGHT
2010, Oracle and/or its affiliates. All rights reserved. 1.0.3 May 09, 2012 MYSQLINDEXCHECK(1)
All times are GMT -4. The time now is 05:16 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy