Sponsored Content
Top Forums UNIX for Dummies Questions & Answers awk solution to duplicate lines based on column Post 302862435 by torchij on Thursday 10th of October 2013 06:01:38 PM
Old 10-10-2013
awk solution to duplicate lines based on column

Hi experts, I have a tab-delimited file with one column containing values separated by a comma. I wish to duplicate the entire line for every value in that comma-delimited field.

For example:

Code:
$cat file
4444     4444            4444     4444
9990     2222,7777       6666     2222   <---this one
1900     1111            2222     4444
1800     0000            5555     8989
1700     3333,4444,5555  8787     4444   <---this one

Code:
$cat output
4444     4444  4444     4444
9990     2222  6666     2222  <---duplicate1
9990     7777  6666     2222  <---duplicate2
1900     1111  2222     4444
1800     0000  5555     8989
1700     3333  8787     4444   <---duplicate1
1700     4444  8787     4444   <---duplicate2
1700     5555  8787     4444   <---duplicate3

Many thanks in advance for your help!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

duplicate row based on single column

I am a newbie to shell scripting .. I have a .csv file. It has 1000 some rows and about 7 columns... but before I insert this data to a table I have to parse it and clean it ..basing on the value of the first column..which a string of phone number type... example below.. column 1 ... (2 Replies)
Discussion started by: mitr
2 Replies

2. Shell Programming and Scripting

AWK Duplicate lines multiple times based on a calculated value

Hi, I'm trying to create an XML sitemap of our dynamic ecommerce sites SEO Friendly URLs and am trying to create the initial page listing. I have a CSV file that looks like the following and need duplicate the lines based on a value which needs calculating. ... (2 Replies)
Discussion started by: jamesfx
2 Replies

3. Shell Programming and Scripting

awk print non matching lines based on column

My item was not answered on previous thread as code given did not work I wanted to print records from file2 where comparing column 1 and 16 for both files find rows where column 16 in file 1 does not match column 16 in file 2 Here was CODE give to issue ~/unix.com$ cat f1... (0 Replies)
Discussion started by: sigh2010
0 Replies

4. Shell Programming and Scripting

Perl: filtering lines based on duplicate values in a column

Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times. 13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG 13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG 13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT 13 18 1 + chromosome 1,... (5 Replies)
Discussion started by: polsum
5 Replies

5. UNIX for Dummies Questions & Answers

awk to sum column field from duplicate row/lines

Hello, I am new to Linux environment , I working on Linux script which should send auto email based on the specific condition from log file. Below is the sample log file Name m/c usage abc xxx 10 abc xxx 20 abc xxx 5 xyz ... (6 Replies)
Discussion started by: asjaiswal
6 Replies

6. Shell Programming and Scripting

awk to sum a column based on duplicate strings in another column and show split totals

Hi, I have a similar input format- A_1 2 B_0 4 A_1 1 B_2 5 A_4 1 and looking to print in this output format with headers. can you suggest in awk?awk because i am doing some pattern matching from parent file to print column 1 of my input using awk already.Thanks! letter number_of_letters... (5 Replies)
Discussion started by: prashob123
5 Replies

7. Shell Programming and Scripting

Remove duplicate rows based on one column

Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed. example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies

8. Shell Programming and Scripting

Removing duplicate lines on first column based with pipe delimiter

Hi, I have tried to remove dublicate lines based on first column with pipe delimiter . but i ma not able to get some uniqu lines Command : sort -t'|' -nuk1 file.txt Input : 38376KZ|09/25/15|1.057 38376KZ|09/25/15|1.057 02006YB|09/25/15|0.859 12593PS|09/25/15|2.803... (2 Replies)
Discussion started by: parithi06
2 Replies

9. Shell Programming and Scripting

Solution for replacement of 4th column with 3rd column in a file using awk/sed preserving delimters

input "A","B","C,D","E","F" "S","T","U,V","W","X" "AA","BB","CC,DD","EEEE","FFF" required output: "A","B","C,D","C,D","F" "S", T","U,V","U,V","X" "AA","BB","CC,DD","CC,DD","FFF" tried using awk but double quotes not preserving for every field. any help to solve this is much... (5 Replies)
Discussion started by: khblts
5 Replies

10. Shell Programming and Scripting

awk to select lines with maximum value of each record based on column value

Hello, I want to get the maximum value of each record separated by empty line based on the 3rd column of each row within each record? Input: A1 chr5D 634 7 82 707 A2 chr5D 637 6 82 713 A3 chr5D 637 5 82 713 A4 chr5D 626 1 82 704... (4 Replies)
Discussion started by: yifangt
4 Replies
DBI::SQL::Nano(3)					User Contributed Perl Documentation					 DBI::SQL::Nano(3)

NAME
DBI::SQL::Nano - a very tiny SQL engine SYNOPSIS
BEGIN { $ENV{DBI_SQL_NANO}=1 } # forces use of Nano rather than SQL::Statement use DBI::SQL::Nano; use Data::Dumper; my $stmt = DBI::SQL::Nano::Statement->new( "SELECT bar,baz FROM foo WHERE qux = 1" ) or die "Couldn't parse"; print Dumper $stmt; DESCRIPTION
DBI::SQL::Nano is meant as a *very* minimal SQL engine for use in situations where SQL::Statement is not available. In most situations you are better off installing SQL::Statement although DBI::SQL::Nano may be faster for some very simple tasks. DBI::SQL::Nano, like SQL::Statement is primarily intended to provide a SQL engine for use with some pure perl DBDs including DBD::DBM, DBD::CSV, DBD::AnyData, and DBD::Excel. It isn't of much use in and of itself. You can dump out the structure of a parsed SQL statement, but that's about it. USAGE
Setting the DBI_SQL_NANO flag By default, when a DBD uses DBI::SQL::Nano, the module will look to see if SQL::Statement is installed. If it is, SQL::Statement objects are used. If SQL::Statement is not available, DBI::SQL::Nano objects are used. In some cases, you may wish to use DBI::SQL::Nano objects even if SQL::Statement is available. To force usage of DBI::SQL::Nano objects regardless of the availability of SQL::Statement, set the environment variable DBI_SQL_NANO to 1. You can set the environment variable in your shell prior to running your script (with SET or EXPORT or whatever), or else you can set it in your script by putting this at the top of the script: BEGIN { $ENV{DBI_SQL_NANO} = 1 } Supported SQL syntax Here's a pseudo-BNF. Square brackets [] indicate optional items; Angle brackets <> indicate items defined elsewhere in the BNF. statement ::= DROP TABLE [IF EXISTS] <table_name> | CREATE TABLE <table_name> <col_def_list> | INSERT INTO <table_name> [<insert_col_list>] VALUES <val_list> | DELETE FROM <table_name> [<where_clause>] | UPDATE <table_name> SET <set_clause> <where_clause> | SELECT <select_col_list> FROM <table_name> [<where_clause>] [<order_clause>] the optional IF EXISTS clause ::= * similar to MySQL - prevents errors when trying to drop a table that doesn't exist identifiers ::= * table and column names should be valid SQL identifiers * especially avoid using spaces and commas in identifiers * note: there is no error checking for invalid names, some will be accepted, others will cause parse failures table_name ::= * only one table (no multiple table operations) * see identifier for valid table names col_def_list ::= * a parens delimited, comma-separated list of column names * see identifier for valid column names * column types and column constraints may be included but are ignored e.g. these are all the same: (id,phrase) (id INT, phrase VARCHAR(40)) (id INT PRIMARY KEY, phrase VARCHAR(40) NOT NULL) * you are *strongly* advised to put in column types even though they are ignored ... it increases portability insert_col_list ::= * a parens delimited, comma-separated list of column names * as in standard SQL, this is optional select_col_list ::= * a comma-separated list of column names * or an asterisk denoting all columns val_list ::= * a parens delimited, comma-separated list of values which can be: * placeholders (an unquoted question mark) * numbers (unquoted numbers) * column names (unquoted strings) * nulls (unquoted word NULL) * strings (delimited with single quote marks); * note: leading and trailing percent mark (%) and underscore (_) can be used as wildcards in quoted strings for use with the LIKE and CLIKE operators * note: escaped single quote marks within strings are not supported, neither are embedded commas, use placeholders instead set_clause ::= * a comma-separated list of column = value pairs * see val_list for acceptable value formats where_clause ::= * a single "column/value <op> column/value" predicate, optionally preceded by "NOT" * note: multiple predicates combined with ORs or ANDs are not supported * see val_list for acceptable value formats * op may be one of: < > >= <= = <> LIKE CLIKE IS * CLIKE is a case insensitive LIKE order_clause ::= column_name [ASC|DESC] * a single column optional ORDER BY clause is supported * as in standard SQL, if neither ASC (ascending) nor DESC (descending) is specified, ASC becomes the default ACKNOWLEDGEMENTS
Tim Bunce provided the original idea for this module, helped me out of the tangled trap of namespace, and provided help and advice all along the way. Although I wrote it from the ground up, it is based on Jochen Weidmann's orignal design of SQL::Statement, so much of the credit for the API goes to him. AUTHOR AND COPYRIGHT
This module is written and maintained by Jeff Zucker < jzucker AT cpan.org > Copyright (C) 2004 by Jeff Zucker, all rights reserved. You may freely distribute and/or modify this module under the terms of either the GNU General Public License (GPL) or the Artistic License, as specified in the Perl README file. perl v5.12.1 2007-07-16 DBI::SQL::Nano(3)
All times are GMT -4. The time now is 08:39 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy