Sponsored Content
Top Forums Shell Programming and Scripting How to remove duplicated column in a text file? Post 302931594 by MadeInGermany on Wednesday 14th of January 2015 04:54:51 AM
Old 01-14-2015
More common practice for processing the fields is a for loop.
The following does not print a trailing space.
Code:
awk '{ for (i=1;i<=NF;i++) {if (!a[$i]++) printf d $i; d=FS}; d=""; split("",a); print "" }' input

A % in the input would confuse printf. More robust is
Code:
awk '{ for (i=1;i<=NF;i++) {if (!a[$i]++) printf "%s",d $i; d=FS}; d=""; split("",a); print "" }' input

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove duplicated xml record in a file under unix

Hi, If i have a file with xml format, i would like to remove duplicated records and save to a new file. Is it possible...to write script to do it? (8 Replies)
Discussion started by: happyv
8 Replies

2. Shell Programming and Scripting

remove duplicated lines without sort

Hi Just wondering whether or not I can remove duplicated lines without sort For example, I use the command who, which shows users who are logging on. In some cases, it shows duplicated lines of users who are logging on more than one terminal. Normally, I would do who | cut -d" " -f1 |... (6 Replies)
Discussion started by: lalelle
6 Replies

3. Shell Programming and Scripting

remove duplicated columns

hi all, i have a file contain multicolumns, this file is sorted by col2 and col3. i want to remove the duplicated columns if the col2 and col3 are the same in another line. example fileA AA BB CC DD CC XX CC DD BB CC ZZ FF DD FF HH HH the output is AA BB CC DD BB CC ZZ FF... (6 Replies)
Discussion started by: kamel.seg
6 Replies

4. Shell Programming and Scripting

Remove part of a column in a text file

I have a text file with delimiter "|" and sometimes the zipcode is in "5th" column or "6th" column. I want to scan the file and remove the "-2323" from the zipcode which has zip+4 digits From blah|blah|foo|bar||blah|945523-232|USA blah|blah|foo|bar||blah|foo|94555-2323|USA To... (8 Replies)
Discussion started by: gubbu
8 Replies

5. UNIX for Dummies Questions & Answers

How to remove duplicated based on longest row & largest value in a column

Hii i have a file with data as shown below. Here i need to remove duplicates of the rows in such a way that it just checks for 2,3,4,5 column for duplicates.When deleting duplicates,retain largest row i.e with many columns with values should be selected.Then it must remove duplicates such that by... (11 Replies)
Discussion started by: reva
11 Replies

6. Shell Programming and Scripting

Help with remove duplicated content

Input file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hcmv-UL70-3p hsa-4486 hcms-US25 hsa-360-5 hcms-US25 hsa-4 hcms-US25 hsa-458 hcms-US25 hsa-44812 . . Desired Output file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hsa-4486... (3 Replies)
Discussion started by: perl_beginner
3 Replies

7. UNIX for Dummies Questions & Answers

Sort csv file by duplicated column value

hello, I have a large file (about 1gb) that is in a file similar to the following: I want to make it so that I can put all the duplicates where column 3 (delimited by the commas) are shown on top. Meaning all people with the same age are listed at the top. The command I used was ... (3 Replies)
Discussion started by: jl487
3 Replies

8. Shell Programming and Scripting

How to remove duplicated lines?

Hi, if i have a file like this: Query=1 a a b c c c d Query=2 b b b c c e . . . (7 Replies)
Discussion started by: the_simpsons
7 Replies

9. Shell Programming and Scripting

How to delete 'duplicated' column values and make a delimited file too?

Hi, I have the following output from an Oracle SQL statement and I want to remove duplicated column values. I know it is possible using Oracle analytical/statistical functions but unfortunately I don't know how to use any of those. So now, I've gone to PLAN B using awk/sed maybe or any... (5 Replies)
Discussion started by: newbie_01
5 Replies

10. Shell Programming and Scripting

awk script to append suffix to column when column has duplicated values

Please help me to get required output for both scenario 1 and scenario 2 and need separate code for both scenario 1 and scenario 2 Scenario 1 i need to do below changes only when column1 is CR and column3 has duplicates rows/values. This inputfile can contain 100 of this duplicated rows of... (1 Reply)
Discussion started by: as7951
1 Replies
IO::Async::PID(3pm)					User Contributed Perl Documentation				       IO::Async::PID(3pm)

NAME
"IO::Async::PID" - event callback on exit of a child process SYNOPSIS
use IO::Async::PID; use POSIX qw( WEXITSTATUS ); use IO::Async::Loop; my $loop = IO::Async::Loop->new; my $kid = $loop->fork( code => sub { print "Child sleeping.. "; sleep 10; print "Child exiting "; return 20; }, ); print "Child process $kid started "; my $pid = IO::Async::PID->new( pid => $kid, on_exit => sub { my ( $self, $exitcode ) = @_; printf "Child process %d exited with status %d ", $self->pid, WEXITSTATUS($exitcode); }, ); $loop->add( $pid ); $loop->run; DESCRIPTION
This subclass of IO::Async::Notifier invokes its callback when a process exits. For most use cases, a IO::Async::Process object provides more control of setting up the process, connecting filehandles to it, sending data to and receiving data from it. EVENTS
The following events are invoked, either using subclass methods or CODE references in parameters: on_exit $exitcode Invoked when the watched process exits. PARAMETERS
The following named parameters may be passed to "new" or "configure": pid => INT The process ID to watch. Must be given before the object has been added to the containing "IO::Async::Loop" object. on_exit => CODE CODE reference for the "on_exit" event. Once the "on_exit" continuation has been invoked, the "IO::Async::PID" object is removed from the containing "IO::Async::Loop" object. METHODS
$process_id = $pid->pid Returns the underlying process ID $pid->kill( $signal ) Sends a signal to the process AUTHOR
Paul Evans <leonerd@leonerd.org.uk> perl v5.14.2 2012-10-24 IO::Async::PID(3pm)
All times are GMT -4. The time now is 05:40 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy