Select distinct values from a flat file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Select distinct values from a flat file
# 8  
Old 05-19-2010
Shorter:
Code:
awk -F "," '!a[$2]++' file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Select distinct sequences from fasta file and list

Hi How can I extract sequences from a fasta file with respect a certain criteria? The beginning of my file (containing in total more than 1000 sequences) looks like this: >H8V34IS02I59VP SDACNDLTIALLQIAREVRVCNPTFSFRWHPQVKDEVMRECFDCIRQGLG YPSMRNDPILIANCMNWHGHPLEEARQWVHQACMSPCPSTKHGFQPFRMA... (6 Replies)
Discussion started by: Marion MPI
6 Replies

2. Shell Programming and Scripting

Finding distinct characters from flat file

Hi....I need one help.... I'm having a files which is having the data as follows... a b c c d d d e f Now I need to find out distinct characters from this file and the output should be as follows - a b c d e f Can you please help me on this? I'm using KSH script. (18 Replies)
Discussion started by: Krishanu Saha
18 Replies

3. Shell Programming and Scripting

Select distinct rows in a file by last column

Hi, I have the following file: LOG:015608::ERR:2310:map_spsrec:Invalid parameter LOG:015608::ERR:2471:map_dgdrec:Invalid parameter LOG:015608::ERR:2487:map_nnmrec:Invalid number LOG:015608::ERR:2310:map_nmrec:Invalid number LOG:015608::ERR:2438:map_nmrec:Invalid number As a delimiter I... (2 Replies)
Discussion started by: apenkov
2 Replies

4. Shell Programming and Scripting

distinct values of all the fields

I am a beginner to scripting, please help me in this regard. How do I create a script that provides a count of distinct values of all the fields in the pipe delimited file ? I have 20 different files with multiple columns in each file. I needed to write a generic script where I give the number... (2 Replies)
Discussion started by: vukkusila
2 Replies

5. UNIX for Dummies Questions & Answers

distinct values of all the fields

I am a beginner to scripting, please help me in this regard. How do I create a script that provides a count of distinct values of all the fields in the pipe delimited file ? I have 20 different files with multiple columns in each file. I needed to write a generic script where I give the number... (1 Reply)
Discussion started by: vukkusila
1 Replies

6. Shell Programming and Scripting

grep distinct values

this is a little more complex than that. I have a text file and I need to find all the distinct words that appear in a line after the word TABLESPACE when I grep for just the word tablespace, I get: how do i parse this a little better so i have a smaller file to read? This is just an... (4 Replies)
Discussion started by: guessingo
4 Replies

7. UNIX for Dummies Questions & Answers

Select Distinct on multiple fields

How do I create a script that provides a count of distinct values of a particular field in a file utilizing commonly available UNIX commands (sh or awk)? Field1|Field2|Field3|Field4 AAA|BBB|CCC|DDD 111|222|333|777 AAA|EEE|ZZZ|EEE 111|555|333|444 AAA|EEE|CCC|DDD 111|222|555|444 For... (2 Replies)
Discussion started by: Refresher
2 Replies

8. Shell Programming and Scripting

Getting Distinct values from second field in a file....

Hi I have a pipe delimited file. I am trying to grab the DISTINCT value from the second field. The file is something like: 1233|apple|ron 1234|apple|elephant 1235|egg|man the output I am trying to get from second field is apple,egg (apple coming only once) Thanks simi (4 Replies)
Discussion started by: simi28
4 Replies

9. Shell Programming and Scripting

Loop through only the distinct values in a file

Datafile has the following data seperated by : FIELD1:FIELD2:FIELD3 D1:/opt/9.1.9:Y D2:/opt/10.1.10:Y D3:/opt/9.1.9:Y D4:/opt/8.1.8:Y D5:/opt/8.1.8:Y D6:/opt/9.1.9:Y D7:/opt/9.1.9:Y D8:/opt/10.1.10:Y D9:/opt/9.1.9:Y D10:/opt/10.1.10:Y I want to do some operations only on the distinct... (2 Replies)
Discussion started by: pbekal
2 Replies

10. UNIX for Dummies Questions & Answers

select distinct row from a file

Hi, buddies out there. I have a text file ( only one column ) which I created using vi editor. The file contains duplicate rows and I would like to select distinct rows, how to go on it using unix command: file content = apple apple orange watermelon apple orange Can it be done... (7 Replies)
Discussion started by: merry susana
7 Replies
Login or Register to Ask a Question
XEN-UPDATE-IMAGE(8)					 Perl Programmers Reference Guide				       XEN-UPDATE-IMAGE(8)

NAME
xen-update-image - Update the software installed upon offline Xen images. SYNOPSIS
xen-update-image [options] imageName1 imageName2 .. imageNameN Help Options: --help Show this scripts help information. --manual Read this scripts manual. --version Show the version number and exit. General Options: --dir Specify the directory which contains the image(s). --lvm Specify the LVM volume group which contains the image(s). --evms Specify the EVMS container which contains the image(s). OPTIONS
--dir Specify the directory which contains the image(s). --evms Specify the EVMS container which contains the image(s). --help Show the script help. --lvm Specify the LVM volume group which contains the image(s). --manual Read the manual. --version Show the version number and exit. DESCRIPTION
xen-update-image is a simple script which allows you to update a Xen image of Debian which has been created with xen-create-image. It does this by mounting the image inside a temporary directory then running: apt-get update apt-get upgrade NOTE If the image is already running within Xen this will cause corruption otherwise it will allow you to update your image without booting it. EXAMPLES
The following assumes there are two images which are not currently running. The images are called 'test.my.flat', and 'x11.my.flat'. Updating both images can be accomplished by executing: xen-update-images --dir=/home/xen test.my.flat x11.my.flat AUTHORS
Steve Kemp, http://www.steve.org.uk/ Axel Beckert, http://noone.org/abe/ StA~Xphane Jourdois LICENSE
Copyright (c) 2005-2009 by Steve Kemp, (c) 2010 by The Xen-Tools Development Team. All rights reserved. This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The LICENSE file contains the full text of the license. 4.3.1 2012-06-30 XEN-UPDATE-IMAGE(8)