Sponsored Content
Top Forums UNIX for Dummies Questions & Answers distinct values of all the fields Post 302461959 by fpmurphy on Tuesday 12th of October 2010 09:55:03 PM
Old 10-12-2010
Bumping up posts or double posting is not permitted in these forums.

Please read the rules, which you agreed to when you registered, if you have not already done so.

You may receive an infraction for this. If so, don't worry, just try to follow the rules more carefully. The infraction will expire in the near future

Thank You.

The UNIX and Linux Forums.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Loop through only the distinct values in a file

Datafile has the following data seperated by : FIELD1:FIELD2:FIELD3 D1:/opt/9.1.9:Y D2:/opt/10.1.10:Y D3:/opt/9.1.9:Y D4:/opt/8.1.8:Y D5:/opt/8.1.8:Y D6:/opt/9.1.9:Y D7:/opt/9.1.9:Y D8:/opt/10.1.10:Y D9:/opt/9.1.9:Y D10:/opt/10.1.10:Y I want to do some operations only on the distinct... (2 Replies)
Discussion started by: pbekal
2 Replies

2. Shell Programming and Scripting

Awk to print distinct col values

Hi Guys... I am newbie to awk and would like a solution to probably one of the simple practical questions. I have a test file that goes as: 1,2,3,4,5,6 7,2,3,8,7,6 9,3,5,6,7,3 8,3,1,1,1,1 4,4,2,2,2,2 I would like to know how AWK can get me the distinct values say for eg: on col2... (22 Replies)
Discussion started by: anduzzi
22 Replies

3. Shell Programming and Scripting

Getting Distinct values from second field in a file....

Hi I have a pipe delimited file. I am trying to grab the DISTINCT value from the second field. The file is something like: 1233|apple|ron 1234|apple|elephant 1235|egg|man the output I am trying to get from second field is apple,egg (apple coming only once) Thanks simi (4 Replies)
Discussion started by: simi28
4 Replies

4. UNIX for Dummies Questions & Answers

Select Distinct on multiple fields

How do I create a script that provides a count of distinct values of a particular field in a file utilizing commonly available UNIX commands (sh or awk)? Field1|Field2|Field3|Field4 AAA|BBB|CCC|DDD 111|222|333|777 AAA|EEE|ZZZ|EEE 111|555|333|444 AAA|EEE|CCC|DDD 111|222|555|444 For... (2 Replies)
Discussion started by: Refresher
2 Replies

5. Shell Programming and Scripting

grep distinct values

this is a little more complex than that. I have a text file and I need to find all the distinct words that appear in a line after the word TABLESPACE when I grep for just the word tablespace, I get: how do i parse this a little better so i have a smaller file to read? This is just an... (4 Replies)
Discussion started by: guessingo
4 Replies

6. Shell Programming and Scripting

To count distinct fields in a row

I have . dat file which contains data in a specific format: 0 3 892 921 342 1 3 921 342 543 2 4 817 562 718 765 3 3 819 562 717 761 i need to compare each field in a row with another field of the same column but different row and cont the... (8 Replies)
Discussion started by: Abhik
8 Replies

7. Shell Programming and Scripting

distinct values of all the fields

I am a beginner to scripting, please help me in this regard. How do I create a script that provides a count of distinct values of all the fields in the pipe delimited file ? I have 20 different files with multiple columns in each file. I needed to write a generic script where I give the number... (2 Replies)
Discussion started by: vukkusila
2 Replies

8. Shell Programming and Scripting

average of distinct values with awk

Hi guys, I am not an expert in shell and I need help with awk command. I have a file with values like 200 1 1 200 7 2 200 6 3 200 5 4 300 3 1 300 7 2 300 6 3 300 4 4 I need resulting file with averages of... (3 Replies)
Discussion started by: saif
3 Replies

9. Shell Programming and Scripting

Find distinct values

Hi, I have two files of the following format file1 chr1:345-456 chr2:123-456 chr2:455-678 chr3:456-789 chr3:444-555 file2 chr1:345-456 chr2:123-456 chr3:456-789 output (2 Replies)
Discussion started by: jacobs.smith
2 Replies

10. Shell Programming and Scripting

Need distinct values from command in a script

Hello, I am using below command srvctl config service -d cmdbut cmdbut_01 (P):/devoragridcn_01/app/oracle> srvctl config service -d cmdbut Service name: boms10.world Service is enabled Server pool: cmdbut_boms10.world Cardinality: 1 Disconnect: false Service role: PRIMARY Management... (7 Replies)
Discussion started by: Vishal_dba
7 Replies
TABMERGE(1p)						User Contributed Perl Documentation					      TABMERGE(1p)

NAME
tabmerge - unify delimited files on common fields SYNOPSIS
tabmerge [action] [options] file1 file2 [...] Actions: --min Take only fields present in all files [DEFAULT] --max Take all fields present -f|--fields=f1[,f2] Take only the fields mentioned in the comma-separated list Options: -l|--list List available fields --fs=x Use "x" as the field separator (default is tab " ") --rs=x Use "x" as the record separator (default is newline " ") -s|--sort=f1[,f2] Sort data ASCII-betically on field(s) --stdout Print data in original delimited format (i.e., not in a table format) --help Show brief help and quit --man Show full documentation DESCRIPTION
This program merges the fields -- not the rows -- of delimited text files. That is, if several files are almost but not quite entirely unlike each other in their structure (in their field names, numbers or orders), this script allows you to easily unify the files into one file with all the same fields. The output can be based on fields as determined by the three "action" flags. For the following examples, consider three files that contain the following fields: +------------+---------------------------------+ | File | Fields | +------------+---------------------------------+ | merge1.tab | name, type, position | | merge2.tab | name, type, position, lod_score | | merge3.tab | name, position | +------------+---------------------------------+ To list all available fields in the files and the number of times they are present: $ tabmerge --list merge* +-----------+-------------------+ | Field | No. Times Present | +-----------+-------------------+ | lod_score | 1 | | name | 3 | | position | 3 | | type | 2 | +-----------+-------------------+ To merge the files on the minimum overlapping fields: $ tabmerge merge* +----------+----------+ | name | position | +----------+----------+ | RM104 | 2.30 | | RM105 | 4.5 | | TX5509 | 10.4 | | UU189 | 19.0 | | Xpsm122 | 3.3 | | Xpsr9556 | 4.5 | | DRTL | 2.30 | | ALTX | 4.5 | | DWRF | 10.4 | +----------+----------+ To merge the files and include all the fields: $ tabmerge --max merge* +-----------+----------+----------+--------+ | lod_score | name | position | type | +-----------+----------+----------+--------+ | | RM104 | 2.30 | RFLP | | | RM105 | 4.5 | RFLP | | | TX5509 | 10.4 | AFLP | | 2.4 | UU189 | 19.0 | SSR | | 1.2 | Xpsm122 | 3.3 | Marker | | 1.2 | Xpsr9556 | 4.5 | Marker | | | DRTL | 2.30 | | | | ALTX | 4.5 | | | | DWRF | 10.4 | | +-----------+----------+----------+--------+ To merge and extract just the "name" and "type" fields: $ tabmerge -f name,type merge* +----------+--------+ | name | type | +----------+--------+ | RM104 | RFLP | | RM105 | RFLP | | TX5509 | AFLP | | UU189 | SSR | | Xpsm122 | Marker | | Xpsr9556 | Marker | | DRTL | | | ALTX | | | DWRF | | +----------+--------+ To merge the files on just the "name" and "lod_score" fields and sort on the name: $ tabmerge -f name,lod_score -s name merge* +----------+-----------+ | name | lod_score | +----------+-----------+ | ALTX | | | DRTL | | | DWRF | | | RM104 | | | RM105 | | | TX5509 | | | UU189 | 2.4 | | Xpsm122 | 1.2 | | Xpsr9556 | 1.2 | +----------+-----------+ To do the same but mimic the original tab-delimited input: $ tabmerge -f name,lod_score -s name --stdout merge* name lod_score ALTX DRTL DWRF RM104 RM105 TX5509 UU189 2.4 Xpsm122 1.2 Xpsr9556 1.2 Why would you want to do this? Suppose you have several delimited text files with nearly the same structure and want to create just one file from them, but the fields may be in a different order in each file and/or some files may contain more or fewer fields than others. (As far-fetched as it may seem, it happens to the author more than he'd like.) SEE ALSO
o Text::RecordParser o Text::TabularDisplay AUTHOR
Ken Youens-Clark <kclark@cpan.org>. LICENSE AND COPYRIGHT
Copyright (C) 2006-10 Ken Youens-Clark. All rights reserved. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. perl v5.10.1 2010-07-26 TABMERGE(1p)
All times are GMT -4. The time now is 02:21 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy