Sponsored Content
Top Forums Shell Programming and Scripting awk to filter out lines containing unique values in a specified column Post 302932680 by owwow14 on Friday 23rd of January 2015 10:54:28 AM
Old 01-23-2015
awk to filter out lines containing unique values in a specified column

Hi,

I have multiple files that each contain four columns of strings:

File1:

Code:
Code:
123	abc	gfh	273
456	ddff	jfh	837
789	ghi	u4u	395

File2:

Code:
Code:
123	abc	dd	fu
456	def	457	nd
891	384	djh	783

I want to compare the strings in Column 1 of File 1 with each other file and Print in my output those lines with values in column 1 that appear in both files.


Desired Result:

Code:
Code:
123	abc	gfh	273
456	ddff	jfh	837
123	abc	dd	fu
456	def	457	nd

Can anyone help me?
Thanks!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Concatenating column values with unique id into single row

Hi, I have a table in Db2 with data say id_1 phase1 id_1 phase2 id_1 phase3 id_2 phase1 id_2 phase2 I need to concatenate the values like id_1 phase1,phase2,phase3 id_2 phase1,phase2 I tried recursive query but in vain as the length of string to be concatenated in quite long. ... (17 Replies)
Discussion started by: jsaravana
17 Replies

2. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

3. Shell Programming and Scripting

counting lines containing two column field values with awk

Hello everybody, I'm trying to count the number of consecutive lines in a text file which have two distinctive column field values. These lines may appear in several line blocks within the file, but I only want a single block to be counted. This was my first approach to tackle the problem (I'm... (6 Replies)
Discussion started by: origamisven
6 Replies

4. Shell Programming and Scripting

AWK, Perl or Shell? Unique strings and their maximum values from 3 column data file

I have a file containing data like so: 2012-01-02 GREEN 4 2012-01-02 GREEN 6 2012-01-02 GREEN 7 2012-01-02 BLUE 4 2012-01-02 BLUE 3 2012-01-02 GREEN 4 2012-01-02 RED 4 2012-01-02 RED 8 2012-01-02 GREEN 4 2012-01-02 YELLOW 5 2012-01-02 YELLOW 2 I can't always predict what the... (4 Replies)
Discussion started by: rich@ardz
4 Replies

5. Shell Programming and Scripting

How to compare the values of a column in awk in a same file and consecutive lines..

I would like to compare the values of 2nd column of consecutive lines of same file in such a way so that if the difference between first value and second value is more than 100 it should print complete line else ignore line. Input File ========== PDB 2500 RTDB 123 RTDB-EAGLE 122 VSCCP 2565... (4 Replies)
Discussion started by: manuswami
4 Replies

6. Shell Programming and Scripting

awk Print New Column For Every Two Lines and Match On Multiple Column Values to print another column

Hi, My input files is like this axis1 0 1 10 axis2 0 1 5 axis1 1 2 -4 axis2 2 3 -3 axis1 3 4 5 axis2 3 4 -1 axis1 4 5 -6 axis2 4 5 1 Now, these are my following tasks 1. Print a first column for every two rows that has the same value followed by a string. 2. Match on the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

7. UNIX for Dummies Questions & Answers

Unique values in a row sum the next column in UNIX

Hi would like to ask you guys any advise regarding my problem I have this kind of data file.txt 111111111,20 111111111,50 222222222,70 333333333,40 444444444,10 444444444,20 I need to get this file1.txt 111111111,70 222222222,70 333333333,40 444444444,30 using this code I can... (6 Replies)
Discussion started by: reks
6 Replies

8. UNIX for Beginners Questions & Answers

Find unique values but only in column 1

Hi All, Does anyone have any suggestions/examples of how i could show only lines where the first field is not duplicated. If the first field is listed more than once it shouldnt be shown even if the other columns make it unique. Example file : 876,RIBDA,EC2 876,RIBDH,EX7 877,RIBDF,E28... (4 Replies)
Discussion started by: mutley2202
4 Replies

9. Shell Programming and Scripting

CSV File:Filter duplicate records from column1 & another column having unique record

Hi Experts, I have csv file with 30, 40 columns Pasting just 2 column for problem description. Need to print error if below combination is not present in file check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same. For all such rows, the field... (7 Replies)
Discussion started by: as7951
7 Replies

10. Shell Programming and Scripting

Count number of unique values in each column of array

What is an efficient way of counting the number of unique values in a 400 column by 1000 row array and outputting the counts per column, assuming the unique values in the array are: A, B, C, D In other words the output should look like: Value COL1 COL2 COL3 A 50 51 52... (16 Replies)
Discussion started by: Geneanalyst
16 Replies
FPUTCSV(3)								 1								FPUTCSV(3)

fputcsv - Format line as CSV and write to file pointer

SYNOPSIS
int fputcsv (resource $handle, array $fields, [string $delimiter = ","], [string $enclosure = '"'], [string $escape_char = " DESCRIPTION
fputcsv(3) formats a line (passed as a $fields array) as CSV and write it (terminated by a newline) to the specified file $handle. PARAMETERS
o $handle -The file pointer must be valid, and must point to a file successfully opened by fopen(3) or fsockopen(3) (and not yet closed by fclose(3)). o $fields - An array of values. o $delimiter - The optional $delimiter parameter sets the field delimiter (one character only). o $enclosure - The optional $enclosure parameter sets the field enclosure (one character only). o $escape_char - The optional $escape_char parameter sets the escape character (one character only). RETURN VALUES
Returns the length of the written string or FALSE on failure. CHANGELOG
+--------+---------------------------------------+ |Version | | | | | | | Description | | | | +--------+---------------------------------------+ | 5.5.4 | | | | | | | The $escape_char parameter was added | | | | +--------+---------------------------------------+ EXAMPLES
Example #1 fputcsv(3) example <?php $list = array ( array('aaa', 'bbb', 'ccc', 'dddd'), array('123', '456', '789'), array('"aaa"', '"bbb"') ); $fp = fopen('file.csv', 'w'); foreach ($list as $fields) { fputcsv($fp, $fields); } fclose($fp); ?> The above example will write the following to file.csv: aaa,bbb,ccc,dddd 123,456,789 """aaa""","""bbb""" NOTES
Note If PHP is not properly recognizing the line endings when reading files either on or created by a Macintosh computer, enabling the auto_detect_line_endings run-time configuration option may help resolve the problem. SEE ALSO
fgetcsv(3). PHP Documentation Group FPUTCSV(3)
All times are GMT -4. The time now is 04:54 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy