COMPARING COLUMNS IN A TEXT FILE


 
Thread Tools Search this Thread
Special Forums UNIX Desktop Questions & Answers COMPARING COLUMNS IN A TEXT FILE
# 1  
Old 03-07-2011
COMPARING COLUMNS IN A TEXT FILE

Hi, Good day.

I currently have this data called database.txt and I would like to check if there are no similar values (all unique) on an entire row considering the whole column data is unique. the data is as follows

cL1 cL2 cL3 cL4
a12 c13 b13 c15
b11 a15 c19 b11
c15 c17 b13 f14

with the first line only as basis for column numbers

I am assuming first the logic in comparing all column to every single column in the data.
I need to do is for example I will compare cL1 (column 1) and cL2 (column 2) for similar row values.
Selecting the two rows I get
a12 c13
b11 a15
c15 c17
Column 1 & 2 are unique to each other
Selecting 2nd and 3rd:
c13 b13
a15 c19
c17 b13
Column 2 & 3 are NOT unique to each other. Reason is that on 3rd column, there are two "b13".
Selecting 3rd & 4th:
b13 c15
c19 b11
b13 f14
Column 3 & 4 are NOT unique to each other. Reason is that on 3rd column, there are two "b13".
Selecting 1st and 3rd... Check again if there are no repetition by column and row
Select 1st and 4th... Check again
Select 2nd and 4th... Check again

if all columns are compared to each other and just one comparison is not unique, then the file is not unique already.

Can you suggest other ways to confirm if my theory is right? And if you have similar script, i would be thankful if you can share it because I'm new here and I have only slept 2 hrs last night banging my head for solutions...
# 2  
Old 03-08-2011
Column, row, sheesh!

Maybe it s a problem because you are not being relational, you are really examining 4 derived one column tables from this data. Sort each column up and compare them, using 'comm' or 'join'. The comm command expects unique data, so run it through 'sort -u'. Check a column against itself with 'sort|uniq -d'. Once you have the problem values and their column, you can use them to find the original rows.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Comparing columns in 2 text files

Hi i have 2 files file1.txt XX,ZZ,XC,EE,RR,BB XC,CF,FG,RG,GH,GH File2.txt DF,GH,MH,FR,FG,GH,NOTOK XX,ZZ,XC,EE,RR,BB,OK result XX,ZZ,XC,EE,RR,BB OK look for column1 , XX and if it matches in File2.txt , retrieve the 7 th field from File2 and print in 3 rd file , ... (9 Replies)
Discussion started by: Shyam_84
9 Replies

2. Shell Programming and Scripting

How to concatenate 2-columns by 2 -columns for a text file?

Hello, I want to concatenate 2-columns by 2-columns separated by colon. How can I do so? For example, I have a text file containing 6 columns separated by tab. I want to concatenate column 1 and 2; column 3 and 4; column 5 and 6, respectively, and put a colon in between. input file: 1 0 0 1... (10 Replies)
Discussion started by: huiyee1
10 Replies

3. Shell Programming and Scripting

Comparing Select Columns from two CSV files in UNIX and create a third file based on comparision

Hi , I want to compare first 3 columns of File A and File B and create a new file File C which will have all rows from File B and will include rows that are present in File A and not in File B based on First 3 column comparison. Thanks in advance for your help. File A A,B,C,45,46... (2 Replies)
Discussion started by: ady_koolz
2 Replies

4. Shell Programming and Scripting

Comparing Columns and writing a new file

I have a table with one column File1.txt 1 2 3 4 5 6 7 8 9 10 Another table with two columns; This has got a subset of entries from File 1 but not unique because they have differing values in col 2. File2.txt 1 a 2 d 2 f 6 r 6 e (3 Replies)
Discussion started by: cs_novice
3 Replies

5. Shell Programming and Scripting

Comparing columns in a file

I have two files. One a small one and another one is big. The smaller one look like this: Filename: 1.tmp 3453 0 326543 1 2321 0 3212 1 The big file looks like this: Filename 1.res 0.3232 2321 9.2922 123 0.983 3212 8.373 326543 0.9 3453 1.098 3432 I want to extract those lines... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

6. UNIX for Dummies Questions & Answers

Removing columns from a text file that do not have any values in second and third columns

I have a text file that has three columns. But at the end of the text file, there are trailing lines that have missing second and third columns: 4 0.04972604 KLHL28 4 0.0497332 CSTB 4 0.04979822 AIF1 4 0.04983331 DECR2 4 0.04990344 KATNB1 4 4 4 4 How can I remove the trailing... (3 Replies)
Discussion started by: evelibertine
3 Replies

7. UNIX for Dummies Questions & Answers

Comparing the 2nd column in two different files and printing corresponding 9th columns in new file

Dear Gurus, I am very new to UNIX. I appreciate your help to manage my files. I have 16 files with equal number of columns in it. Each file has 9 columns separated by space. I need to compare the values in the second column of first file and obtain the corresponding value in the 9th column... (12 Replies)
Discussion started by: Unilearn
12 Replies

8. UNIX for Dummies Questions & Answers

How to convert text to columns in tab delimited text file

Hello Gurus, I have a text file containing nearly 12,000 tab delimited characters with 4000 rows. If the file size is small, excel can convert the text into coloumns. However, the file that I have is very big. Can some body help me in solving this problem? The input file example, ... (6 Replies)
Discussion started by: Unilearn
6 Replies

9. Shell Programming and Scripting

Comparing Columns and printing the difference from a particular file

Gurus, I have one file which is having multiple columns and also this file is not always contain the exact columns; sometimes it contains 5 columns or 12 columns. Now, I need to find the difference from that particular file. Here is the sample file: param1 | 10 | 20 | 30 | param2 | 10 |... (6 Replies)
Discussion started by: buzzusa
6 Replies

10. Shell Programming and Scripting

Help with comparing columns from a csv file

Hi there, I have an csv file. I want to compare the 16th and 18th columns. They contain alpha numeric characters. Some are same and some are different. We have to pick the ones which are different. But with certain rules. 16th col. 18th col. ---------- ... (1 Reply)
Discussion started by: sickboy
1 Replies
Login or Register to Ask a Question