Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Column comparison between two files: moved from another post

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Column comparison between two files: moved from another post
# 1  
Old 10-02-2010
Column comparison between two files: moved from another post
I have two files as follows:
Code:
Pval.txt
ID    Pvalue
91500    0.004
91700    0.007
91800    0.12
91900    0.05
92000    0.99

Freq.txt
ID    Frequency
90000    56
91500    29
91600    78
91700    60
91800    77
91900    66
92000    70
92100    50
92200    60

I need to compare 1st column of Pval.txt to the first column of freq.txt. When it matches, I want to write the corresponding frequency as a new third column in Pval.txt.

danmero's shell script
Code:
while read a b;do while read c d;do if [[ $a = $c ]]; then printf "%s\t%s\t%s\n" $a $b $d;fi;done < Freq.txt ;done < Pval.txt

and awk one liner:
Code:
awk 'NR==FNR{a[$1]=$2;next}a[$1]{print $0"\t"a[$1]}' Freq.txt Pval.txt

gives an output with the pvalue and frequency columns merging as shown

Code:
ID      Frequency
91500   29004
91700   60007
91800   7712
91900   6605
92000   7099

However my output should be as follows:
Code:
ID    Pvalue    Frequency
91500    0.004    29
91700    0.007    60
91800    0.12    77
91900    0.05    66
92000    0.99    70

any help appreciated

thanks
CSN
Last edited by cs_novice; 10-02-2010 at 03:31 PM..
# 2  
Old 10-02-2010
Code:
join Pval.txt Freq.txt

# 3  
Old 10-02-2010
Quote:
Originally Posted by bartus11
Code:
join Pval.txt Freq.txt

Hi Bartus
I do not want to join the files. I edited my post to make my question clearer.

thanks
Siva
# 4  
Old 10-02-2010
Quote:
Originally Posted by cs_novice
Hi Bartus
I do not want to join the files. I edited my post to make my question clearer.

thanks
Siva
Did you try this command?
# 5  
Old 10-02-2010
Quote:
Originally Posted by bartus11
Did you try this command?
Hi Bartus
Yes I tried the command you suggested
Code:
join Pval.txt Freq.txt

and I get:

Code:
Frequency
 2900 0.004
 6000 0.007
 7700 0.12
 6600 0.05
 7000 0.99

# 6  
Old 10-02-2010
MySQL
Code:
# ./justdoit
ID    Pvalue    Frequency
91500    0.004    29
91700    0.007    60
91800    0.12    77
91900    0.05    66
92000    0.99    70

Code:
 ## justdoit ##
#!/bin/bash
rm -f newfileXX
(sed -n '1p' Pval.txt ; sed -n '1s/.*  *\(.*\)/\1/p' Freq.txt) | sed 'N;s/\n/    /' >> newfileXX
for i in $(sed 's/\(.*\)  *.*/\1/;1d' Pval.txt)
 do
  ix=( $(sed 's/\(.*\)  *.*/\1/;1d' Freq.txt ) )
  x=1
  for ax in ${ix[@]}
   do
   ((x++))
    if [ $i -eq $ax ] ; then
     ( sed -n "/$i/p" Pval.txt ; sed -n "$x s/.*  *\(.*\)/\1/p" Freq.txt ) | sed 'N;s/\n/    /' >> newfileXX
    fi
   done
 done
more newfileXX

@ygemici
sed-lovers
This User Gave Thanks to ygemici For This Post:
cs_novice
# 7  
Old 10-02-2010
Quote:
Originally Posted by cs_novice
However my output should be as follows:
Code:
ID    Pvalue    Frequency
91500    0.004    29
91700    0.007    60
91800    0.12    77
91900    0.05    66
92000    0.99    70

any help appreciated
Works for meSmilie
Code:
# awk 'NR==FNR{a[$1]=$2;next}a[$1]{print $0"\t"a[$1]}' Freq.txt Pval.txt
ID    Pvalue    Frequency
91500    0.004  29
91700    0.007  60
91800    0.12   77
91900    0.05   66
92000    0.99   70
# while read a b;do while read c d;do if [[ $a = $c ]]; then printf "%s\t%s\t%s\n" $a $b $d;fi;done < Freq.txt ;done < Pval.txt
ID      Pvalue  Frequency
91500   0.004   29
91700   0.007   60
91800   0.12    77
91900   0.05    66
92000   0.99    70

Please provide more info about your system.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to count the number of files moved?

I'm writing a script for searching substring in file content and then moving found files. So far I've wrote script shown below grep -lir 'stringtofind' $1 | xargs mv -t $2 How can i count number of files moved? (4 Replies)
Discussion started by: Kadikis
4 Replies

2. Shell Programming and Scripting

Need help in column comparison & adding extra line to files

Hi, I wanted to check whether the x,y,z coordinates of two files are equal or not. At times, when one file is converted to another suitable file extension , there are some chances that the data mismatch would happen during the conversion. In order to avoid the data misfit, i would like to... (6 Replies)
Discussion started by: b@l@ji
6 Replies

3. Linux

Possible Cause of Files Not Being Moved?

Hi ULF, Good day! I'm working on a LINUX Suse server and I have an entry in CRON which looks like this below: 0 5 * * * /usr/bin/find /opt/nsfw/var/partition-all/ -name "RCV_SASN*" -exec mv '{}' /opt/nsfw/var/rcv-archive/ \; This tool runs everyday at 5am and it will just move the files... (7 Replies)
Discussion started by: rymnd_12345
7 Replies

4. Shell Programming and Scripting

column value comparison in a file

Hi, Can any one help with my below requirement. i need to compare each line by line and in each line i have to compare some columns values with previous line column values in perl script. Can any one help me........! its very urgent. Thanks (3 Replies)
Discussion started by: jam_prasanna
3 Replies

5. Shell Programming and Scripting

List moved files in text file

Hi. I am actually doing all of this on OSX, but using unix apps and script. I have built my own transparent rsync/open directory/mobility/etc set of scripts for the firm I work at, and it is all almost complete except for ONE THING. I have the classic problem with rsync where if a user... (0 Replies)
Discussion started by: Ashtefere
0 Replies

6. UNIX for Advanced & Expert Users

How to know the user who moved the files to other dir

Hi, I want to know the user ID who moved a file from one directory to another Directory. Example: File1 created by user A is present in dirA then some one has moved it to dirB using "mv" command I want to know the user ID who moved the file to dirB. As far as i know "ls -lrt" command... (1 Reply)
Discussion started by: srilaxmi
1 Replies

7. Solaris

files updated in last 10 hours should be moved

Hi, I would like to move all files that are updated in last 10 hrs. to some temporary folder. Please help. (3 Replies)
Discussion started by: sanjay1979
3 Replies

8. UNIX for Dummies Questions & Answers

Showing Moved Files

Hi everyone, In a directory I have files with various extensions. I would like to move all the files ending in .L2 into a directory: ~/test. But I would also like to show which files are being moved. Of course I could type: $ ls *.L2 $ mv *.L2 ~/test Is there a way I can combine these two... (5 Replies)
Discussion started by: msb65
5 Replies

9. UNIX for Dummies Questions & Answers

rsync, which files where moved?

Hello, I am using rsync to make sure that my folder "local" mirrors the remote directory "remote". When a file is copied from "remote" to "local", I need to apply a bash script to it. What would be a neat way to do that? Thanks ps: is there a way to edit the title of the thread (I am a bit... (5 Replies)
Discussion started by: JCR
5 Replies

10. Shell Programming and Scripting

Getting a list of filenames of moved files

I'm moving a list of files of some extension and I wish to output the moved filenames into a text file, I tried using the command below, but after all the files are moved, I got a blank file. find /abc/temp -type f -mtime +365 \( -name "*.bak" -o -name "*.log" \) -exec mv -f {} /junk \; >>... (3 Replies)
Discussion started by: chengwei
3 Replies
Login or Register to Ask a Question