Sponsored Content
Full Discussion: Match columns several files
Top Forums Shell Programming and Scripting Match columns several files Post 302791333 by @man on Monday 8th of April 2013 11:06:37 AM
Old 04-08-2013
Thanks Pamu for your time.

here are two problems! :|
1. I have several files. instead of writing the names manually can I use something like *.txt ?

2. I am more dumb than what I thought! :| :| actually the columns that I wanna check similarity are $4 from all files and ref file. and in the output should comes the $3 of ref file to $1 of output and $13 of each file as the values in the right positions.

I tried to modify it myself but it goes forever and then crashes! :|

could you please give me some updates?

Thanks A LOT!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match strings in two files and compare columns of both

Good Morning, I was wondering if anybody could tell me how to achieve the following, preferably with a little commenting for understanding. I have 2 files, each with multiple rows with multiple columns. I need to find each row where the value in column 1 of file 1 matches column 1... (10 Replies)
Discussion started by: GarciasMuffin
10 Replies

2. UNIX for Dummies Questions & Answers

Two files; if cells match then copy over other columns

My current issue is dealing with two space delimited files. The first file has column 1 as the sample ID's, then columns 2 - n as the observations. The second file has column 1 as the sample ID's, column 2 as the mother ID's, column 3 as the father ID's, column 4 as the gender, and column 5... (3 Replies)
Discussion started by: Renyulb28
3 Replies

3. Shell Programming and Scripting

Match files based on either of the two columns awk

Dear Shell experts, I have 2 files with structure: File 1: ID and count head test_GI_count1.txt 1000094 2 10039307 1 10039641 1 10047177 11 10047359 1 1008555 2 10120302 1 10120672 13 10121776 1 10121865 32 And 2nd file: head Protein_gi_GeneID_symbol.txt protein_gi GeneID... (11 Replies)
Discussion started by: smitra
11 Replies

4. Shell Programming and Scripting

Match two columns from two files and print output

Hello, I have two files which are of the following format File 1 which has two columns Protein_ID Substitution NP_997239 T53R NP_060668 V267M NP_058515 P856A NP_001206 T55M NP_006601 D371Y ... (2 Replies)
Discussion started by: nans
2 Replies

5. Shell Programming and Scripting

Match the columns between 2 files

I have two files I want to match ids in the 5th column of the file 1 with the first column of the file 2 and get the description for the matched ids as shown in the output sno nm no nm2 ID 1 cc 574372 yyyi |6810|51234| 2 bb 119721 nmjk |6810|51234|51179| ... (4 Replies)
Discussion started by: raj_k
4 Replies

6. Shell Programming and Scripting

Return first two columns if match found among two files

Hi, I have FileA with one column. File B with 15 columns separated by comma delimiter. I need to compare the FILEA value with all 15 columns of FILEB... if matches, need to return the 1st, 2nd column values of FILEB. How to achieve this through shell script? Thanks in advance. (5 Replies)
Discussion started by: vamsikrishna928
5 Replies

7. Shell Programming and Scripting

Match first two columns and average third from multiple files

I have the following format of input from multiple files File 1 24.01 -81.01 1.0 24.02 -81.02 5.0 24.03 -81.03 0.0 File 2 24.01 -81.01 2.0 24.02 -81.02 -5.0 24.03 -81.03 10.0 I need to scan through the files and when the first 2 columns match I... (18 Replies)
Discussion started by: ncwxpanther
18 Replies

8. UNIX for Dummies Questions & Answers

Match the columns between two files and output

Hi Help, I have two files namely a.txt and b.txt a.txt looks like a.txt 1 2 2 1 3 3 2 4 4 4 5 6 6 7 7 b.txt looks like, b.txt 1 2 1 1 3 2 2 4 3 3 4 4 4 5 5 (2 Replies)
Discussion started by: Indra2011
2 Replies

9. Shell Programming and Scripting

Comparing two columns in two files and printing a third based on a match

Hello all, First post here. I did not notice a previous post to help me down the right path. I am looking to compare a column in a CSV file against another file (which is not a column match one for one) but more or less when a match is made, I would like to append a third column that contains a... (17 Replies)
Discussion started by: dis0wned
17 Replies

10. UNIX for Beginners Questions & Answers

Data match 2 files based on first 2 columns matching only and join if match

Hi, i have 2 files , the data i need to match is in masterfile and i need to pull out column 3 from master if column 1 and 2 match and output entire row to new file I have tried with join and awk and i keep getting blank outputs or same file is there an easier way than what i am... (4 Replies)
Discussion started by: axis88
4 Replies
NCBO(1) 						      General Commands Manual							   NCBO(1)

NAME
ncbo - netCDF Binary Operator SYNTAX
ncbo [-3] [-4] [-6] [-A] [-C] [-c] [-D dbg] [-d dim,[ min][,[ max]]] [-F] [-h] [-L dfl_lvl] [-l path] [-O] [-p path] [-R] [-r] [-t thr_nbr] [-v var[,...]] [-X box] [-x] file_1 file_2 file_3 DESCRIPTION
ncbo subtracts variables in file_2 from the corresponding variables (those with the same name) in file_1 and stores the results in file_3. Variables in file_2 are broadcast to conform to the corresponding variable in file_1 if necessary. Broadcasting a variable means creating data in non-existing dimensions from the data in existing dimensions. For example, a two dimensional variable in file_2 can be subtracted from a four, three, or two (but not one or zero) dimensional variable (of the same name) in file_1. This functionality allows the user to compute anomalies from the mean. Note that variables in file_1 are not broadcast to conform to the dimensions in file_2. Thus, ncbo, the number of dimensions, or rank, of any processed variable in file_1 must be greater than or equal to the rank of the same variable in file_2. Furthermore, the size of all dimensions common to both file_1 and file_2 must be equal. When computing anomalies from the mean it is often the case that file_2 was created by applying an averaging operator to a file with the same dimensions as file_1, if not file_1 itself. In these cases, creating file_2 with ncra rather than ncwa will cause the ncbo operation to fail. For concreteness say the record dimension in file_1 is time. If file_2 were created by averaging file_1 over the time dimension with the ncra operator rather than with the ncwa operator, then file_2 will have a time dimension of size 1 rather than having no time dimension at all In this case the input files to ncbo, file_1 and file_2, will have unequally sized time dimensions which causes ncbo to fail. To prevent this from occuring, use ncwa to remove the time dimension from file_2. An example is given below. ncbo will never difference coordinate variables or variables of type NC_CHAR or NC_BYTE. This ensures that coordinates like (e.g., lati- tude and longitude) are physically meaningful in the output file, file_3. This behavior is hardcoded. ncbo applies special rules to some NCAR CSM fields (e.g., ORO). See NCAR CSM Conventions for a complete description. Finally, we note that ncflint (ncflint netCDF File Interpolator) can be also perform file subtraction (as well as addition, multiplication and interpolation). EXAMPLES
Say files 85_0112.nc and 86_0112.nc each contain 12 months of data. Compute the change in the monthly averages from 1985 to 1986: ncbo 86_0112.nc 85_0112.nc 86m85_0112.nc The following examples demonstrate the broadcasting feature of ncbo. Say we wish to compute the monthly anomalies of T from the yearly average of T for the year 1985. First we create the 1985 average from the monthly data, which is stored with the record dimension time. ncra 85_0112.nc 85.nc ncwa -O -a time 85.nc 85.nc The second command, ncwa, gets rid of the time dimension of size 1 that ncra left in 85.nc. Now none of the variables in 85.nc has a time dimension. A quicker way to accomplish this is to use ncwa from the beginning: ncwa -a time 85_0112.nc 85.nc We are now ready to use ncbo to compute the anomalies for 1985: ncbo -v T 85_0112.nc 85.nc t_anm_85_0112.nc Each of the 12 records in t_anm_85_0112.nc now contains the monthly deviation of T from the annual mean of T for each gridpoint. Say we wish to compute the monthly gridpoint anomalies from the zonal annual mean. A zonal mean is a quantity that has been averaged over the longitudinal (or x) direction. First we use ncwa to average over longitudinal direction lon, creating xavg_85.nc, the zonal mean of 85.nc. Then we use ncbo to subtract the zonal annual means from the monthly gridpoint data: ncwa -a lon 85.nc xavg_85.nc ncbo 85_0112.nc xavg_85.nc tx_anm_85_0112.nc Assuming 85_0112.nc has dimensions time and lon, this example only works if xavg_85.nc has no time or lon dimension. As a final example, say we have five years of monthly data (i.e., 60 months) stored in 8501_8912.nc and we wish to create a file which con- tains the twelve month seasonal cycle of the average monthly anomaly from the five-year mean of this data. The following method is just one permutation of many which will accomplish the same result. First use ncwa to create the file containing the five-year mean: ncwa -a time 8501_8912.nc 8589.nc Next use ncbo to create a file containing the difference of each month's data from the five-year mean: ncbo 8501_8912.nc 8589.nc t_anm_8501_8912.nc Now use ncks to group the five January anomalies together in one file, and use ncra to create the average anomaly for all five Januarys. These commands are embedded in a shell loop so they are repeated for all twelve months: foreach idx (01 02 03 04 05 06 07 08 09 10 11 12) ncks -F -d time,,,12 t_anm_8501_8912.nc foo. ncra foo. t_anm_8589_.nc end Note that ncra understands the stride argument so the two commands inside the loop may be combined into the single command ncra -F -d time,,,12 t_anm_8501_8912.nc foo. Finally, use ncrcat to concatenate the 12 average monthly anomaly files into one twelve-record file which contains the entire seasonal cycle of the monthly anomalies: ncrcat t_anm_8589_??.nc t_anm_8589_0112.nc AUTHOR
NCO manual pages written by Charlie Zender and Brian Mays. REPORTING BUGS
Report bugs to <http://sf.net/bugs/?group_id=3331>. COPYRIGHT
Copyright (C) 1995-2010 Charlie Zender This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU- LAR PURPOSE. SEE ALSO
The full documentation for NCO is maintained as a Texinfo manual called the NCO User's Guide. Because NCO is mathematical in nature, the documentation includes TeX-intensive portions not viewable on character-based displays. Hence the only complete and authoritative versions of the NCO User's Guide are the PDF (recommended), DVI, and Postscript versions at <http://nco.sf.net/nco.pdf>, <http://nco.sf.net/nco.dvi>, and <http://nco.sf.net/nco.ps>, respectively. HTML and XML versions are available at <http://nco.sf.net/nco.html> and <http://nco.sf.net/nco.xml>, respectively. If the info and NCO programs are properly installed at your site, the command info nco should give you access to the complete manual, except for the TeX-intensive portions. HOMEPAGE
The NCO homepage at <http://nco.sf.net> contains more information. NCBO(1)
All times are GMT -4. The time now is 04:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy