Hi Ikki!
Thanks for your reply! I use awk bacause I'm more familiar with its sintax, i've just used sed a couple of time!
My files contain datas from genetic chips, and each file belongs to a person.
Each file has ~360000 lines.
You're right, I have more than one file shorter than the other..at the moment there are 95 shorter files. But they could increase, and yes, in shorter files it's possible that a line starting with cnvi0000004 is followed by a line starting with cnvi0000006 or cnvi0000008...it depends on how many records are missing for this person, but all input file are sorted by the first column.
As I said I've written an R-script that works, but it is extremely slow. In this script I compare a list of "complete" names with another and see if there are differences. Once i found elements that aren't in the short list, i add them in this list in order to have elements with same length. In this way I can merge all column and insert tabs instead of missing datas. I post the R-code:
Now I'm trying to port this in a shell script to have a faster response. My purpose was to avoid files preprocessing if i can, because of the large amount of data stored in each file, if possible.
I tried your command also, and it is running, the only problem is to add manually 1664 columns for the cut command, but I think I can work on It!
Hope I have been clear enough, and greatly appreciate your help!
Hello,
My apologies if this has been posted elsewhere, I have had a look at several threads but I am still confused how to use these functions. I have two files, each with 5 columns:
File A: (tab-delimited)
PDB CHAIN Start End Fragment
1avq A 171 176 awyfan
1avq A 172 177 wyfany
1c7k A 2 7... (3 Replies)
I have n files (for ex:64 files) with one similar column. Is it possible to combine them all based on that column ?
file1
ax100 20 30 40
ax200 22 33 44
file2
ax100 10 20 40
ax200 12 13 44
file2
ax100 0 0 4
ax200 2 3 4 (9 Replies)
Hi Friends,
I have a file1 with 3400 records that are tab separated and I have a file2 with 6220 records. I want to merge both these files. I tried using join file1 and file2 after sorting. But, the records should be (3400*6220 = 21148000). Instead, I get only around 11133567. Is there anything... (13 Replies)
Hi,
I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column).
I want to write a script to join the files by the first common column so that in the... (5 Replies)
hi guys,
i need help
I need to join file2 to file1 when column 3 in my file1 and column 1 in my file2 in the same string
file1
AA|RR|ESKIM
RE|DD|RED
WE|WW|SUPSS
file2
ESKIM|ES
SUPSS|SS
Output
AA|RR|ESKIM|ES
RE|DD|RED|
WE|WW|SUPSS|SS (3 Replies)
Hello,
I have a file with 2 columns ( tableName , ColumnName) delimited by a Pipe like below . File is sorted by ColumnName.
Table1|Column1
Table2|Column1
Table5|Column1
Table3|Column2
Table2|Column2
Table4|Column3
Table2|Column3
Table2|Column4
Table5|Column4
Table2|Column5
From... (6 Replies)
Hello,
This post is already here but want to do this with another way
Merge multiples files with multiples duplicates keys by filling "NULL" the void columns for anothers joinning files
file1.csv:
1|abc
1|def
2|ghi
2|jkl
3|mno
3|pqr
file2.csv:
1|123|jojo
1|NULL|bibi... (2 Replies)
Discussion started by: yjacknewton
2 Replies
LEARN ABOUT DEBIAN
funmerge
funmerge(1) SAORD Documentation funmerge(1)NAME
funmerge - merge one or more Funtools table files
SYNOPSIS
funmerge [-w|-x] -f [colname] <iname1> <iname2> ... <oname>
OPTIONS -f # output a column specifying file from which this event came
-w # adjust position values using WCS info
-x # adjust position values using WCS info and save old values
DESCRIPTION
funmerge merges FITS data from one or more FITS Binary Table files or raw event files.
The first argument to the program specifies the first input FITS table or raw event file. If "stdin" is specified, data are read from the
standard input. Use Funtools Bracket Notation to specify FITS extensions and row filters. Subsequent arguments specify additional event
files and tables to merge. (NB: Stdin cannot not be used for any of these additional input file arguments.) The last argument is the out-
put FITS file. The columns in each input table must be identical.
If an input file begins with the '@' character, it is processed as an include file, i.e., as a text file containing event file names (as
well as blank lines and/or comment lines starting with the '#' sign). If standard input is specified as an include file ('@stdin'), then
file names are read from the standard input until EOF (^D). Event files and include files can be mixed on a command line.
Rows from each table are written sequentially to the output file. If the switch -f [colname] is specified on the command line, an addi-
tional column is added to each row containing the number of the file from which that row was taken (starting from one). In this case, the
corresponding file names are stored in the header parameters having the prefix FUNFIL, i.e., FUNFIL01, FUNFIL02, etc.
Using the -w switch (or -x switch as described below), funmerge also can adjust the position column values using the WCS information in
each file. (By position columns, we mean the columns that the table is binned on, i.e., those columns defined by the bincols= switch, or
(X,Y) by default.) To perform WCS alignment, the WCS of the first file is taken as the base WCS. Each position in subsequent files is
adjusted by first converting it to the sky coordinate in its own WCS coordinate system, then by converting this sky position to the sky
position of the base WCS, and finally converting back to a pixel position in the base system. Note that in order to perform WCS alignment,
the appropriate WCS and TLMIN/TLMAX keywords must already exist in each FITS file.
When performing WCS alignment, you can save the original positions in the output file by using the -x (for "xtra") switch instead of the
-w switch (i.e., using this switch also implies using -w) The old positions are saved in columns having the same name as the original
positional columns, with the added prefix "OLD_".
Examples:
Merge two tables, and preserve the originating file number for each row in the column called "FILE" (along with the corresponding file name
in the header):
[sh] funmerge -f "FILE" test.ev test2.ev merge.ev
Merge two tables with WCS alignment, saving the old position values in 2 additional columns:
[sh] funmerge -x test.ev test2.ev merge.ev
This program only works on raw event files and binary tables. We have not yet implemented image and array merging.
SEE ALSO
See funtools(7) for a list of Funtools help pages
version 1.4.2 January 2, 2008 funmerge(1)