Sponsored Content
Top Forums Shell Programming and Scripting Merge 70 files into one data matrix Post 302258188 by dennis.jacob on Friday 14th of November 2008 02:08:23 AM
Old 11-14-2008
Try this one, but this can be better done completely in awk:


Code:
for each in *
do
sort $each | paste out - >out.tmp
mv out.tmp out
done
grep -vE 'random|uniq' out | awk 'BEGIN{str="unique "; for(i=1;i<=70;i++) str=str"identifier"i" "; print str;} { first=$1; gsub(/[a-z]*/,"",$0); print first $0; }'

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

extract data from a data matrix with filter criteria

Here is what old matrix look like, IDs X1 X2 Y1 Y2 10914061 -0.364613333 -0.362922333 0.001691 -0.450094667 10855062 0.845956333 0.860396667 0.014440333 1.483899333... (7 Replies)
Discussion started by: ssshen
7 Replies

2. Shell Programming and Scripting

two-column data to matrix in AWK

Howdy, I need to convert an association data matrix, currently in a two-column format, into a matrix with numbers indicating the number of associations. I've been looking around for AWK code in the list, but could not find anything. Here's an example of what I want to perform: original... (10 Replies)
Discussion started by: sramirez
10 Replies

3. Ubuntu

How to convert full data matrix to linearised left data matrix?

Hi all, Is there a way to convert full data matrix to linearised left data matrix? e.g full data matrix Bh1 Bh2 Bh3 Bh4 Bh5 Bh6 Bh7 Bh1 0 0.241058 0.236129 0.244397 0.237479 0.240767 0.245245 Bh2 0.241058 0 0.240594 0.241931 0.241975 ... (8 Replies)
Discussion started by: evoll
8 Replies

4. Shell Programming and Scripting

Reformatting data in matrix form

Hi, Some assistance with respect to the following problem will be very helpful. I want to reformat my dataset in the following manner for subsequent analysis. I have first column values (which repeat for each value of 2nd column) which are names, the second column specifies position ad the... (1 Reply)
Discussion started by: newbie83
1 Replies

5. Shell Programming and Scripting

Merge multiple tables into big matrix

Hi all, I have a complex (beyond my biological expertise) problem at hand. I need to merge multiple files into 1 big matrix. Please help me with some code. Inp1 Ang_0 chr1 98 T A Ang_0 chr1 352 G A Ang_0 chr1 425 C T Ang_0 chr2 ... (1 Reply)
Discussion started by: newbie83
1 Replies

6. Shell Programming and Scripting

AWK to match and merge data from 2 files into 1.

Hello, hopefully this is an easy on for the AWK guru's out there. I'm having some trouble figuring out how to match+merge data in 2 files into 1 single report. I've got my 2 files filtered and delimited, just need to MATCH $3 in file1 to $1 in file2, then put $0 from File1 and $2+$3 from File2... (6 Replies)
Discussion started by: right_coaster
6 Replies

7. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

8. Shell Programming and Scripting

How to merge the multiple data files as a single file?

Hi Experts, I have created multiple scripts and send the output to new file, getting this output to my mailbox on daily basis. I would like to send the all outputs to a single file, need to merge all file outputs on a single file. For example, Created script for df -h > df.doc grep... (7 Replies)
Discussion started by: seenuvasan1985
7 Replies

9. Shell Programming and Scripting

Merge files and copy some data using sed or awk

Copy data from other file to paste cat file1: Name: server1.data.com data1 server1 running Name: server3.data.com data3 server3 running cat file2: server1 good server2 bad network not ok server3 good Output: (10 Replies)
Discussion started by: kenshinhimura
10 Replies

10. Shell Programming and Scripting

Merge and Sort tabular data from different text files

I have 42 text files; each containing up to 34 lines with following structure; file1 H-01 23 H-03 5 H-05 9 H-02 14 . . file2 H-01 17 H-02 43 H-04 7 H-05 8 H-03 7 . . file3 (6 Replies)
Discussion started by: Syeda Sumayya
6 Replies
UNBUFFER(1)						      General Commands Manual						       UNBUFFER(1)

NAME
unbuffer - unbuffer output SYNOPSIS
unbuffer program [ args ] INTRODUCTION
unbuffer disables the output buffering that occurs when program output is redirected from non-interactive programs. For example, suppose you are watching the output from a fifo by running it through od and then more. od -c /tmp/fifo | more You will not see anything until a full page of output has been produced. You can disable this automatic buffering as follows: unbuffer od -c /tmp/fifo | more Normally, unbuffer does not read from stdin. This simplifies use of unbuffer in some situations. To use unbuffer in a pipeline, use the -p flag. Example: process1 | unbuffer -p process2 | process3 CAVEATS
unbuffer -p may appear to work incorrectly if a process feeding input to unbuffer exits. Consider: process1 | unbuffer -p process2 | process3 If process1 exits, process2 may not yet have finished. It is impossible for unbuffer to know long to wait for process2 and process2 may not ever finish, for example, if it is a filter. For expediency, unbuffer simply exits when it encounters an EOF from either its input or process2. In order to have a version of unbuffer that worked in all situations, an oracle would be necessary. If you want an application-specific solution, workarounds or hand-coded Expect may be more suitable. For example, the following example shows how to allow grep to finish pro- cessing when the cat before it finishes first. Using cat to feed grep would never require unbuffer in real life. It is merely a place- holder for some imaginary process that may or may not finish. Similarly, the final cat at the end of the pipeline is also a placeholder for another process. $ cat /tmp/abcdef.log | grep abc | cat abcdef xxxabc defxxx $ cat /tmp/abcdef.log | unbuffer grep abc | cat $ (cat /tmp/abcdef.log ; sleep 1) | unbuffer grep abc | cat abcdef xxxabc defxxx $ BUGS
The man page is longer than the program. SEE ALSO
"Exploring Expect: A Tcl-Based Toolkit for Automating Interactive Programs" by Don Libes, O'Reilly and Associates, January 1995. AUTHOR
Don Libes, National Institute of Standards and Technology 1 June 1994 UNBUFFER(1)
All times are GMT -4. The time now is 04:21 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy