Matrix to 3 col sorted


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Matrix to 3 col sorted
# 8  
Old 09-27-2017
Your initial approach can be optimized a little
Code:
awk -F"\t" 'NR==1{nl=split($0,lines);next} {for(i=4;i<=nl;i++) print lines[i],$1"__"$2"__"$3,$i}' tmp | sort -k1.2,1n

The .2 offset omits the first character in the field#1 when making the sort key.
I am not sure if all sort versions retain the order of field#2; some might need sort -k1.2,1n -k2,2 that is certainly slower again.
# 9  
Old 09-27-2017
And here is an awk solution with a minimum memory consumption
Code:
awk -F"\t" '
NR==1{nl=split($0,lines);next}
{for(i=4;i<=nl;i++) out[i]=(((i in out) ? (out[i] RS) : "") $1 "__" $2 "__" $3 " " $i)}
END {for (i=4;i<=nl;i++) {o=out[i];gsub(RS,(RS lines[i] OFS),o);print lines[i],o}}
' tmp

# 10  
Old 09-27-2017
And last but not least, read the file a 3000 times. Seems stupid but is fast enough if your OS has a big buffer cache.
Code:
#!/bin/bash
inputfile=tmp
IFS=$'\t' read -a lines < $inputfile
for ((i=4; i<=${#lines[@]}; i++))
do
  awk -F "\t" -v col=$i -v name="${lines[i-1]}" 'NR>1 {print name,($1 "__" $2 "__" $3),$col}' $inputfile
done

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Modifying col values based on another col

Hi, Please help with this. I have several excel files (with and .xlsx format) with 10-15 columns each. They all have the same type of data but the columns are not ordered in the same way. Here is a 3 column example. What I want to do add the alphabet from column 2 to column 3, provided... (9 Replies)
Discussion started by: newbie83
9 Replies

2. Shell Programming and Scripting

Printing from col x to end of line, except last col

Hello, I have some tab delimited data and I need to move the last col. I could hard code it, awk '{ print $1,$NF,$2,$3,$4,etc }' infile > outfile but it would be nice to know the syntax to print a range cols. I know in cut you can do, cut -f 1,4-8,11- to print fields 1,... (8 Replies)
Discussion started by: LMHmedchem
8 Replies

3. Shell Programming and Scripting

awk? adjacency matrix to adjacency list / correlation matrix to list

Hi everyone I am very new at awk but think that that might be the best strategy for this. I have a matrix very similar to a correlation matrix and in practical terms I need to convert it into a list containing the values from the matrix (one value per line) with the first field of the line (row... (5 Replies)
Discussion started by: stonemonkey
5 Replies

4. UNIX for Advanced & Expert Users

Print line based on highest value of col (B) and repetion of values in col (A)

Hello everyone, I am writing a script to process data from the ATP world tour. I have a file which contains: t=540 y=2011 r=1 p=N409 t=540 y=2011 r=2 p=N409 t=540 y=2011 r=3 p=N409 t=540 y=2011 r=4 p=N409 t=520 y=2011 r=1 p=N409 t=520 y=2011 r=2 p=N409 t=520 y=2011 r=3 p=N409 The... (4 Replies)
Discussion started by: imahmoud
4 Replies

5. Ubuntu

How to convert full data matrix to linearised left data matrix?

Hi all, Is there a way to convert full data matrix to linearised left data matrix? e.g full data matrix Bh1 Bh2 Bh3 Bh4 Bh5 Bh6 Bh7 Bh1 0 0.241058 0.236129 0.244397 0.237479 0.240767 0.245245 Bh2 0.241058 0 0.240594 0.241931 0.241975 ... (8 Replies)
Discussion started by: evoll
8 Replies

6. Shell Programming and Scripting

how to add new col in a file

Hi, Experts, I have a requirement as following: my source file: a a a b b c c c c I need add one more colume as following: 1 a 2 a 3 a 1 b 2 b 1 c 2 c (4 Replies)
Discussion started by: ken002
4 Replies

7. Shell Programming and Scripting

i can't cut the third col

SW_dist_intr false Enable SW distribution of interrupts True autorestart true Automatically REBOOT OS after a crash True boottype disk N/A False capacity_inc 1.00 ... (7 Replies)
Discussion started by: maxim42
7 Replies

8. Shell Programming and Scripting

diagonal matrix to square matrix

Hello, all! I am struggling with a short script to read a diagonal matrix for later retrieval. 1.000 0.234 0.435 0.123 0.012 0.102 0.325 0.412 0.087 0.098 1.000 0.111 0.412 0.115 0.058 0.091 0.190 0.045 0.058 1.000 0.205 0.542 0.335 0.054 0.117 0.203 0.125 1.000 0.587 0.159 0.357... (11 Replies)
Discussion started by: yifangt
11 Replies

9. Ubuntu

Match col 1 of File 1 with col 1 File 2 and create a 3rd file

Hello, I have a 1.6 GB file that I would like to modify by matching some ids in col1 with the ids in col 1 of file2.txt and save the results into a 3rd file. For example: File 1 has 1411 rows, I ignore how many columns it has (thousands) File 2 has 311 rows, 1 column Would like to... (7 Replies)
Discussion started by: sogi
7 Replies

10. UNIX for Advanced & Expert Users

Help On col command

Hello Can Any1 tell me the difference between the col command and the col command with the -f option. I tried running both of them but i can't see any difference. Please guide me. (1 Reply)
Discussion started by: rahulrathod
1 Replies
Login or Register to Ask a Question