Constructing a Matrix


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Constructing a Matrix
# 1  
Old 02-24-2014
Constructing a Matrix

Hi,

I do have couple of files in a folder. The names of each of the files have a pattern.
Code:
ahet_005678.txt
ahet_005898.txt
ahet_007678.txt
ahet_004778.txt
...
...
ahet_002378.txt

Each of the above files have the same pattern of data with 4 columns and have an header for the last 3 columns.

Code:
vi ahet_005678.txt

  A1 A2 A3
1 0.5 0 0
2 0 1 0.571
3 0.010 6 3.333
4 0 0 0
5 0.2 0 0
6 0 0 0
7 0.00527 1 1.667
8 0 0 0
9 0 0 0


vi ahet_005898.txt

  A1 A2 A3
1 0 0 0
2 0.5 1 0.571
3 0.010 6 3.333
4 0.34 0 0
5 0 0 0
6 0.09 0 0
7 0.00527 1 1.667
8 0 0 0
9 0 0 0

vi ahet_007678.txt

  A1 A2 A3
1 0.9 0 0
2 0.5 1 0.571
3 0.010 6 3.333
4 0 0 0
5 0.67 0 0
6 0 0 0
7 0.00527 1 1.667
8 0 0 0
9 0.45 0 0

I would like to get a single output file by greping the contents of each of files and a portion of the name of each of the file. More specifically, grep the numerical portion after the '_' in each of the file names, for example, in the file name ahet_005898.txt, get (grep)005898 and also the 2rd column of that file and then paste it horizontally in the result file. Desired out put file is in the matrix format
Code:
005678 0.5 0 0.10 0 0.2 0 0.00527 0 0
005898 0 0.5 0.010 0.34 0 0.09 0.00527 0 0
007678 0.9 0.5 0.010 0 0.67 0 0.00527 0 0.45
....
.....
....

Please let me know the best way in awk to grep portions of the name of the file and the second column and make a matrix using all the files in a directory.
# 2  
Old 02-24-2014
What have you tried?
# 3  
Old 02-24-2014
I tried the following in sed
Code:
sed -e :a -e '{N; s/\n/ /g; ta}' infile

But it only selects the first column. I am not sure how to extract the file name and print the second column horizontally for all the 1000's of file I have in the directory.
# 4  
Old 02-24-2014
An awk approach

Code:
awk 'f!=FILENAME{f=g=FILENAME;gsub(/.*_|.txt/,x,g);printf "\n"g} 
     NF>3{printf " "$2} 
     END{printf "\n"}' ahet_00*

--ahamed
# 5  
Old 02-25-2014
some kind of python code

Code:
import glob
for i in glob.glob('leo/*.txt'):
 name=i[i.index('_')+1:i.rindex('.')]
 with open(i) as file:
  l=[name]
  for line in file:
    if line.startswith(' '):
        pass
    else:
        l.append(line.split(" ")[1])
  print(' '.join(l))

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk? adjacency matrix to adjacency list / correlation matrix to list

Hi everyone I am very new at awk but think that that might be the best strategy for this. I have a matrix very similar to a correlation matrix and in practical terms I need to convert it into a list containing the values from the matrix (one value per line) with the first field of the line (row... (5 Replies)
Discussion started by: stonemonkey
5 Replies

2. Ubuntu

How to convert full data matrix to linearised left data matrix?

Hi all, Is there a way to convert full data matrix to linearised left data matrix? e.g full data matrix Bh1 Bh2 Bh3 Bh4 Bh5 Bh6 Bh7 Bh1 0 0.241058 0.236129 0.244397 0.237479 0.240767 0.245245 Bh2 0.241058 0 0.240594 0.241931 0.241975 ... (8 Replies)
Discussion started by: evoll
8 Replies

3. UNIX for Dummies Questions & Answers

constructing a file for printing

I have a text file, data.txt, which looks like :- head1 head2 head3 data1 1 data 1 2 data 1 3 data 2 1 data 2 2 data 2 3 data 3 1 data 3 2 data 3 3 etc etc I would like to print this file, using the lp -d command, with a heading line, as follows :- Report for whatever department as... (2 Replies)
Discussion started by: malts18
2 Replies

4. Shell Programming and Scripting

Constructing numbers from input lines

I have a file with the information shown and I want to capture the entry having the rgdt tag and taking the value (the location of the represents the decimal point, for example 0p50 represents 0.50) I then want to divide the number at the end of each line by the value 0.50 I want to do... (7 Replies)
Discussion started by: kristinu
7 Replies

5. Shell Programming and Scripting

Constructing a string by reading the contents of 2 files

Hi all - I am trying to write a script to generate a string that gets sent to an IP phone using wget. This is to upload a phone book of up to 100 entries so the string needs to be constructed in one 'hit'. I am a relative newbie to shell scripts but willing to try anything! I have 2 files: ... (3 Replies)
Discussion started by: cakerack
3 Replies

6. Shell Programming and Scripting

Help With Constructing A Korn Shell Search Loop

Hello All, I am a statistician and I am very new to the world of ksh programming. Daily, I analyze millions of rows of data and land information to DB2 tables. I have recently been asked to develop a ksh script to FTP an export file containing line item data from the production environment to the... (2 Replies)
Discussion started by: jonesdk5
2 Replies

7. Shell Programming and Scripting

diagonal matrix to square matrix

Hello, all! I am struggling with a short script to read a diagonal matrix for later retrieval. 1.000 0.234 0.435 0.123 0.012 0.102 0.325 0.412 0.087 0.098 1.000 0.111 0.412 0.115 0.058 0.091 0.190 0.045 0.058 1.000 0.205 0.542 0.335 0.054 0.117 0.203 0.125 1.000 0.587 0.159 0.357... (11 Replies)
Discussion started by: yifangt
11 Replies

8. Shell Programming and Scripting

Suggestions on constructing a log file

Hi, I have a series of BASH shell scripts that run together. I would like it so that these scripts construct a log file as they run. This log file would ideally be a text file that contains exactly (including blank lines) what is output to the terminal. What is the best way to accomplish... (3 Replies)
Discussion started by: msb65
3 Replies

9. Programming

Constructing a server daemon / questions on handling clients

Hi All, I'm interested in the best way to serve multiple clients from a server process. Now, our server is going to serve data on a custom protocol and it is going to get / process that data from a number of possible back-ends including files (text or xml) and MySQL/some other database.... (2 Replies)
Discussion started by: dragonfly
2 Replies
Login or Register to Ask a Question