Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Manually built the code below.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Manually built the code below.
# 1  
Old 10-11-2011
Manually build the code below.
Hi All,

I am faced with a problem. I came to know that large scale problems in MATLAB are very slow and with loops they are even slower.

I have the MATLAB script below that does some large scale computation for my school project work. I've been running this code for long now but all it did was to create more problems for me both in space and time complexities. So, I remove the loop and showing you the "non-loop" code below. (Maybe others can use it if they want to solve a similar problem Smilie )

Code:
str = int2str ( j );
   string='/data/word_coordinates/';
   str=strcat(string,str);
   file_coordinate_word = strcat ( str , '.wcor' );

   coordinate_vectors=dlmread(file_coordinate_word);

   for coordinates=1:numlines,
       W(coordinates,1:200)=term_vectors(coordinate_vectors(coordinates),:);
   end
   D=document_vectors( j , : );
   b=sdpvar(1,numlines);
   e=ones(1,numlines,1);
   Constraints=[0<=b<=1,(b*e')==1];
   solvesdp(Constraints,[],norm(D-b*W,2));
   fprintf (fd ,'%f ',double(b));
   fprintf ( fd , '%c\n' , '' );
   clear Constraints
   clear D
   clear W
   clear b
   clear coordinate_vectors
   clear coordinates
   clear e
   clear file_coordinate_word
   clear j
   clear str
   clear string

What I used to do was previously using a "for loop" I used to generate values for j, but that was too slow. I then used to compute number of lines in each file, which again takes up some CPU time.

What I have done now is using the command created a file:

file_list.txt
Code:
find . -name "*.dat" | xargs wc -l

Generated the file names along with number of lines. (I had to remove some ./ characters though)

So, now my file_list.txt file look like this. I have even sorted the filenames (second column sorted):

Code:
1234 1
3456 2
56454 3
1233 4
543 6

In the above list, first column contains the numbers of lines in each file and second column contains the file identifier names (my files are all named as 1.dat, 2.dat, 3.dat 6.dat etc) I removed .dat from file_list.txt file.

PS: The file identifier names are not continuously named like 1 2 3 4 until end but in between one or two files are missing like you see above after 4, 6 comes in.

Now, I want to use file_list.txt to create a "very long" MATLAB script "without loop". So, now what I intend to do is to replace all 'j' variable in my MATLAB script to file identifier values (that is second column in file_list.txt) sequentially line by line (it should not ignore any line else my results will all be meaningless).
Second, the 'numlines' variable in my MATLAB script has to be replaced with the number of lines from file_list.txt (first column).

My intended output file should look something like below, so that if I run the code by pressing RUN button entire script begins to run at once.


Code:
str = int2str ( 1 );//PLEASE SEE THE CHANGE HERE
   string='/data/word_coordinates/';
   str=strcat(string,str);
   file_coordinate_word = strcat ( str , '.wcor' );

   coordinate_vectors=dlmread(file_coordinate_word);

   for coordinates=1:1234,//CHANGE HERE
       W(coordinates,1:200)=term_vectors(coordinate_vectors(coordinates),:);
   end
   D=document_vectors( 1 , : );//CHANGE HERE
   b=sdpvar(1,1234);//CHANGE HERE
   e=ones(1,1234,1);//CHANGE HERE
   Constraints=[0<=b<=1,(b*e')==1];
   solvesdp(Constraints,[],norm(D-b*W,2));
   fprintf (fd ,'%f ',double(b));
   fprintf ( fd , '%c\n' , '' );
   clear Constraints
   clear D
   clear W
   clear b
   clear coordinate_vectors
   clear coordinates
   clear e
   clear file_coordinate_word
   clear j
   clear str
   clear string

   str = int2str ( 2 );//CHANGE HERE
   string='/data/word_coordinates/';
   str=strcat(string,str);
   file_coordinate_word = strcat ( str , '.wcor' );

   coordinate_vectors=dlmread(file_coordinate_word);

   for coordinates=1:3456,//CHANGE HERE
       W(coordinates,1:200)=term_vectors(coordinate_vectors(coordinates),:);
   end
   D=document_vectors( 2 , : );//CHANGE HERE
   b=sdpvar(1,3456);//CHANGE HERE
   e=ones(1,3456,1);//CHANGE HERE
   Constraints=[0<=b<=1,(b*e')==1];
   solvesdp(Constraints,[],norm(D-b*W,2));
   fprintf (fd ,'%f ',double(b));
   fprintf ( fd , '%c\n' , '' );
   clear Constraints
   clear D
   clear W
   clear b
   clear coordinate_vectors
   clear coordinates
   clear e
   clear file_coordinate_word
   clear j
   clear str
   clear string

[/CODE]

And this is how it continues in the entire file to create one single MATLAB script file which I can run overnight. Unfortunately, my file list file is too huge to do it manually.

I am open to suggestions too, if anyone can kindly suggest me any better way of doing what I intend to do. I am using Linux.
Last edited by shoaibjameel123; 10-11-2011 at 06:58 AM.. Reason: Minor grammar
# 2  
Old 10-18-2011
Yup. You can do this by making a "hereto" document and replacing your CHANGE HERE parameters with shell variable names, like $LINECOUNT and $FILENAME. THen wrap that in a big loop.

But m4 offers a sleeker way. Name your placeholders as above but without the $ sign and save the changes to a file, say matlab.tmpl
Code:
cat file_list.txt |
while read linecount filename; do
   m4 -DFILENAME="$filename" -DLINECOUNT="$linecount" matlab.tmpl
done > bigfile.m

Last edited by otheus; 10-18-2011 at 09:13 AM.. Reason: I thought m4 could handle macros beginning with $. It cannot
This User Gave Thanks to otheus For This Post:
shoaibjameel123
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Using gFORTRAN to compile something built for g77

Hi, I am having a problem compiling a program with gfortran. The program compiles with g77 and f77 but I don't have those. I edited the makefile from FC = g77 to FC = gfortran but when I run it I get a slew of undefined reference errors. Any suggestions? (8 Replies)
Discussion started by: butson
8 Replies

2. UNIX for Advanced & Expert Users

Kernel Modules Not geting built

I installed in VM the Mandriva Linux with 2.6.27 kernel. But presently when I fire make the modules .ko does not get built. I get the following output on firing command in the kernel module folder. $ make Building first_driver.c ... make: Entering directory... (1 Reply)
Discussion started by: rupeshkp728
1 Replies

3. AIX

OpenSSH built in chroot facility

Hi all, I'm trying to set up a chroot sftp using OpenSSH. But I'm still having problems. I'm using AIX 5.3 My system and OpenSSH version as follows host1:/>oslevel 5.3.0.0 host1:/>oslevel -r 5300-10 host1:/>ssh -V OpenSSH_5.0p1, OpenSSL 0.9.8h 28 May 2008 host1:/>lslpp -l | grep open... (2 Replies)
Discussion started by: h@foorsa.biz
2 Replies

4. Shell Programming and Scripting

bash built-in

Is there any command or VARIABLE in unix to display only bash builtin commands?. Some days back I worked on that, but now I do not remember. Can anyone please reply for this?... (4 Replies)
Discussion started by: gwgreen1
4 Replies

5. Linux

distro with built in java.

i installed fedora core6, then came to know that java is not supplied with it, can u help me by naming some distros with built in java, does RHL5+ has it? (2 Replies)
Discussion started by: zius_oram
2 Replies

6. Shell Programming and Scripting

built-in hex editor?

I'm using Mac OS 10.2.2 and I need to find a command line hex editor. I figure there's already one built-in, but I just don't know enough about Unix, yet. Basically, what I want to do is tell the hexeditor to open a file, replace a specific offset with a new value, then put the resulting file... (7 Replies)
Discussion started by: Loriel
7 Replies

7. Gentoo

Which GCC Built My Kernel?

I'm playing around with the kqemu accelerator for the QEMU virtualization software. I can't remember which version of gcc I used to build my kernel. I have 3.4.6 and 4.1.1 on this (Gentoo) system. I seem to remember there is some command I can run against binaries that will tell me what gcc (and... (5 Replies)
Discussion started by: deckard
5 Replies

8. Shell Programming and Scripting

ksh built-in function

Does anyone know why the following expression return an error ] while the following one not ?? Thanks (1 Reply)
Discussion started by: solea
1 Replies

9. Shell Programming and Scripting

BUILT-IN command scripts

I am new to Unix scripting. I would like to know if someone can point me to a site which lists any built-in commands in there scripted form. I thought to start with the basics and learn from example. Thanks JSP (2 Replies)
Discussion started by: JSP
2 Replies

10. UNIX for Dummies Questions & Answers

awk built in variables

Dear experts I am learning awk command through some books on Solaris 8. I have tested the folloing command awk 'BEGIN { print match ("And" , /d/)}' then the result is as following awk: syntax error near line 1 awk: illegal statement near line 1 Could you please help on this and just... (4 Replies)
Discussion started by: Reza Nazarian
4 Replies
Login or Register to Ask a Question