Redirecting stdout inside a loop - Page 3 | Unix Linux Forums | UNIX for Dummies Questions & Answers

  Go Back    


UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

Redirecting stdout inside a loop

UNIX for Dummies Questions & Answers


Tags
redirect stout loop sed awk

Reply    
 
Thread Tools Search this Thread Display Modes
    #15  
Old 08-11-2014
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 15 September 2014, 7:56 PM EDT
Location: Saskatchewan
Posts: 19,352
Thanks: 786
Thanked 3,255 Times in 3,051 Posts
Quote:
Originally Posted by hmortens View Post
is it possible to organize it real time? like as it is being written to file?
That's your program's output. Once it's written to the file, it's too late to do anything about it inside the loop. Organize it before you put it into the file.


Quote:
I was not specific about this before because I thought it would be the same pattern search, but I need to pull lines containing "No. input reads:", "No. reads in analysis:", "mapped...reads to genomes(s)", and "Detected:"
If you want to pile all three commands together then filter them, do it like this, before it gets to the file:


Code:
for ... in ...
do
        # All commands get piped through egrep, which then prints to stdout
        (
                command1
                command2
                command3
        ) | egrep 'expression1|expression2|expression3'

        # This file gets printed to stdout
        cat restoffile

# Everything printed to stdout gets saved to onesingleoutputfile, in order
done > onesingleoutputfile

The ( ) join them together in a subshell, letting you put a pipe on the end to filter the contents of all three.

Last edited by Corona688; 08-11-2014 at 05:54 PM..
Sponsored Links
    #16  
Old 4 Weeks Ago
hmortens hmortens is offline
Registered User
 
Join Date: Aug 2014
Last Activity: 21 August 2014, 2:05 PM EDT
Location: Research Triangle Park, NC
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
how would you integrate this with summary.txt in this example?
I am guessing I would put the
Code:
for ... in ...
do
        # All commands get piped through egrep, which then prints to stdout
        (
                command1
                command2
                command3
        ) | egrep 'expression1|expression2|expression3'

for i in "$LOCATION"/*_fastqc; do
	infile3=`ls $i/summary.txt`
	echo "***testing***" $infile3
cat <$infile3 >>summary.fastqc.$date_formatted
done
 # This file gets printed to stdout ADD summary.txt HERE???
        cat restoffile summary.fastqc.$date_formatted
# Everything printed to stdout gets saved to onesingleoutputfile, in order
done > onesingleoutputfile

Sponsored Links
    #17  
Old 4 Weeks Ago
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 15 September 2014, 7:56 PM EDT
Location: Saskatchewan
Posts: 19,352
Thanks: 786
Thanked 3,255 Times in 3,051 Posts
Quote:
Originally Posted by hmortens View Post
how would you integrate this with summary.txt in this example?
I have no idea.

I am completely unable to give details, due to your complete inability to give details. I don't think that "$LOCATION"/*_fastqc is what you want since that will match all fastqc as I understand it, but cannot say what you do want.

I still don't know which things of your code generate what bits of your output. I still don't know where most of your files are stored... I am not even sure if these fastqc files are inside or outside your 'sample' things or not, which would have helped me answer your last question. In four days and three pages of pulling teeth, I have found one eye, one ear, and one toe of the elephant. I suppose these must be proprietary things you're not allowed to divulge.

I do note that you're blindly continuing to redirect >>summary.fastqc.$date_formatted even though you've told me you don't want to put it there. To not redirect into that file, I suggest not redirecting into that file... It will end up in stdout instead, which will be caught by the outer loop.

If that's not exactly when you wanted it, then print it sooner -- or later. If you wanted only part of it, then filter it then and there. If you print exactly what you want, in the order you want, that's what you'll get in your output.

I think I have given you all the techniques you need.

Last edited by Corona688; 4 Weeks Ago at 11:41 AM..
    #18  
Old 4 Weeks Ago
hmortens hmortens is offline
Registered User
 
Join Date: Aug 2014
Last Activity: 21 August 2014, 2:05 PM EDT
Location: Research Triangle Park, NC
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
wow. communication breakdown. Part of the difficulty may be that the input is large, genomic files and the various output of each program is statistical in nature. I did not think this was of interest to the linux forum or consequence for manipulating the stdout and single outfile that I have pasted here. thanks for your help. I will post any solution.
Sponsored Links
    #19  
Old 4 Weeks Ago
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 15 September 2014, 7:56 PM EDT
Location: Saskatchewan
Posts: 19,352
Thanks: 786
Thanked 3,255 Times in 3,051 Posts
When you have a big problem, a small model of it is sometimes sufficient to find out what you need. But not too small a model. You could also break the problem into halves, to make it simpler to deal with. Right now we either solve all of it or nothing...

You've also been confusing some terms, and I think I sense some confusion on how 'stdout' and redirection in general work, your questions may not mean what you think they do, causing confusion on both sides. Perhaps if you explained what you wanted, instead of the way you wanted to accomplish it.

Perhaps something like:

Quote:
Part 1: I have 3 programs which print <three sets of madeup lines>, of which I need <these subsets of them> as selected by <these madeup strings>, printed in <this order>. There is also a single file which looks like <these pretend contents> which must be printed in its entirety, after <those subsets>, but before <this subset>. How do I get <this exact pretend-output>, given those exact pretend-inputs?
Quote:
Part 2: Same problem, but now there are thousands of these which must be looped over. Each is in its own folder. They have <this string> in common, changing <this last part> for each folder. Additionally, <these contents> are are in <individual different folders>, but the two parts have <this string> in common if that helps you pair them up.
This is what I've been trying to get. This is sufficient to build a working scale-model of your problem, which we can code around and test.

Once we get that working, you can fill in the pretend bits with real bits and see if it still works for the larger problem.

Or you could keep hiding bits and dodging questions and hope we can guess the rest...

Last edited by Corona688; 4 Weeks Ago at 01:54 PM..
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Redirecting stdout problem igurov Shell Programming and Scripting 2 06-20-2012 05:20 PM
redirecting to stdout in betwen command phoenix_nebula Shell Programming and Scripting 4 03-16-2010 01:09 AM
Redirecting several outputs to /dev/stdout aplaydoc UNIX for Dummies Questions & Answers 1 03-06-2009 10:42 PM
Redirecting part of output to stdout Legend986 Shell Programming and Scripting 2 10-17-2008 03:20 PM
implicitly redirecting stdout to a file ALTRUNVRSOFLN Shell Programming and Scripting 1 09-23-2008 03:42 PM



All times are GMT -4. The time now is 05:25 AM.