Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Extract information from file

Shell Programming and Scripting


Reply    
 
Thread Tools Search this Thread Display Modes
    #8  
Old Unix and Linux 2 Weeks Ago
vedanta vedanta is offline
Registered User
 
Join Date: Mar 2013
Last Activity: 15 May 2017, 3:11 AM EDT
Posts: 26
Thanks: 11
Thanked 0 Times in 0 Posts
when I replaced *.sh by the line in your original code

I am getting below error:

awk: cmd. line:37: fatal: can not open file 'awk' for reading ( no such file or directory )
Sponsored Links
    #9  
Old Unix and Linux 2 Weeks Ago
RavinderSingh13 RavinderSingh13 is offline Forum Advisor  
Registered User
 
Join Date: May 2013
Last Activity: 27 May 2017, 12:25 PM EDT
Location: Chennai
Posts: 2,477
Thanks: 552
Thanked 1,172 Times in 1,055 Posts
Quote:
Originally Posted by vedanta View Post
when I replaced *.sh by the line in your original code
I am getting below error:
awk: cmd. line:37: fatal: can not open file 'awk' for reading ( no such file or directory )
Hello Vedanta,

Are you running this awk program too as a .sh script? If yes then you may need to consider that it will take this script also as this script is also ending with .sh. If this is not the case then you may need to post whatever files are present in the directory and we may need to see what's going on.

Thanks,
R. Singh
Sponsored Links
    #10  
Old Unix and Linux 2 Weeks Ago
vedanta vedanta is offline
Registered User
 
Join Date: Mar 2013
Last Activity: 15 May 2017, 3:11 AM EDT
Posts: 26
Thanks: 11
Thanked 0 Times in 0 Posts
I am running the program as a .sh script.


Code:
#!/bin/ksh
awk ' NR==1 {
                        print "File      | Jobname |  node  | pfile               | metafile            | tname | priority | delay" # to be replaced by tab
                   }
              /maxdelay/{
                                delay=$NF;
                                next
                        }
              /node/    {
                                node=$NF;
                                next
                        }
              /tname/   {
                                name=$NF;
                                next
                        }
              /pfile/   {
                                file=file?file","$NF:$NF;
                                next
                        }
              /metafile/{
                                metafile=metafile?metafile","$NF:$NF;
                                next
                        }
              /jobname/ {
                                jobname=$NF;
                                next
                        }
              /priority/{
                                pri=$NF;
                                next
                        }
              /exit/    {
                                print FILENAME OFS jobname OFS node OFS file OFS metafile OFS name OFS pri OFS delay;
                                #jobname=node=file=metafile=name=pri=delay="";
                        }
             ' OFS="\t"   awk '/^submit/{print FILENAME;nextfile}' *.sh  # replaced *.sh with awk

Please see last line. I want to pick only selected files ( 1000 files from many files).

I executed like below

Code:
ksh test.ksh

    #11  
Old Unix and Linux 2 Weeks Ago
RavinderSingh13 RavinderSingh13 is offline Forum Advisor  
Registered User
 
Join Date: May 2013
Last Activity: 27 May 2017, 12:25 PM EDT
Location: Chennai
Posts: 2,477
Thanks: 552
Thanked 1,172 Times in 1,055 Posts
Quote:
Originally Posted by vedanta View Post
I am running the program as a .sh script.


Code:
#!/bin/ksh
awk ' NR==1 {
                        print "File      | Jobname |  node  | pfile               | metafile            | tname | priority | delay" # to be replaced by tab
                   }
              /maxdelay/{
                                delay=$NF;
                                next
                        }
              /node/    {
                                node=$NF;
                                next
                        }
              /tname/   {
                                name=$NF;
                                next
                        }
              /pfile/   {
                                file=file?file","$NF:$NF;
                                next
                        }
              /metafile/{
                                metafile=metafile?metafile","$NF:$NF;
                                next
                        }
              /jobname/ {
                                jobname=$NF;
                                next
                        }
              /priority/{
                                pri=$NF;
                                next
                        }
              /exit/    {
                                print FILENAME OFS jobname OFS node OFS file OFS metafile OFS name OFS pri OFS delay;
                                #jobname=node=file=metafile=name=pri=delay="";
                        }
             ' OFS="\t"   awk '/^submit/{print FILENAME;nextfile}' *.sh  # replaced *.sh with awk

Please see last line. I want to pick only selected files ( 1000 files from many files).

I executed like below

Code:
ksh test.ksh

Hello Vedanta,

I am shocked, as I have already told you not to mix 2 codes. My first code was to get the output in your expected shape from a Input_file and second code which was posted by you I corrected(which I provided fair warning like don't mix them and open a new thread on same).

Would like to request you if you can segregate your requirements as it is very confusing now.

NOTE: Above your code will NOT work in this style you are using 2 awk by providing 1 time Input_file which is *.sh.

Thanks,
R. Singh
Sponsored Links
    #12  
Old Unix and Linux 2 Weeks Ago
vedanta vedanta is offline
Registered User
 
Join Date: Mar 2013
Last Activity: 15 May 2017, 3:11 AM EDT
Posts: 26
Thanks: 11
Thanked 0 Times in 0 Posts
Hi,
I want to pull all the files that have the pattern like the input file given. In a directory there are say 5000 files out of which 1000 files have the text starting with 'submit file'.
and from those files I want to get the output I mentioned earlier.

First filter those files which have the text starting with 'submit file' ( out of 5000, I would have 1000 such input file ) and read those input files which have have the text starting with 'submit file' and get the output in the required format I mentioned earlier. So basically, I want to filter out the files and read those file as input only and get required output. thanks
Sponsored Links
    #13  
Old Unix and Linux 2 Weeks Ago
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is online now Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 28 May 2017, 4:24 AM EDT
Location: San Jose, CA, USA
Posts: 10,308
Thanks: 521
Thanked 3,587 Times in 3,055 Posts
Quote:
Originally Posted by vedanta View Post
Hi,
I want to pull all the files that have the pattern like the input file given. In a directory there are say 5000 files out of which 1000 files have the text starting with 'submit file'.
and from those files I want to get the output I mentioned earlier.

First filter those files which have the text starting with 'submit file' ( out of 5000, I would have 1000 such input file ) and read those input files which have have the text starting with 'submit file' and get the output in the required format I mentioned earlier. So basically, I want to filter out the files and read those file as input only and get required output. thanks
This is very convoluted logic. Using awk to read all of your 5000 files to get a list of files containing a certain string to use as arguments to another awk script is grossly inefficient since you have to read each of your 5000 and then read the selected 1000 files a second time. There is very seldom a need to invoke awk twice, but if you must you have to actually invoke awk twice instead of just using the command line arguments that would be used to invoke awk as operands to awk as in:

Code:
awk '...
...
             ' OFS="\t"   $(awk '/^submit/{print FILENAME;nextfile}' *.sh)  # replaced *.sh with awk

or:

Code:
awk '...
...
             ' OFS="\t"   $(grep -l '^submit' *.sh)  # filter *.sh with grep

But, as I said before, putting it all in a single awk script would be much more efficient:

Code:
#!/bin/ksh
awk ' NR==1 {
                        print "File      | Jobname |  node  | pfile               | metafile            | tname | priority | delay" # to be replaced by tab
                   }
              /maxdelay/{
                                delay=$NF;
                                next
                        }
              /node/    {
                                node=$NF;
                                next
                        }
              /tname/   {
                                name=$NF;
                                next
                        }
              /pfile/   {
                                file=file?file","$NF:$NF;
                                next
                        }
              /metafile/{
                                metafile=metafile?metafile","$NF:$NF;
                                next
                        }
              /jobname/ {
                                jobname=$NF;
                                next
                        }
              /priority/{
                                pri=$NF;
                                next
                        }
              /^submit/{
                                printit=1;
                                next
                        }
              /exit/ && printit{
                                print FILENAME OFS jobname OFS node OFS file OFS metafile OFS name OFS pri OFS delay;
                                #jobname=node=file=metafile=name=pri=delay="";
                                printit=0
                                nextfile
                        }
             ' OFS="\t" *.sh

The Following User Says Thank You to Don Cragun For This Useful Post:
vedanta (1 Week Ago)
Sponsored Links
    #14  
Old Unix and Linux 1 Week Ago
vedanta vedanta is offline
Registered User
 
Join Date: Mar 2013
Last Activity: 15 May 2017, 3:11 AM EDT
Posts: 26
Thanks: 11
Thanked 0 Times in 0 Posts
Many thanks!
One problem I am seeing here is that awk also is reading the comment lines.
Is there a way to ignore all comment lines at the very beginning? Should I need to create a separate post for this? Thanks.


I have used below as a solution

added !/^#/ eg,
/pfile/ && !/^#/ { ..



Last edited by vedanta; 1 Week Ago at 03:11 AM..
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Extract information from file jiam912 Shell Programming and Scripting 11 11-12-2016 01:06 AM
Extract information from txt file edekP Shell Programming and Scripting 11 11-09-2016 12:32 PM
How to extract information from a file? the_simpsons Shell Programming and Scripting 9 05-20-2014 05:47 PM
Extract various information from a log file SilvesterJ Shell Programming and Scripting 4 09-27-2011 12:00 PM
extract and format information from a file sujoy101 Shell Programming and Scripting 12 04-16-2008 06:58 AM



All times are GMT -4. The time now is 04:46 AM.