Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Read a File line by Line in unix

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read a File line by Line in unix
# 1  
Old 08-12-2011
Read a File line by Line in unix
Hi ,

I have few files in a directory .
I need to read all the files in that direcory and search for certain criteria in each file and write into a outputfile.
I am also writing the each file name to the outputfile .
For the above criteria , i have written the script.
*****************************************************
Code:
#!/bin/sh
rm -f  LookupFile.txt
for i in `ls *.log`
do
echo -ne $i >> LookupFile.txt
awk  '/CMN_1716 Lookup/
     /Creating Lookup Cache/
     /Lookup cache creation completed/' $i >> LookupFile.txt
echo  $i >> LookupFile.txt
done

***********************************************************
o/p for the above script
Code:
LookupFile.txt--Filename
 

(sessionlogname1)TRANSF_1_1_1> CMN_1716 Lookup [lookuptransformationname1] uses database connection [Relational:DataWarehouse] in codepage [UTF-8 encoding of Unicode]
TRANSF_1_1_1> DBG_21079 Creating Lookup Cache : (Fri Aug 05 00:46:31 2011)
TRANSF_1_1_1> DBG_21294 Lookup cache creation completed : (Fri Aug 05 00:47:15 2011)
TRANSF_1_1_1> CMN_1716 Lookup [lookuptransformationname2] uses database connection [Relational:DataWarehouse] in codepage [UTF-8 encoding of Unicode]
TRANSF_1_1_1> DBG_21079 Creating Lookup Cache : (Fri Aug 05 00:47:16 2011)
TRANSF_1_1_1> DBG_21294 Lookup cache creation completed : (Fri Aug 05 00:47:16 2011)
sessionlogname1
(sessionlogname2)TRANSF_1_1_1> CMN_1716 Lookup [lookuptransformationname] uses database connection [Relational:DataWarehouse] in codepage [UTF-8 encoding of Unicode]
TRANSF_1_1_1> DBG_21079 Creating Lookup Cache : (Mon Aug 08 03:42:15 2011)
TRANSF_1_1_1> DBG_21294 Lookup cache creation completed : (Mon Aug 08 03:42:15 2011)
sessionlogname2

*****************************************
In the second stage, i need to read the LookupFile.txt file line by line and get the required information
for each lookup transformation into one line and write to a file.
So the file should be in the below mentioned format.

--------Final file structure------------
Code:
Filename1,lookuptransformationname1,cache creation time(Fri Aug 05 00:46:31 2011),cache completed time(Fri Aug 05 00:47:15 2011)
Filename1,lookuptransformationname2,Fri Aug 05 00:47:16 2011,Fri Aug 05 00:47:16 2011
Filename2,lookuptransformationname,Mon Aug 08 03:42:15 2011,Mon Aug 08 03:42:15 2011

So for each lookuptransformation in the file , it should write into one line with the assoiciated information.

Please help me in this issue .
Thanks,
Burwood
Last edited by pludi; 08-12-2011 at 06:00 AM..
# 2  
Old 08-12-2011
Try and adapt the following awk script that makes all the work (I hope) :
Code:
awk -v FS='[][()]' '

    FNR==1 {
        printLookup();
        FileName = FILENAME;
    }

    /CMN_1716 Lookup/ {
        printLookup();
        LookupTransName = (/^\(/ ? $4 : $2);
        next;
    }

    /Creating Lookup Cache/ {
        CreationTime = $2;
        next;
    }

    /Lookup cache creation completed/ {
        CompleteTime = $2;
        next;
    }

    END {
        printLookup();
    }

    function printLookup(from) {
        if (! LookupTransName) return;
        printf "%s,%s,%s,%s\n", FileName, LookupTransName, CreationTime, CompleteTime;
        LookupTransName = CreationTime = CompleteTime = "";
    }


' *.log </dev/null 2>/dev/null

Jean-Pierre.
This User Gave Thanks to aigles For This Post:
burwood
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script UNIX to read text file line by line

i have a text file as belows, it includes 2 columns, 1st is the column name, 2nd is the file_name data_file.txt column_name file_name col1 file1 col2 file2 col3 file1 col4 file1 col5 file2 now, i would like to... (4 Replies)
Discussion started by: tester111
4 Replies

2. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

3. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

4. Shell Programming and Scripting

How to read a two files, line by line in UNIX script and how to assign shell variable to awk ..?

Input are file and file1 file contains store.bal product.bal category.bal admin.bal file1 contains flip.store.bal ::FFFF:BADC:CD28,::FFFF:558E:11C5,6,8,2,1,::FFFF:81C8:CA8B,::FFFF:BADC:CD28,1,0,0,0,::FFFF:81C8:11C5,2,1,0,0,::FFFF:81DC:3111,1,0,1,0 store.bal.... (2 Replies)
Discussion started by: veeruasu
2 Replies

5. Shell Programming and Scripting

How to read a file, line by line in UNIX script?

#!/bin/bash file1="yxd" for file in `cat file3.txt`; do cat $file1 | awk '{split($1,dp,"."); lend=length(dp); if(lend>2) { dom=dp"."dp; split($2,ipl,","); for(i=1; i<=length(ipl); i++) { if(ipl~/:/) { if(ipl~/:/) i++; rip=ipl; i++; raq=ipl; rtq=0; si=i+4; for(;i<si;i++) rtq+=ipl; rsub="";... (2 Replies)
Discussion started by: veeruasu
2 Replies

6. Shell Programming and Scripting

how to read the contents of two files line by line and compare the line by line?

Hi All, I'm trying to figure out which are the trusted-ips and which are not using a script file.. I have a file named 'ip-list.txt' which contains some ip addresses and another file named 'trusted-ip-list.txt' which also contains some ip addresses. I want to read a line from... (4 Replies)
Discussion started by: mjavalkar
4 Replies

7. Shell Programming and Scripting

Shell script to read a text file line by line & process it...

Hi , I am trying to write an shell, which reads a text file (from a location) having a list of numbers of strictly 5 digits only ex: 33144 Now my script will check : 1) that each entry is only 5 digits & numeric only, no alphabets, & its not empty. 2)then it executes a shell script called... (8 Replies)
Discussion started by: new_to_shell
8 Replies

8. Shell Programming and Scripting

Read file line by line and process the line to generate another file

Hi, i have file which contains data as below(Only sample shown, it may contain more data similar to the one shown here) i need to read this file line by line and generate an output file like the one below i.e based on N value the number of MSISDNs will vary, if N=1 then the following... (14 Replies)
Discussion started by: aemunathan
14 Replies

9. Shell Programming and Scripting

bash: read file line by line (lines have '\0') - not full line has read???

I am using the while-loop to read a file. The file has lines with null-terminated strings (words, actually.) What I have by that reading - just a first word up to '\0'! I need to have whole string up to 'new line' - (LF, 10#10, 16#A) What I am doing wrong? #make file 'grb' with... (6 Replies)
Discussion started by: alex_5161
6 Replies

10. Shell Programming and Scripting

shell script to read a line in gps receiver log file and append that line to new file

Hi, I have gps receiver log..its giving readings .like below Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GPSD,R=1 $GPGSV,3,1,11,08,16,328,40,11,36,127,00,28,33,283,39,20,11,165,00*71... (3 Replies)
Discussion started by: gudivada213
3 Replies
Login or Register to Ask a Question