Read file line by line and process the line to generate another file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read file line by line and process the line to generate another file
# 1  
Old 04-17-2011
Read file line by line and process the line to generate another file

Hi,
i have file which contains data as below(Only sample shown, it may contain more data similar to the one shown here)
Quote:
[IMS]
DH=95
SEC=1
[SEC1]
N=1
MSISDN1=123456789
SMSMSG=2050-IBS-ICF-FurnishingSect-IV in BSC ANagar down at 18:27:56 08/04/2011
[IMS]
HEADERMSG=OMCR-MA-UP
DH=95
SEC=1
[SEC1]
N=6
MSISDN1=12345
MSISDN2=23456
MSISDN3=34567
MSISDN4=45678
MSISDN5=56789
MSISDN6=67890
SMSMSG=1453-CM-QTRS-IV5-G in solo Road II UP at 13:04:12 15/04/11
[IMS]
HEADERMSG=OMCR-MA-UP
DH=95
SEC=1
[SEC1]
N=5
MSISDN1=12345
MSISDN2=32453
MSISDN3=456789
MSISDN4=145678
MSISDN5=567890
SMSMSG=1222-mz-University-V1 in ARoad UP at 13:04:13 15/04/11
i need to read this file line by line and generate an output file like the one below
Quote:
123456789=2050-IBS-ICF-FurnishingSect-IV in BSC ANagar down at 18:27:56 08/04/2011
12345+23456+34567+45678+56789+67890=1453-CM-QTRS-IV5-G in solo Road II UP at 13:04:12 15/04/11
12345+32453+456789+145678+567890=1222-mz-University-V1 in ARoad UP at 13:04:13 15/04/11
i.e based on N value the number of MSISDNs will vary, if N=1 then the following MSISDN will be only one and the next line is SMSMSG. I need to take MSISDN1 value and its corresponding SMSMSG in the next line and form it like an equation MSISDN=SMSMSG

similar way if N=5 then the next five lines will have the MSISDNs 1 to 5 and i need to make them as one value by separating them by + ( Not adding) i.e MSISDN1+MSISDN2+MSISDN3+MSISDN4+MSISDN5=SMSMSG( the message in the next line)

i have started in the following way ...but logically not correct i hope...is there any better way to do this.

Code:
#!/bin/bash
# Shell script utility to read a file line line.
# Once line is read it can be process in processLine() function
# You can call script as follows, to read myfile.txt:
# ./readline myfile.txt
# Following example will read line from standard input device aka keyboard:
# ./readline

# User define Function (UDF)
processLine(){
  line="$@" # get all args
  #  just echo them, but you may need to customize it according to your need
  # for example, F1 will store first field of $line, see readline2 script
  # for more examples
  # F1=$(echo $line | awk '{ print $1 }')

 # echo $line
}
 
### Main script stars here ###
# Store file name
FILE=""
 
# Make sure we get file name as command line argument
# Else read it from standard input device
if [ "$1" == "" ]; then
   FILE="/dev/stdin"
else
   FILE="$1"
   # make sure file exist and readable
   if [ ! -f $FILE ]; then
        echo "$FILE : does not exists"
        exit 1
   elif [ ! -r $FILE ]; then
        echo "$FILE: can not read"
        exit 2
   fi
fi
# read $FILE using the file descriptors
 
# Set loop separator to end of line
BAKIFS=$IFS
IFS=$(echo -en "\n\b")
exec 3<&0
exec 0<"$FILE"
while read -r line
do
        # use $line variable to process line in processLine() function
       n=""
       n=$(echo $line | /usr/xpg4/bin/awk -F"=" -v Ncnt="N" '$1==Ncnt { print $2 } ')
       if [ "$n" != "" ]; then
       for (( c=1; c<=$n; c++ ))
       do
          ......here i have to read the next line and filter out MSISDNs and SMSMSG
        processLine $line
done
exec 0<&3
 
# restore $IFS which was used to determine what the field separators are
IFS=$BAKIFS
exit 0

pls help me...
# 2  
Old 04-17-2011
Try this

Code:
awk '/N=/ {
  val=substr($0,3);for(i=0;i<val;i++){
    pre="";if(i>0){pre="+"};getline;split($0, arr, "=");str=str pre arr[2];}
  getline;split($0, arr, "=");str=str"="arr[2];print str;str="";
}' file

regards,
Ahamed
This User Gave Thanks to ahamed101 For This Post:
# 3  
Old 04-17-2011
Hi,

yes its ok....thanks
# 4  
Old 04-18-2011
Hi,

for this input file its malfunctioning. why so?
Quote:
[IMS]
DH=95
SEC=1
[SEC1]
N=1
MSISDN1=278
SMSMSG=1415-3P-G in BSC jo Road II down at 17:01:14 18/04/2011
[IMS]
HEADERMSG=OMCR-MA-UP
DH=95
SEC=1
[SEC1]
N=1
MSISDN1=90278
SMSMSG=1415-3P-G in jo Road II UP at 17:13:14 18/04/11
the result is coming out like the one below.

Quote:
=1415-3P-G in BSC jo Road II down at 17:01:14 18/04/2011
=1415-3P-G in jo Road II UP at 17:13:14 18/04/11
what could be issue....
# 5  
Old 04-18-2011
Code:
awk -F= '/MSISDN/{s=(s=="")?$2:s"+"$2}/SMSMSG/{s=s"="$2;print s;s=""}' file

# 6  
Old 04-18-2011
Hi

now the output is
Quote:
278
=1415-3P-G in BSC jo Road II down at 17:01:14 18/04/2011
90278
=1415-3P-G in jo Road II UP at 17:13:14 18/04/11
i want the output as
Quote:
278=1415-3P-G in BSC jo Road II down at 17:01:14 18/04/2011
90278=1415-3P-G in jo Road II UP at 17:13:14 18/04/11
# 7  
Old 04-18-2011
one way,
Code:
awk '{ printf "%s", $0; getline; printf "%s\n", $0 }' inputfile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

With script bash, read file line per line starting at the end

Hello, I'm works on Ubuntu server My goal : I would like to read file line per line, but i want to started at the end of file. Currently, I use instructions : while read line; do COMMAND done < /var/log/apache2/access.log But, the first line, i don't want this. The file is long... (5 Replies)
Discussion started by: Fuziion
5 Replies

2. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

3. Shell Programming and Scripting

Bash script to read a file from particular line till required line and process

Hi All, Am trying to write wrapper shell/bash script on a utility tool for which i need to pass 2 files as arugment to execute utility tool. Wraper script am trying is to do with above metion 2 files. utility tool accepts : a. userinfo file : which contains username b. item file : which... (2 Replies)
Discussion started by: Optimus81
2 Replies

4. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

5. Shell Programming and Scripting

Shell script to read multiple options from file, line by line

Hi all I have spent half a day trying to create a shell script which reads a configuration file on a line by line basis. The idea of the file is that each will contain server information, such as IP address and various port numbers. The line could also be blank (The file is user created). Here... (1 Reply)
Discussion started by: haggismn
1 Replies

6. Shell Programming and Scripting

How to read a file line by line and store it in a variable to execute a program ?

Hello, I am quite new in shell scripting and I would like to write a little scritp to run a program on some parameters files. all my parameters files are in the same directory, so pick them up with ls *.para >>dirafter that I have a dir file like that: param1.para param2.para etc... I... (2 Replies)
Discussion started by: shadok
2 Replies

7. Shell Programming and Scripting

Shell script to read a text file line by line & process it...

Hi , I am trying to write an shell, which reads a text file (from a location) having a list of numbers of strictly 5 digits only ex: 33144 Now my script will check : 1) that each entry is only 5 digits & numeric only, no alphabets, & its not empty. 2)then it executes a shell script called... (8 Replies)
Discussion started by: new_to_shell
8 Replies

8. Shell Programming and Scripting

bash: read file line by line (lines have '\0') - not full line has read???

I am using the while-loop to read a file. The file has lines with null-terminated strings (words, actually.) What I have by that reading - just a first word up to '\0'! I need to have whole string up to 'new line' - (LF, 10#10, 16#A) What I am doing wrong? #make file 'grb' with... (6 Replies)
Discussion started by: alex_5161
6 Replies

9. Shell Programming and Scripting

shell script to read a line in gps receiver log file and append that line to new file

Hi, I have gps receiver log..its giving readings .like below Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GPSD,R=1 $GPGSV,3,1,11,08,16,328,40,11,36,127,00,28,33,283,39,20,11,165,00*71... (3 Replies)
Discussion started by: gudivada213
3 Replies

10. Shell Programming and Scripting

How to read/process a .gz file, one line at a time?

Hello I'm stuck trying to solve this KSH issue and I'm hoping someone out there can offer some suggestions. I want to read lots of large .gz files one line at a time in order to compare its Error entries with a list of known errors. I can't simply do "foreach ERROR do gzcat *.gz |grep... (2 Replies)
Discussion started by: dayscripter
2 Replies
Login or Register to Ask a Question