Read file line by line and process the line to generate another file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read file line by line and process the line to generate another file
# 8  
Old 04-18-2011
Hello, aemunathan:

Since your initial attempt is a shell script, I thought you'd find the following simple sh script instructive.

Code:
#!/bin/sh

while IFS== read -r key value; do
    case $key in
        MSISDN*) s=$s+$value ;;
         SMSMSG) printf '%s=%s\n' "${s#+}" "$value"
                 s= ;;
    esac
done

It reads from stdin, though I'm sure you can modify it to handle filename arguments. If there's anything about it that you don't understand, just ask.

Regards,
Alister
# 9  
Old 04-18-2011
hi,

i hope i understood ur example. and i tried to add the input and output file as below, but it doesn't work.

Code:
#!/bin/sh
while IFS== read -r key value; do
    case $key in
        MSISDN*) s=$s+$value ;;
         SMSMSG) printf '%s=%s\n' "${s#+}" "$value"
                 s= ;;
    esac
done < /aemu/smsT.ini > /aemu/SMSfileGen.txt

if i use -r option am getting
Quote:
./SMSfileGen: -r: is not an identifier
after removing -r am getting
Quote:
./SMSfileGen: bad substitution
Pls help me
# 10  
Old 04-18-2011
Which shell and operating system are you using? I tested with bash and it worked perfectly on the sample data you provided in a couple of earlier posts. Perhaps your /bin/sh isn't up to the task. Although the script should be compatible with a reasonably posix-compliant sh, perhaps you could try changing the first line to #!/bin/bash, as in your original script.
This User Gave Thanks to alister For This Post:
# 11  
Old 04-19-2011
Hi,

yes i tried already with ksh. its running but the output is the same...one thing i observed, if i do a more input file name (smsT.ini) and copy the content of the file and create a new file name(smsT.txt) and paste it there and use that file it gives the required output...File size also getting decreased from the original file size value since while pasting all spaces are replaced to single space.

How to come out of this magical moments

tried as below but no hope

Code:
#!/bin/ksh
#sed -e "s/[ <tab>]*//g" /aemu/smsT.ini > /aemu/smsT.txt
while read line
do
echo $line | tr -s " "
done < /aemu/smsT.ini > /aemu/smsT.txt
while IFS== read -r key value; do
    case $key in
        MSISDN*) s=$s+$value ;;
         SMSMSG) printf '%s=%s\n' "${s#+}" "$value"
                 s= ;;
    esac
done< /aemu/smsT.txt > /aemu/SMSfileGen.txt

---------- Post updated 04-19-11 at 03:58 PM ---------- Previous update was 04-18-11 at 11:27 PM ----------

Hi,

Nothing is helping the cause...
pls tell me to read this output file so that i can assign the first row value to one variable and second row to another variable and repeat the process. Am going to use only two variables.
Quote:
278
=1415-3P-G in BSC jo Road II down at 17:01:14 18/04/2011
90278
=1415-3P-G in jo Road II UP at 17:13:14 18/04/11
assign like the one below.

Quote:
ms=278
sms=1415-3P-G in BSC jo Road II down at 17:01:14 18/04/2011
do something here...with these two variables
and loop through to assign the other two data in the same variable
ms=90278
sms=1415-3P-G in jo Road II UP at 17:13:14 18/04/11
do something here...with these two variables
---------- Post updated at 06:09 PM ---------- Previous update was at 03:58 PM ----------

Hi,

what is the mistake in the below code ?
Code:
num=2
count=$num
while read line
do
if [ "$count" != "1" ]; then
strMSID=$(echo $line)
count=`expr $count - 1`
else
lines=`echo $line | tr -s " "`
smstext=`echo $lines | sed -e 's/:/-/g'`
strsms=`echo $smstext | sed -e 's/ /+/g'`
echo $strsms
count=$num
echo "GET  /cgi-bin/sendsms?username=an&password=ad&to=8445&text=$strsms  HTTP/1.1"
(sleep 4; echo "GET  /cgi-bin/sendsms?username=an&password=ad&to=8445&text=$strsms  HTTP/1.1"; echo ""; sl
eep 5) | telnet 10.10.6.6 9080
fi
done< /home/SMS/SMSfile.txt

it give echoes out like
Quote:
1629-ParamanandaMdlySt-IV5-G+in+BSC+hbr+down+at+17-32-29+19/04/2011
GET /cgi-bin/sendsms?username=an&password=ad&to=8445&text=1629-ParamanandaMdlySt-IV5-G+in+BSC+hbr+dow HTTP/1.1-29+19/04/2011
Pls help me out..

Last edited by aemunathan; 04-18-2011 at 03:22 PM..
# 12  
Old 04-19-2011
I think your file must have some control characters in it that are not visible and also not copied into your example text.
That would explain why the code works when I use it on the examples given. Perhaps try to remove the control characters and then try the awk command again?
This User Gave Thanks to kato For This Post:
# 13  
Old 04-19-2011
Hi

Its really great finding. Yes there was some hidden characters. ^M

and its removed and working fine.....

Quote:
tr -d "\015" < hello.c > hello.unix.c
# 14  
Old 04-19-2011
Its working for me though

Code:
/user/ahamed $ awk '/N=/ {
  val=substr($0,3);for(i=0;i<val;i++){
    pre="";if(i>0){pre="+"};getline;split($0, arr, "=");str=str pre arr[2];}
>   val=substr($0,3);for(i=0;i<val;i++){
>     pre="";if(i>0){pre="+"};getline;split($0, arr, "=");str=str pre arr[2];}
>   getline;split($0, arr, "=");str=str"="arr[2];print str;str="";
}' file> }' file
278=1415-3P-G in BSC jo Road II down at 17:01:14 18/04/2011
90278=1415-3P-G in jo Road II UP at 17:13:14 18/04/11

regards,
Ahamed

---------- Post updated at 09:04 AM ---------- Previous update was at 09:02 AM ----------

Didn't realize the thread was 2 page long and issue got resolved.

regards,
Ahamed
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

With script bash, read file line per line starting at the end

Hello, I'm works on Ubuntu server My goal : I would like to read file line per line, but i want to started at the end of file. Currently, I use instructions : while read line; do COMMAND done < /var/log/apache2/access.log But, the first line, i don't want this. The file is long... (5 Replies)
Discussion started by: Fuziion
5 Replies

2. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

3. Shell Programming and Scripting

Bash script to read a file from particular line till required line and process

Hi All, Am trying to write wrapper shell/bash script on a utility tool for which i need to pass 2 files as arugment to execute utility tool. Wraper script am trying is to do with above metion 2 files. utility tool accepts : a. userinfo file : which contains username b. item file : which... (2 Replies)
Discussion started by: Optimus81
2 Replies

4. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

5. Shell Programming and Scripting

Shell script to read multiple options from file, line by line

Hi all I have spent half a day trying to create a shell script which reads a configuration file on a line by line basis. The idea of the file is that each will contain server information, such as IP address and various port numbers. The line could also be blank (The file is user created). Here... (1 Reply)
Discussion started by: haggismn
1 Replies

6. Shell Programming and Scripting

How to read a file line by line and store it in a variable to execute a program ?

Hello, I am quite new in shell scripting and I would like to write a little scritp to run a program on some parameters files. all my parameters files are in the same directory, so pick them up with ls *.para >>dirafter that I have a dir file like that: param1.para param2.para etc... I... (2 Replies)
Discussion started by: shadok
2 Replies

7. Shell Programming and Scripting

Shell script to read a text file line by line & process it...

Hi , I am trying to write an shell, which reads a text file (from a location) having a list of numbers of strictly 5 digits only ex: 33144 Now my script will check : 1) that each entry is only 5 digits & numeric only, no alphabets, & its not empty. 2)then it executes a shell script called... (8 Replies)
Discussion started by: new_to_shell
8 Replies

8. Shell Programming and Scripting

bash: read file line by line (lines have '\0') - not full line has read???

I am using the while-loop to read a file. The file has lines with null-terminated strings (words, actually.) What I have by that reading - just a first word up to '\0'! I need to have whole string up to 'new line' - (LF, 10#10, 16#A) What I am doing wrong? #make file 'grb' with... (6 Replies)
Discussion started by: alex_5161
6 Replies

9. Shell Programming and Scripting

shell script to read a line in gps receiver log file and append that line to new file

Hi, I have gps receiver log..its giving readings .like below Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GPSD,R=1 $GPGSV,3,1,11,08,16,328,40,11,36,127,00,28,33,283,39,20,11,165,00*71... (3 Replies)
Discussion started by: gudivada213
3 Replies

10. Shell Programming and Scripting

How to read/process a .gz file, one line at a time?

Hello I'm stuck trying to solve this KSH issue and I'm hoping someone out there can offer some suggestions. I want to read lots of large .gz files one line at a time in order to compare its Error entries with a list of known errors. I can't simply do "foreach ERROR do gzcat *.gz |grep... (2 Replies)
Discussion started by: dayscripter
2 Replies
Login or Register to Ask a Question