File count from where it left??


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File count from where it left??
# 1  
Old 04-25-2013
File count from where it left??

i have this code
Code:
Code:
num=1
dat10=`date "+%m/%d/%Y"  -d "+10 days"`
dat=`date "+%m/%d/%Y"`
set -x
grep "text=.&" /root/Softwares/apache-tomcat-5.5.30/logs/catalina.out|awk -F"from=" '{print $2}'|awk -F"opid" '{print $1}'|sort|uniq > pnos

while read line
do
psql -U poss -d emsver -c "insert into ent(phone_number) values ('$line');"
grep "text=.&" /root/Softwares/apache-tomcat-5.5.30/logs/catalina.out|grep $line|awk -F"text=" '{print $2}'|awk -F "&from" '{print $1}' > services
while read sline
do
psql -U poss -d emsver -c "insert into uiotion(phone_number,service_name,start_date,end_date,last_content_deliver_time) values ('$line','$sline','$dat','$dat10',current_date);"
done < services
num=`wc -l pnos|awk '{print $2}'`
done < pnos

pnos file contains

Code:
Code:
2132421
3124321421
4214242424
4242421421

so once the code iterates the pnos times i want it to iterate next time from where it left for example pnos have 5 lines in it, nect time wn it is iterating it shul iterate from 6th line it shul not strt fm the 1st line.....
# 2  
Old 04-25-2013
I can't really test this, but I think it will work, if I understand the question:
Code:
pnos_save_file=$HOME/pnos_save_file.txt
num=1
dat10=`date "+%m/%d/%Y"  -d "+10 days"`
dat=`date "+%m/%d/%Y"`
set -x
grep "text=.&" /root/Softwares/apache-tomcat-5.5.30/logs/catalina.out|awk -F"from=" '{print $2}'|awk -F"opid" '{print $1}'|sort|uniq > pnos

if [ ! -f "$pnos_save_file" ]; then
  echo 1 > "$pnos_save_file"
fi

old_start_line=`cat $pnos_save_file`

tail -n +$old_start_line pnos > pnos.tail

while read line
  do
  psql -U poss -d emsver -c "insert into ent(phone_number) values ('$line');"
  grep "text=.&" /root/Softwares/apache-tomcat-5.5.30/logs/catalina.out|grep $line|awk -F"text=" '{print $2}'|awk -F "&from" '{print $1}' > services
  while read sline
    do
    psql -U poss -d emsver -c "insert into uiotion(phone_number,service_name,start_date,end_date,last_content_deliver_time) values ('$line','$sline','$dat','$dat10',current_date);"
    done < services
  num=`wc -l pnos|awk '{print $2}'`
done < pnos.tail

new_start_line=`cat pnos | wc -l`
new_start_line=`expr $new_start_line + 1`
echo $new_start_line > "$pnos_save_file"

This User Gave Thanks to hanson44 For This Post:
# 3  
Old 04-25-2013
thanks hanson Smilie it worked Smilie
# 4  
Old 04-25-2013
Great! I'm really glad to hear that. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Ftp with bash, append file where left off

I'm working on a bash script to finish uploading a file. I need a way to get $filesize so that "restart $filesize" will work. Here is my script: ftp -n -v <<END_SCRIPT open ftp.$domain user $user@$domain $password size $file restart $filesize put $file quit END_SCRIPTWayne Sallee... (9 Replies)
Discussion started by: WayneSallee
9 Replies

2. Shell Programming and Scripting

Speed : awk command to count the occurrences of fields from one file present in the other file

Hi, file1.txt AAA BBB CCC DDD file2.txt abc|AAA|AAAabcbcs|fnwufnq bca|nwruqf|AAA|fwfwwefwef fmimwe|BBB|fnqwufw|wufbqw wcdbi|CCC|wefnwin|wfwwf DDD|wabvfav|wqef|fwbwqfwfe i need the count of rows of file1.txt present in the file2.txt required output: AAA 2 (10 Replies)
Discussion started by: mdkm
10 Replies

3. Solaris

File system full - not removed: No space left on device

Does anyone have any advise on trying to clean up a full filesystem? I can't rm any files because of the follow: not removed: No space left on device Any help would be very much appreciated. (10 Replies)
Discussion started by: craigsky
10 Replies

4. Shell Programming and Scripting

FASTEN count line of dat file and compare with the CTRL file

Hi All, I thinking on how to accelerate the speed on calculate the dat file against the number of records CTRL file. There are about 300 to 400 folder directories that contains both DAT and CTL files. DAT contain all the flat files records CTL is the reference check file for the... (3 Replies)
Discussion started by: ckwan
3 Replies

5. Shell Programming and Scripting

Left Align of Text File

Input: hiddenhausen 99.60 y 1.05 2.50 -550 -110 1:25:200 herbstein 99.021 n 1.05 2.50 -550 -110 1:25:200 bangalore 98.82 y 1.05 2.50 -550 -110 1:25:200 golm 98.8 y 1.05 2.50 -550 -110 1:25:200 para 98.82 n 1.05 2.50 -550 -110 1:25:200 bogen 98.61 n 1.05 2.50 -550 -110 1:25:200 saintandre... (5 Replies)
Discussion started by: asavaliya
5 Replies

6. Shell Programming and Scripting

Count the delimeter from a file and delete the row if delimeter count doesnt match.

I have a file containing about 5 million rows, in the file there are some records which has extra delimiter at random position. (we dont know the positions), now we have to Count the delimeter from each row and if the count of delimeter is not matching then I want to delete those rows from the... (5 Replies)
Discussion started by: Akumar1
5 Replies

7. Shell Programming and Scripting

Parse file from remote server to calculate count of string existence in that file

Hi I need to parse the file of same name which exist on different servers and calculate the count of string existed in both files. Say a file abc.log exist on 2 servers. I want to search for string "test" on both files and calculate the total count of search string's existence. For... (6 Replies)
Discussion started by: poweroflinux
6 Replies

8. Shell Programming and Scripting

Howto process log file from where it left

Hi all I am doing some matching apache access log processing.Now i want a some way that i can start log search/process from where it left.Can any one give me ideads. (1 Reply)
Discussion started by: aliahsan81
1 Replies

9. Shell Programming and Scripting

Getting Sum, Count and Distinct Count of a file

Hi all this is a UNIX question. I have a large flat file with millions of records. col1|col2|col3 1|a|b 2|c|d 3|e|f 3|g|h footer**** I am supposed to calculate the sum of col1 1+2+3+3=9, count of col1 1,2,3,3=4, and distinct count of col1 1,2,3=c3 I would like it if you avoid... (4 Replies)
Discussion started by: singhabhijit
4 Replies

10. UNIX for Dummies Questions & Answers

How to count the record count in an EBCDIC file.

How do I get the record count in an EBCDIC file on a Linux Box. :confused: (1 Reply)
Discussion started by: oracle8
1 Replies
Login or Register to Ask a Question