Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Automated script to take 1000 records from the file every week.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Automated script to take 1000 records from the file every week.
# 1  
Old 09-30-2011
Automated script to take 1000 records from the file every week.
I have a file having n number of records .i want first 1000 records from the file and store in temporary file to process on sunday.

I want script should should automatically take the next 1000 records for processing on next sunday.

can we do it using head and tail

head -1000 | tail -1000

then next week
head -2000 | tail -1000 like wise ..

but i have to change it from 1000 to 2000 till it reach n..so that it will read every record from the file.

Please suggest.
# 2  
Old 09-30-2011
you can create a file where you can store the multiplying factor ... like 2 for day 2 ...3 for day 3 .... which u can read in the begininning and then update at the end ... and then reset when it reaches n
# 3  
Old 09-30-2011
1) simply grab first 1000 records and save last line number of grabbed record, next day grab last line number + 1000 records and save the last line number again...

2) rotate your main record file each night and rename it into date ( YYYY-MM-DD )
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automated File Transfer Script

We are receiving data feed files in SFTP location daily. so the folder structure in SFTP location is abc/def/studyname_1/outbound/zipped files So we will be getting different studies and for each study a folder is created abc/def/studyname_2/outbound/zipped files , ... (7 Replies)
Discussion started by: Sidhant
7 Replies

2. Shell Programming and Scripting

Shell script to filter records in a zip file that contains matching columns from another file

Not sure if this is the correct forum for this question. I have two files. file1.zip, file2 Input: file1.zip col1, col2 , col3 a , b , 0:0:0:0:0:c436:9346:d40b x, y, 0:0:0:0:0:880:39f9:c9a7 m, n , 0:0:0:0:0:80c7:9161:fe00 file2.txt col1 c4:36:93:46:d4:0b... (1 Reply)
Discussion started by: anil.v
1 Replies

3. Shell Programming and Scripting

Deleting local server file from automated FTP script

Hi, I want to delete a file on the local server, while connected to remote server through FTP. I am using the below code for this $FTP_CMD -v -n $HOST <<*! >> $LOGFILE 2>&1 user $USER $PASSWORD cd $DIR ... (11 Replies)
Discussion started by: jhilmil
11 Replies

4. Shell Programming and Scripting

Need Help:Shell Script for Solaris to change the dates in a file by one week

I have to increase the date by one week in an input when script is executed in solaris. I was able to acheive this using ksh script that is working in Linux enivironment, when i execute the same script in Solaris i am getting below error: /var/tmp\n\r-> ./script.ksh date: illegal option -- d... (3 Replies)
Discussion started by: sriramanaramoju
3 Replies

5. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies

6. Shell Programming and Scripting

Script to find the string contain in which file 1000 files.

Hi Greetings i have 1000 files (xmlfiles) and i need to find which file contain the string over 1000 file all file end with txt i tried with grep -c "string" *.txt i am getting an error -bash: /bin/egrep: Argument list too long i have put an script like below #!/bin/bash for line... (9 Replies)
Discussion started by: venikathir
9 Replies

7. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

8. UNIX for Advanced & Expert Users

Automated SCP script passing password to preserve source file timestamp

Hi My requirement is i want to copy files from remote server to the local server and also i need to preserve the timestamp of the remote file. By using scp -p , it is working fine in the interactive call but it is not preserving he file timestamp when i use it in the non interactive scp call... (1 Reply)
Discussion started by: skumar75
1 Replies

9. Shell Programming and Scripting

awk script required for finding records in 1 file with corresponding another file.

Hi, I have a .txt file (uniqfields.txt) with 3 fields separated by " | " (pipe symbol). This file contains unique values with respect to all these 3 fields taken together. There are about 40,000 SORTED records (rows) in this file. Sample records are given below. 1TVAO|OVEPT|VO... (2 Replies)
Discussion started by: RRVARMA
2 Replies

10. Shell Programming and Scripting

Count No of Records in File without counting Header and Trailer Records

I have a flat file and need to count no of records in the file less the header and the trailer record. I would appreciate any and all asistance Thanks Hadi Lalani (2 Replies)
Discussion started by: guiguy
2 Replies
Login or Register to Ask a Question