Shell Programming and Scripting

"p" is just a variable I am using to control the execution of the code within the curly braces. If p=0, it will not enter, if p=1 it will enter.
So, we need to execute the code in the first iteration and hence "p" is initialized to 1 in the BEGIN block. Once it enters the block, we will turn it off i.e. p=0.
When the range limit is reached, we need to re-initialize end, increment n and decrement howmanytimes and so I set p=1 within the if loop.

HTH
--ahamed

Thanks ahamed i get it now.

Just curious, is it possible to play around with NR values as in the algorithm I had mentioned? or is it not possible with AWK.

Anything is possible If you can clarify what you want to do with NR, may be we can help.

--ahamed

Why not split the whole file in one go? With this line of awk, you split the whole file into files with filenames like 00000000-00001000.exp for the record range below 1000, 00001000-00002000.exp for the next range and so on.

My input:

Code:

$ cat input.txt 
1	10
2	22
3	33
4	44
5	55
6	66
7	77
8	88
9	99
10	110
11	111
12	112
13	113
14	144
15	155
16	166

Command:

Code:

$ awk 'BEGIN{per=5} {step=int($1/per);fname=sprintf("%09d-%09d.exp",step*per,(step*per)+per);print >>fname}' input.txt

$ grep '' *exp

Code:

000000000-000000005.exp:1	10
000000000-000000005.exp:2	22
000000000-000000005.exp:3	33
000000000-000000005.exp:4	44
000000005-000000010.exp:5	55
000000005-000000010.exp:6	66
000000005-000000010.exp:7	77
000000005-000000010.exp:8	88
000000005-000000010.exp:9	99
000000010-000000015.exp:10	110
000000010-000000015.exp:11	111
000000010-000000015.exp:12	112
000000010-000000015.exp:13	113
000000010-000000015.exp:14	144
000000015-000000020.exp:15	155
000000015-000000020.exp:16	166

I used a per (period) of 5 since my file is very small. Change this value to 1000 for your application.

If your file grows from time to time you really need a way to start exporting where you left off the last time. That is not difficult to do.

Greetings/Groeten,
Eric

---------- Post updated at 07:03 PM ---------- Previous update was at 06:46 PM ----------

While composing my reply I left my PC for a while and did not see there was already an answer given in the meantime. Sorry, Ahamed! I hope I didn't offend you.

Still, I chose a different approach, so maybe it has some use for anyone anyway.

Greetings,
Eric

@eric ...

the op -- from what i understand -- needed to be able to grab chunks of the original file into manageable chunks therefore the question ...

however ...

if your solution is not needed here, it may be more appropriate to another one or it may give the op another option that he/she has not yet thought of ... so never apologize about offering another solution ... we are all here to learn ...

I'm new on this forum (my second day!), but this was already the second time I gave a solution to a problem Ahamed already solved. I thought it was time to point out I was not competing or anything.

On the other hand: maybe I do want to compete, but not where it comes to producing the most brilliant solution (I would definately loose that battle...), but by providing a solution with the best explanation to give the OP knowledge to build upon instead of mere awe.

Greetings,
Eric

@edehont- Thanks I will try it out soon, I am just looking at it.

@ahamed- What I had planned for was something like this-
1. get the range
2. store the values within the range to a file, save the last matching line number
3. get new range, and start looking from the stored line number.
4.repeat over the file

Thanks again!