Awk: Print out overlapping chunks of file - rows 0-20,10-30,20-40 etc.


 
Thread Tools Search this Thread
# 1  
Awk: Print out overlapping chunks of file - rows 0-20,10-30,20-40 etc.

First time poster, but the forum has saved my bacon more times than... Lots.

Anyway, I have a text file, and wanted to use Awk (or any other sensible program) to print out overlapping sections, or arbitrary length. To describe by example, for file

1
2
3
4
5
etc...

I want the out put to be lines 1-20, then lines 10-30, then 20-40 and so on. I can't find a sensible way to make awk "go back on itself" however.

I could do the whole thing with a horrible loop, running the file through awk separately for each one, and changing a variable, but the files I am operating on ate large and there are lots of them, so ideally I would avoid slower methods.

Any help appreciated!
# 2  
Hi matfald,
welcome to unix.com!

You may try something like this:

Code:
awk '{ nr[NR] = $0 }
!(NR % s) && NR >= o {
  for (i = NR - o ? NR - o : 1 ; i <= NR; i++)
    print nr[i]
  }' o=20 s=10 infile

This User Gave Thanks to radoulov For This Post:
# 3  
Looks very much like that works perfectly. Thanks very much!

I was surprised to see you can tell awk stuff outside the " ' "s but I guess you learn something new every day!
# 4  
Yes,
see the GNU awk manual.
 

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #76
Difficulty: Easy
Richard Stallman's GNU project was also started in response to AT&T's increasingly restrictive Unix software licensing terms.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk- Indexing a list of numbers in file2 to print certain rows in file1

Hi Does anyone know of an efficient way to index a column of data in file2 to print the coresponding row in file1 which corresponds to the data in file2 AND 30 rows preceding and after the row in file1. For example suppose you have a list of numbers in file2 (single column) as follows:... (6 Replies)
Discussion started by: Geneanalyst
6 Replies

2. Shell Programming and Scripting

Deleting duplicated chunks in a file using awk/sed

Hi all, I'd always appreciate all helps from this site. I would like to delete duplicated chunks of strings on the same row(?). One chunk is comprised of four lines such as: path name starting point ending point voltage number I would like to delete duplicated chunks on the same... (5 Replies)
Discussion started by: jypark22
5 Replies

3. Shell Programming and Scripting

Print rows with value zero or less than zero using awk

I tried this. It working fine for small tables. But it is giving values greater than zero in a big file with 8556 columns. Does any one know why ? awk 'FNR == 1{print;next}{for(i=8556;i<=NF;i++) if($i <= 0){print;next}}' input (5 Replies)
Discussion started by: quincyjones
5 Replies

4. Shell Programming and Scripting

Identify the overlapping and non overlapping regions

file1 chr pos1 pos2 pos3 pos4 1)chr1 1000 2000 3000 4000 2)chr1 1380 1480 6800 7800 3)chr1 6700 7700 1200 2200 4)chr2 8500 9500 5670 6670 file2 chr pos1 pos2 pos3 pos4 1)chr2 8500 9500 5000 6000 2)chr1 6700 7700 1200 2200 3)chr1 1380 1480 6700 7700 4)chr1 1000 2000 4900 5900 I... (2 Replies)
Discussion started by: data_miner
2 Replies

5. Shell Programming and Scripting

Print the overlapping entries in 2 files to separate file

I have two files that contain overlapping positions. i want to put them together each overlapping entries in both files in to a new file (the entries of first file first and the entries of second file next) followed by blank line then next overlapping entries and so on. input1 chr1 22 ... (10 Replies)
Discussion started by: raj_k
10 Replies

6. Shell Programming and Scripting

awk command to print only selected rows in a particular column specified by column name

Dear All, I have a data file input.csv like below. (Only five column shown here for example.) Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 3,2,4,5,6 5,3,5,5,6 From this I want the below output Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 where the second column... (4 Replies)
Discussion started by: ks_reddy
4 Replies

7. Shell Programming and Scripting

awk for splitting file in constant chunks

Hi gurus, I wanted to split main file in 20 files with 2500 lines in each file. My main file conatins total 2500*20 lines. Following awk I made, but it is breaking with error. awk '{ for (i = 1; i <= 20; i++) { starts=2500*$i-1; ends=2500*$i; NR>=starts && NR<=ends {f=My$i".txt"; print >> f;... (10 Replies)
Discussion started by: mukesh.lalwani
10 Replies

8. Shell Programming and Scripting

remove chunks of text from file

All, So, I have an ldif file that contains about 6500 users worth of data. Some users have a block of text I'd like to remove, while some don't. Example (block of text in question is the block starting with "authAuthority: ;Kerberosv5"): User with text block: # username, users,... (7 Replies)
Discussion started by: staze
7 Replies

9. Shell Programming and Scripting

read file and print additional rows till current year

Hi all i have a file like 2006,1,2 2007,2,3 2008,3,4 I will read this and my output should be like 2006,1,2 2007,1,2 2008,1,2 2007,2,3 2008,2,3 2008,3,4 Giving the explanation, we will read the first line of the file and if the year any other than current year, we will print as many... (1 Reply)
Discussion started by: vasuarjula
1 Replies

10. Shell Programming and Scripting

print selected rows with awk

Hi everybody: Could anybody tell me how I can print from a file a selected rows with awk. In my case I only want print in another file all the rows from NR=8 to NR=2459 and the increment each 8 times. I tried to this: awk '{for (i=8; i=2459; i+=8); NR==i}' file1 > file2 But doesn't... (6 Replies)
Discussion started by: tonet
6 Replies

Featured Tech Videos