Help with awk script to get missing numbers in column 1


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with awk script to get missing numbers in column 1
# 1  
Old 11-05-2013
Help with awk script to get missing numbers in column 1

Hello to all,

I have show below a file separated by commas. In first column has numbers where the last number is 13.
Code:
1,4
2,6
3,7
5,2
6,5
7,5
8,65
9,10
11,78
13,2

What I want to know is which numbers are missing from 1 to 13 (in this case 13 is last number in column 1). My real file
has more than 5 million lines.

The output expected is as below:
Code:
4
10
12

Thanks in advance for any help.
# 2  
Old 11-05-2013
Try

Code:
$ awk -F, '{while(++x<$1)print x}' file
4
10
12


Last edited by Akshay Hegde; 11-05-2013 at 01:56 PM.. Reason: Bold
# 3  
Old 11-05-2013
More awk... More awk please...

Code:
awk -F, '$1>a+1{print NR}{a=$1}' file
4
9
10

# 4  
Old 11-05-2013
Quote:
Originally Posted by in2nix4life
More awk... More awk please...

Code:
awk -F, '$1>a+1{print NR}{a=$1}' file
4
9
10

9 is there in file
# 5  
Old 11-05-2013
Hello Akshay Hegde,

Thank you for your help.

It works with sample file but is not working with real file.

Real file has 5,440,177 lines and the last number in column 1 is 5440255. So, substracting both there are 78 numbers that are missing in column 1.
But trying your script I get more than 33 million of lines and I stopped since it seems enters in an infinite loop.

Is there a way to preload an array from 1 to N (N=13 in this case, in real file N=5440255)? in order to compare array which values from column 1 are not in array?

Thanks again
# 6  
Old 11-05-2013
Oops, thanks Akshay. Here's the remix:

Code:
awk '($1!=a+1){print a+1}{a=$1}' file
4
10
12

# 7  
Old 11-05-2013
Quote:
Originally Posted by in2nix4life
Code:
awk '($1!=a+1){print a+1}{a=$1}' file
4
10
12

Hello in2nix4life,

Thanks for the help.

I've tried your script, the expected number of missing values in real file are 78 but is printing 108 numbers.

Is there a way to compare column 1 with a preloaded array that contains elements from 1 to N? in order to print values of array that are not present in column1?

Thanks again for the help.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script to append suffix to column when column has duplicated values

Please help me to get required output for both scenario 1 and scenario 2 and need separate code for both scenario 1 and scenario 2 Scenario 1 i need to do below changes only when column1 is CR and column3 has duplicates rows/values. This inputfile can contain 100 of this duplicated rows of... (1 Reply)
Discussion started by: as7951
1 Replies

2. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000... (4 Replies)
Discussion started by: malandisa
4 Replies

3. UNIX for Dummies Questions & Answers

How to combine and insert missing consecutive numbers - awk or script?

Hi all, I have two (2) sets of files that are based on some snapshots of database that I want to merge and insert any missing sequential number. Below are example representation of these files: file1: DATE TIME COL1 COL2 COL3 COL4 ID 01/10/2013 0800 100 ... (3 Replies)
Discussion started by: newbie_01
3 Replies

4. Shell Programming and Scripting

AWK script to create max value of 3rd column, grouping by first column

Hi, I need an awk script (or whatever shell-construct) that would take data like below and get the max value of 3 column, when grouping by the 1st column. clientname,day-of-month,max-users ----------------------------------- client1,20120610,5 client2,20120610,2 client3,20120610,7... (3 Replies)
Discussion started by: ckmehta
3 Replies

5. Shell Programming and Scripting

Fill missing numbers in second column with zeros

Hi All, I have 100 files with names like this: 1.dat, 2.dat, 3.dat until 100.dat. My dat files look like this: 42323 0 438939 1 434 0 0.9383 3434 120.23 3 234 As you can see in the second column, some numbers are missing. I want to fill those missing places with 0's in all... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

6. Shell Programming and Scripting

how to include the missing column in the original file using awk

Hi Experts, The content of the raw file: date,nomsgsent,nomsgnotdeliver,nomsgdelay 201003251000,1000,1,2 201003251000,900,0,0 201003251000,1450,0,0 201003251000,1230,0,0 However, sometimes, the column will missing in the raw files: e.g. date,nomsgsent,nomsgdelay... (8 Replies)
Discussion started by: natalie23
8 Replies

7. Shell Programming and Scripting

AWK processing -numbers to another column

Hi Guys, I'm trying to clean up my home logger file and can't seem to work this out. Here is my data: 10-19-2009 08:39 00.2 00.0 00.7 01.1 49.1 0.0 11.9 270.1 -49.1 220.9 10-19-2009 08:40 00.2 00.0 00.7 00.7 49.1 0.0 171.9 171.9 49.1 220.9 10-19-2009 08:41 00.1 00.0 00.7 00.8 24.5 0.0... (2 Replies)
Discussion started by: BeJay
2 Replies

8. Shell Programming and Scripting

trying to make an AWK code for ordering numbers in a column from least to highest

Hi all, I have a large column of numbers like 5.6789 2.4578 9.4678 13.5673 1.6589 ..... I am trying to make an awk code so that awk can easily go through the column and arrange the numbers from least to highest like 1.6589 2.4578 5.6789 ....... can anybody suggest, how can I do... (5 Replies)
Discussion started by: ananyob
5 Replies

9. Shell Programming and Scripting

ksh/awk help - output missing numbers

Here is what I am trying to do: I have a list of numbers that I pulled from an awk command in a column like so: 1 3 4 7 8 I want to find which numbers in the list are missing out of a range. So let's say I want to find out from the list above which numbers are missing from the... (6 Replies)
Discussion started by: afavis
6 Replies
Login or Register to Ask a Question