Sponsored Content
Top Forums Shell Programming and Scripting Help with awk script to get missing numbers in column 1 Post 302871103 by Ophiuchus on Tuesday 5th of November 2013 01:25:06 PM
Old 11-05-2013
Hello Akshay Hegde,

Thank you for your help.

It works with sample file but is not working with real file.

Real file has 5,440,177 lines and the last number in column 1 is 5440255. So, substracting both there are 78 numbers that are missing in column 1.
But trying your script I get more than 33 million of lines and I stopped since it seems enters in an infinite loop.

Is there a way to preload an array from 1 to N (N=13 in this case, in real file N=5440255)? in order to compare array which values from column 1 are not in array?

Thanks again
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh/awk help - output missing numbers

Here is what I am trying to do: I have a list of numbers that I pulled from an awk command in a column like so: 1 3 4 7 8 I want to find which numbers in the list are missing out of a range. So let's say I want to find out from the list above which numbers are missing from the... (6 Replies)
Discussion started by: afavis
6 Replies

2. Shell Programming and Scripting

trying to make an AWK code for ordering numbers in a column from least to highest

Hi all, I have a large column of numbers like 5.6789 2.4578 9.4678 13.5673 1.6589 ..... I am trying to make an awk code so that awk can easily go through the column and arrange the numbers from least to highest like 1.6589 2.4578 5.6789 ....... can anybody suggest, how can I do... (5 Replies)
Discussion started by: ananyob
5 Replies

3. Shell Programming and Scripting

AWK processing -numbers to another column

Hi Guys, I'm trying to clean up my home logger file and can't seem to work this out. Here is my data: 10-19-2009 08:39 00.2 00.0 00.7 01.1 49.1 0.0 11.9 270.1 -49.1 220.9 10-19-2009 08:40 00.2 00.0 00.7 00.7 49.1 0.0 171.9 171.9 49.1 220.9 10-19-2009 08:41 00.1 00.0 00.7 00.8 24.5 0.0... (2 Replies)
Discussion started by: BeJay
2 Replies

4. Shell Programming and Scripting

how to include the missing column in the original file using awk

Hi Experts, The content of the raw file: date,nomsgsent,nomsgnotdeliver,nomsgdelay 201003251000,1000,1,2 201003251000,900,0,0 201003251000,1450,0,0 201003251000,1230,0,0 However, sometimes, the column will missing in the raw files: e.g. date,nomsgsent,nomsgdelay... (8 Replies)
Discussion started by: natalie23
8 Replies

5. Shell Programming and Scripting

Fill missing numbers in second column with zeros

Hi All, I have 100 files with names like this: 1.dat, 2.dat, 3.dat until 100.dat. My dat files look like this: 42323 0 438939 1 434 0 0.9383 3434 120.23 3 234 As you can see in the second column, some numbers are missing. I want to fill those missing places with 0's in all... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

6. Shell Programming and Scripting

AWK script to create max value of 3rd column, grouping by first column

Hi, I need an awk script (or whatever shell-construct) that would take data like below and get the max value of 3 column, when grouping by the 1st column. clientname,day-of-month,max-users ----------------------------------- client1,20120610,5 client2,20120610,2 client3,20120610,7... (3 Replies)
Discussion started by: ckmehta
3 Replies

7. UNIX for Dummies Questions & Answers

How to combine and insert missing consecutive numbers - awk or script?

Hi all, I have two (2) sets of files that are based on some snapshots of database that I want to merge and insert any missing sequential number. Below are example representation of these files: file1: DATE TIME COL1 COL2 COL3 COL4 ID 01/10/2013 0800 100 ... (3 Replies)
Discussion started by: newbie_01
3 Replies

8. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000... (4 Replies)
Discussion started by: malandisa
4 Replies

9. Shell Programming and Scripting

awk script to append suffix to column when column has duplicated values

Please help me to get required output for both scenario 1 and scenario 2 and need separate code for both scenario 1 and scenario 2 Scenario 1 i need to do below changes only when column1 is CR and column3 has duplicates rows/values. This inputfile can contain 100 of this duplicated rows of... (1 Reply)
Discussion started by: as7951
1 Replies
tabs(1) 						      General Commands Manual							   tabs(1)

NAME
tabs - Sets tab stops on terminals SYNOPSIS
tabs [-n] [-T terminal] [+m [margin]] tabs [predefined_tab_flag] [-T terminal] [+m [margin]] tabs [-T terminal] [+m [margin]] number [,number]... tabs [--tab_format_file] The tabs command clears up to 20 previous tabs and sets up to 40 tabs on the terminal according to the supplied tab specifications. STANDARDS
Interfaces documented on this reference page conform to industry standards as follows: tabs: XCU5.0 Refer to the standards(5) reference page for more information about industry standards and associated tags. OPTIONS
Specifies repetitive tab stops separated by a uniform number of column positions, n, where n is a single-digit number. The default usage of tabs with no arguments is equivalent to tabs -8. If you use -0, the tab stops are cleared and no new ones are set. Identifies the terminal so that tabs can set tabs and margins correctly. The terminal argument is one of the conventional terminal names supported by your system. [Tru64 UNIX] If the terminal you specify is not known to the system, tabs tries a general value that works for most terminals. If you do not provide a -T option, tabs uses the TERM shell variable. Moves all tabs to the right number columns, and makes column number+1 the left margin. If m is given without a value, 10 is assumed. The leftmost margin on most terminals is defined by m0. Specifies that tabs should be set to a structured language format that is known to the system. See the section Predefined Tab Flags for a description of these flags. [Tru64 UNIX] Causes tabs to read a file named tab_format_file for format information. The first line of the file must be in the format shown in the section Format Specifications to use this method. The file may contain other lines which are ignored by tabs. Note [Tru64 UNIX] The double dash (--) format is mandatory with this option. OPERANDS
Sets tabs at the named column numbers (a list in ascending order, separated by commas). You can specify up to 40 numbers. If any number except the first has a plus sign prefix, the prefixed number is added to the previous number for the next setting. Thus, the tab lists 1,10,20,30 and 1,10,+10,+10 provide the same tab settings. DESCRIPTION
If you use the tabs command with no options or operands, the terminal tabs are reset to the system defaults for your terminal type. If only the -T option is used, tabs are reset to the defaults for that terminal type. [Tru64 UNIX] When you use the tabs command, always consider the leftmost column number to be 1, even if your terminal refers to it as 0 (zero). Tab-stop position n means that tabbing to position n causes the next character output to be in the n+1th column position on that line. Predefined Tab Flags [Tru64 UNIX] The flags described in the following list provide formats required by most structured programming languages. Some of these flags require that a particular format line be present in the file being manipulated. This is indicated in the list. Sets the tabs to 1, 10, 16, 36, and 72. Sets the tabs to 1, 10, 16, 40, and 72. Sets the tabs to 1, 8, 12, 16, 20, and 55 (COBOL normal format) Sets the tabs to 1, 6, 10, 14, and 49 (COBOL compact format, columns 1 to 6 omitted). With this code, the first column position corresponds to card col- umn 7. One space gets you to column 8, and a tab reaches column 12. Files using this code must include the following format specifica- tion: <:t-c2 m6 s66 d:> [Tru64 UNIX] See Format Specifications later in this reference page. Sets the tabs to 1, 6, 10, 14, 18, 22, 26, 30, 34, 38, 42, 46, 50, 54, 58, 62, and 67 (COBOL compact format with more tabs than -c2). This is the recommended format for COBOL. Files using this code must include the following format specification: <:t-c3 m6 s66 d:> Sets the tabs to 1, 7, 11, 15, 19, and 23 (FORTRAN). Sets the tabs to 1, 5, 9, 13, 17, 21, 25, 29, 33, 37, 41, 45, 49, 53, 57, and 61 (PL/I). Sets the tabs to 1, 10, and 55 (SNOBOL). Sets the tabs to 1, 12, 20, and 44. Format Specifications [Tru64 UNIX] A format specification consists of a sequence of arguments separated by blanks and enclosed in brackets and colons: <: :>. Each argument consists of a keyletter and an optional value which immediately follows it. The following arguments can be used: [Tru64 UNIX] Specifies tab settings. The value for tabs can be: [Tru64 UNIX] A list of column numbers separated by commas, indicating tab stops at the specified columns. [Tru64 UNIX] A - (dash) followed by a number n, specifying tabs stops every n columns. [Tru64 UNIX] A preset tab specification, for example -a2. [Tru64 UNIX] Specifies a maximum line size, or length. The size specification must be an integer. (The value of size is checked after tabs have been expanded but before the margin is prepended.) [Tru64 UNIX] Specifies a number of spa- ces to be inserted at the beginning of each line. The margin specification must be an integer. [Tru64 UNIX] Indicates that the line con- taining the format specification is to be deleted from the converted file. [Tru64 UNIX] If a format specification can be disguised as a comment, for example * <:t5,10,15 s75 m5:> *, you do not need to include the d keyletter. [Tru64 UNIX] Indicates that the current format should prevail only until another format specification is encountered in the file. [Tru64 UNIX] Default values of t-8 and m0 are assumed if t and m arguments are not included in the specification; if s is not included, line size is not checked. If the first line of a file does not contain a format specification, these defaults are assumed for the entire file. NOTES
The tabs command assumes that standard output is the terminal. If standard output is redirected, the results are unpredicable. Full oper- ation of all capabilities may be restricted by the hardware. EXIT STATUS
The following exit values are returned: Successful completion. An error occurred. EXAMPLES
To set tabs every four spaces, enter: tabs -4 To clear all tabs, enter: tabs ENVIRONMENT VARIABLES
The following environment variables affect the execution of tabs: Provides a default value for the internationalization variables that are unset or null. If LANG is unset or null, the corresponding value from the default locale is used. If any of the internationalization vari- ables contain an invalid setting, the utility behaves as if none of the variables had been defined. If set to a non-empty string value, overrides the values of all the other internationalization variables. Determines the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments). Determines the locale for the for- mat and contents of diagnostic messages written to standard error. Determines the location of message catalogues for the processing of LC_MESSAGES. Determines the terminal type if the -T option is not used. SEE ALSO
Commands: expand(1), nroff(1), stty(1), tset(1) Standards: standards(5) tabs(1)
All times are GMT -4. The time now is 05:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy