Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Check/print missing number in a consecutive range and remove duplicate numbers
Top Forums Shell Programming and Scripting Check/print missing number in a consecutive range and remove duplicate numbers Post 302995008 by newbie_01 on Thursday 30th of March 2017 02:20:11 PM
Old 03-30-2017
Hi all,

I've uploaded the script that I am using at the moment and some test data. It works like I intend it to, just thought maybe it can be improved somehow. So all the ones that are FAILED are the ones missing in the range of consecutive series

I didn't know I can use sort | uniq -c to check for duplicate, thanks to RudiC.

Checking up on the link below if it can be used instead.

awk to insert missing string based on pattern in file

Code:
$ ./x.ksh

-------------------------------------------------
- Running check_gap on test_clean.txt ...
-------------------------------------------------

- [ test_clean.txt.1.uniq ] ... starting gap check
   ... = Checking for 79811 => PASSED
   ... = Checking for 79812 => PASSED
   ... = Checking for 79813 => PASSED
   ... = Checking for 79814 => PASSED
   ... = Checking for 79815 => PASSED

- [ test_clean.txt.2.uniq ] ... starting gap check
   ... = Checking for 86754 => PASSED
   ... = Checking for 86755 => PASSED
   ... = Checking for 86756 => PASSED
   ... = Checking for 86757 => PASSED
   ... = Checking for 86758 => PASSED

- [ test_clean.txt.3.uniq ] ... starting gap check
   ... = Checking for 82692 => PASSED
   ... = Checking for 82693 => PASSED
   ... = Checking for 82694 => PASSED
   ... = Checking for 82695 => PASSED
   ... = Checking for 82696 => PASSED



-------------------------------------------------
- Running check_gap on test_gap.txt ...
-------------------------------------------------

- [ test_gap.txt.1.uniq ] ... starting gap check
   ... = Checking for 79811 => PASSED
   ... = Checking for 79812 => FAILED
   ... = Checking for 79813 => PASSED
   ... = Checking for 79814 => PASSED
   ... = Checking for 79815 => PASSED
   ... = Checking for 79816 => FAILED
   ... = Checking for 79817 => FAILED
   ... = Checking for 79818 => FAILED
   ... = Checking for 79819 => PASSED

- [ test_gap.txt.2.uniq ] ... starting gap check
   ... = Checking for 86754 => PASSED
   ... = Checking for 86755 => PASSED
   ... = Checking for 86756 => FAILED
   ... = Checking for 86757 => PASSED
   ... = Checking for 86758 => PASSED
   ... = Checking for 86759 => FAILED
   ... = Checking for 86760 => FAILED
   ... = Checking for 86761 => FAILED
   ... = Checking for 86762 => FAILED
   ... = Checking for 86763 => FAILED
   ... = Checking for 86764 => FAILED
   ... = Checking for 86765 => PASSED

- [ test_gap.txt.3.uniq ] ... starting gap check
   ... = Checking for 82692 => PASSED
   ... = Checking for 82693 => PASSED
   ... = Checking for 82694 => FAILED
   ... = Checking for 82695 => PASSED
   ... = Checking for 82696 => PASSED
   ... = Checking for 82697 => FAILED
   ... = Checking for 82698 => FAILED
   ... = Checking for 82699 => FAILED
   ... = Checking for 82700 => PASSED



-------------------------------------------------
- Running check_gap on test_dup.txt ...
-------------------------------------------------

- loggrp = 1 contains duplicated copies of the log
      3 1_79812_01234567.arc
      2 1_79819_01234567.arc

- [ test_dup.txt.1.uniq ] ... starting gap check
   ... = Checking for 79811 => PASSED
   ... = Checking for 79812 => PASSED
   ... = Checking for 79813 => PASSED
   ... = Checking for 79814 => PASSED
   ... = Checking for 79815 => PASSED
   ... = Checking for 79816 => FAILED
   ... = Checking for 79817 => FAILED
   ... = Checking for 79818 => FAILED
   ... = Checking for 79819 => PASSED

- loggrp = 2 contains duplicated copies of the log
      3 2_86756_01234567.arc
      2 2_86758_01234567.arc

- [ test_dup.txt.2.uniq ] ... starting gap check
   ... = Checking for 86754 => PASSED
   ... = Checking for 86755 => PASSED
   ... = Checking for 86756 => PASSED
   ... = Checking for 86757 => PASSED
   ... = Checking for 86758 => PASSED
   ... = Checking for 86759 => FAILED
   ... = Checking for 86760 => PASSED

- loggrp = 3 contains duplicated copies of the log
      2 3_82692_01234567.arc
      4 3_82696_01234567.arc
      2 3_82700_01234567.arc

- [ test_dup.txt.3.uniq ] ... starting gap check
   ... = Checking for 82692 => PASSED
   ... = Checking for 82693 => PASSED
   ... = Checking for 82694 => PASSED
   ... = Checking for 82695 => PASSED
   ... = Checking for 82696 => PASSED
   ... = Checking for 82697 => FAILED
   ... = Checking for 82698 => PASSED
   ... = Checking for 82699 => FAILED
   ... = Checking for 82700 => PASSED

test_gap.txt (328 Bytes) 
test_clean.txt (328 Bytes) 
test_dup.txt (658 Bytes) 
x.sh (2.6 KB) 
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Fill in missing numbers in range

I need to edit a list of numbers on the following form: 1 1.0 2 1.4 5 2.1 7 1.9 I want: 1 1.0 2 1.4 3 0.0 4 0.0 5 2.1 6 0.0 7 1.9 (i want to add the missing number in column 1 together with 0.0 in column 2). I guess it is rather trivial but i didn't even manage to read column... (5 Replies)
Discussion started by: bistru
5 Replies

2. Shell Programming and Scripting

Inserting a range of consecutive numbers into a text file

I have a text file in the following format .... START 1,1 2,1 3,1 .. .. 9,1 10,1 END .... I want to change to the output to .... START 1,1 2,1 3,1 .. (4 Replies)
Discussion started by: VNR
4 Replies

3. UNIX for Dummies Questions & Answers

Help required on Printing of Numbers, which are missing in the range

Hi Experts, Need help on printing of numbers, which are missing in the range. Pls find the details below Input 1000000002 1000000007 1234007940 1234007946 Output 1000000003 1000000004 1000000005 1000000006 1234007941 (2 Replies)
Discussion started by: krao
2 Replies

4. Shell Programming and Scripting

remove html tags,consecutive duplicate lines

I need help with a script that will remove all HTML tags from an HTML document and remove any consecutive duplicate lines, and save it as a text document. The user should have the option of including the name of an html file as an argument for the script, but if none is provided, then the script... (7 Replies)
Discussion started by: clicstic
7 Replies

5. Shell Programming and Scripting

remove consecutive duplicate rows

I have some data that looks like, 1 3300665.mol 3300665 5177008 102.093 2 3300665.mol 3300665 5177008 102.093 3 3294015.mol 3294015 5131552 102.114 4 3294015.mol 3294015 5131552 102.114 5 3293734.mol 3293734 5129625 104.152 6 3293734.mol ... (13 Replies)
Discussion started by: LMHmedchem
13 Replies

6. Shell Programming and Scripting

Print consecutive numbers in column2

Hi, I have an input file of the following style input.txt The 4000 at the end indicates the total no. of columns in that row. I would like to replace all -1s with consecutive 1 and 2 and print the whole line again. So, the output would be output.txt Thanks in advance. (7 Replies)
Discussion started by: jacobs.smith
7 Replies

7. Shell Programming and Scripting

Request to check:remove entries with duplicate numbers in first row

Hi I have a file 1 xyz 456 1 xyz 456 1 xyz 456 2 abc 8459 3 gfd 657 4 ghf 658 4 ghf 658 I want the output 1 xyz 456 2 abc 8459 3 gfd 657 4 ghf 658 (3 Replies)
Discussion started by: manigrover
3 Replies

8. UNIX for Dummies Questions & Answers

How to combine and insert missing consecutive numbers - awk or script?

Hi all, I have two (2) sets of files that are based on some snapshots of database that I want to merge and insert any missing sequential number. Below are example representation of these files: file1: DATE TIME COL1 COL2 COL3 COL4 ID 01/10/2013 0800 100 ... (3 Replies)
Discussion started by: newbie_01
3 Replies

9. Shell Programming and Scripting

Remove duplicate entries based on the range

I have file like this: chr start end chr15 99874874 99875874 chr15 99875173 99876173 aa1 chr15 99874923 99875923 chr15 99875173 99876173 aa1 chr15 99874962 99875962 chr15 99875173 99876173 aa1 chr1 ... (7 Replies)
Discussion started by: raj_k
7 Replies

10. Shell Programming and Scripting

Remove duplicate consecutive lines with specific string

Hello, I'm trying to remove the duplicate consecutive lines with specific string "WARNING". File.txt abc; WARNING 2345 WARNING 2345 WARNING 2345 WARNING 2345 WARNING 2345 bcd; abc; 123 123 123 WARNING 1234 WARNING 2345 WARNING 2345 efgh; (6 Replies)
Discussion started by: Mannu2525
6 Replies
All times are GMT -4. The time now is 05:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy