Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to mark the row based on col value.?
Top Forums Shell Programming and Scripting How to mark the row based on col value.? Post 302934688 by Don Cragun on Tuesday 10th of February 2015 01:55:10 AM
Old 02-10-2015
From post #1 in this thread it isn't clear whether your input file contains a header line or not and it appears that you want a two line header in your output file (with the first header line consisting of a single space character).

Rather than working on pairs of lines and starting counting lines on line 1 or line 2, the following code produces the desired header and then groups all sets of one or more adjacent lines that have three fields and contain the same numeric value in the first field adding a "yes" to each line in the set on output if none of the 3rd fields in the set on input is 0 or an empty string, and adding a "no" to each line in the set on output if one or more of the 3rd fields in the set on input is 0 or an empty string:
Code:
#!/bin/ksh
awk '
BEGIN {	printf(" \nID, AMT, AMT1, ind\n")
	FS = "[ ,]+"
	OFS = ", "
}
function pd(	i) {
	for(i = 1; i <= cnt; i++)
		print line[i], ind
	ind = "yes"
	cnt = 0
}
NF == 3 && ($1 + 0) == $1 {
	if($1 != last)
		pd()
	last = $1
	line[++cnt] = $0
	if($3 == 0)
		ind = "no"
}
END {	pd()
}' sample.txt

If sample.txt contains the text you showed in post #1 in this thread optionally containing one or more blank lines and any number of either, both, or neither of the following header lines:
Code:
ID, AMT, AMT1

or:
Code:
ID AMT, AMT1

and any number of comment lines (starting with a # or an alphabetic character); it produces the output:
Code:
 
ID, AMT, AMT1, ind
100,10, 2, yes
100,20, 3, yes
200,30, 0, no
200, 40, 0, no
300, 20, 2, yes
300, 50, 2, yes
400, 20, 1, no
400, 60, 0, no

as you requested.

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk, /usr/xpg6bin/awk, or nawk.

This was tested using the Korn shell, but will work with any shell that uses basic Bourne shell syntax.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Print line based on highest value of col (B) and repetion of values in col (A)

Hello everyone, I am writing a script to process data from the ATP world tour. I have a file which contains: t=540 y=2011 r=1 p=N409 t=540 y=2011 r=2 p=N409 t=540 y=2011 r=3 p=N409 t=540 y=2011 r=4 p=N409 t=520 y=2011 r=1 p=N409 t=520 y=2011 r=2 p=N409 t=520 y=2011 r=3 p=N409 The... (4 Replies)
Discussion started by: imahmoud
4 Replies

2. UNIX for Dummies Questions & Answers

how to join files with diff col # and row #?

I am a new user of Unix/Linux, so this question might be a bit simple! I am trying to join two (very large) files that both have different # of cols and rows in each file. I want to keep 'all' rows and 'all' cols from both files in the joint file, and the primary key variables are in the rows.... (1 Reply)
Discussion started by: BNasir
1 Replies

3. Shell Programming and Scripting

Find duplicate based on 'n' fields and mark the duplicate as 'D'

Hi, In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'. In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies

4. Shell Programming and Scripting

Change col to row using shell script..Very Complex

Hi guys I have file A with Below Data ABC123 X1 X2 X3 ABC123 Y1 Y33 Y4 ABC123 Z1 ZS2 ZL3 ABC234 P1 PP3 PP9 ABC234 Q1 ABC234 R1 P09 PO332 PO331 OKI12 .. .. .. Now I want file B as below ABC123 X1 X2 X3;Y1 Y33 Y4;Z1 ZS2 ZL3 ABC234 P1 PP3 PP9;Q1;R1 P09 PO332 PO331 OKI12... (1 Reply)
Discussion started by: asavaliya
1 Replies

5. Shell Programming and Scripting

Printing from col x to end of line, except last col

Hello, I have some tab delimited data and I need to move the last col. I could hard code it, awk '{ print $1,$NF,$2,$3,$4,etc }' infile > outfile but it would be nice to know the syntax to print a range cols. I know in cut you can do, cut -f 1,4-8,11- to print fields 1,... (8 Replies)
Discussion started by: LMHmedchem
8 Replies

6. Shell Programming and Scripting

Modifying col values based on another col

Hi, Please help with this. I have several excel files (with and .xlsx format) with 10-15 columns each. They all have the same type of data but the columns are not ordered in the same way. Here is a 3 column example. What I want to do add the alphabet from column 2 to column 3, provided... (9 Replies)
Discussion started by: newbie83
9 Replies

7. Shell Programming and Scripting

Bin iteratively based on each row

Hi I have a file some thing like below. I want to bin the data. My Bin size is 100. items number HELIX1 75 HELIX6 160 HELIX2 88 HELIX19 114 HELIX5 61 HELIX4 167 it should consider each elemet under the number column and bin all the lines like below with 100... (7 Replies)
Discussion started by: and_y
7 Replies

8. Shell Programming and Scripting

Field widths based on a row

I want to specify field width based on the row with FTR. I can acheive this if column width is constant with: awk 'BEGIN { FIELDWIDTHS = "20 7 14 30" }{print $1,$4}' file file:COL1 COL2 CL3 FTR AA8 S2 CAT2 your comments CC7 ... (5 Replies)
Discussion started by: aydj
5 Replies

9. Emergency UNIX and Linux Support

Read values in each col starting 3rd row.Print occurrence value.

Hello Friends, Hope all are doing fine. Here is a tricky issue. my input file is like this 07 10 14 20 21 03 15 27 30 32 01 10 11 19 30 02 06 14 15 17 01 06 20 25 29 Logic: 1. Please print another column as "0-0-0-0-0" for the first and second rows. 2. Read the first column... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

10. Shell Programming and Scripting

Splitting single row into multiple rows based on for every 10 digits of last field of the row

Hi ALL, We have requirement in a file, i have multiple rows. Example below: Input file rows 01,1,102319,0,0,70,26,U,1,331,000000113200000011920000001212 01,1,102319,0,1,80,20,U,1,241,00000059420000006021 I need my output file should be as mentioned below. Last field should split for... (4 Replies)
Discussion started by: kotra
4 Replies

LEARN ABOUT DEBIAN

bup-margin

bup-margin(1)						      General Commands Manual						     bup-margin(1)

NAME

       bup-margin - figure out your deduplication safety margin

SYNOPSIS

       bup margin [options...]

DESCRIPTION

       bup margin  iterates  through  all  objects  in	your  bup repository, calculating the largest number of prefix bits shared between any two
       entries.  This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.

       For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45.  That  means  a  46-bit
       hash  would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
       its first 46 bits.

       The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects.  Since SHA-1 hashes have 160 bits,
       that  leaves 115 bits of margin.  Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
       with far fewer objects.

       If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see	if
       you're getting dangerously close to 160 bits.

OPTIONS

       --predict
	      Guess  the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
	      from the guess.  This is potentially useful for tuning an interpolation search algorithm.

       --ignore-midx
	      don't use .midx files, use only .idx files.  This is only really useful when used with --predict.

EXAMPLE

	      $ bup margin
	      Reading indexes: 100.00% (1612581/1612581), done.
	      40
	      40 matching prefix bits
	      1.94 bits per doubling
	      120 bits (61.86 doublings) remaining
	      4.19338e+18 times larger is possible

	      Everyone on earth could have 625878182 data sets
	      like yours, all in one repository, and we would
	      expect 1 object collision.

	      $ bup margin --predict
	      PackIdxList: using 1 index.
	      Reading indexes: 100.00% (1612581/1612581), done.
	      915 of 1612581 (0.057%)

SEE ALSO

       bup-midx(1), bup-save(1)

BUP

       Part of the bup(1) suite.

AUTHORS

       Avery Pennarun <apenwarr@gmail.com>.

Bup unknown-															     bup-margin(1)
All times are GMT -4. The time now is 06:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy