Read values in each col starting 3rd row.Print occurrence value.


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Old 04-10-2016
Read values in each col starting 3rd row.Print occurrence value.

Hello Friends,

Hope all are doing fine.

Here is a tricky issue.

my input file is like this
Code:
07 10 14 20 21
03 15 27 30 32
01 10 11 19 30
02 06 14 15 17
01 06 20 25 29


Logic:
1. Please print another column as "0-0-0-0-0" for the first and second rows.
2. Read the first column of third row, which is 1. Look for this value in all columns of first and second row. 1 is not present in first or second rows, so print a value 2 for this.
3. Then read the second column of third row, which is 10. There is 10 in first and not in second rows. So, it basically skipped the second row only. Now print a value of 1.
4. Then read the third column of third row, which is 11. 11 does not appear in first or second rows, so print a value of 2.
5. 19 has no appearances in first or second rows, so its value will be 2.
6. 30 did not appear in first row, but it appears in second row, which is the IMMEDIATE row of the current row that is being read which is row no.3. So, since no rows were skipped, we will print 0 for this one

So far the output will be

Code:
07 10 14 20 21 0-0-0-0-0
03 15 27 30 32 0-0-0-0-0
01 10 11 19 30 2-1-2-2-0
02 06 14 15 17
01 06 20 25 29

Logic Continued:
7. Now read the first column of fourth row, which is 2. Look for this value across anywhere in all the three rows above. Since 2 is not present, we will print a value 3 for this. Because it is not present in the three rows above.
8. Now read the second column of 4th row, which is 6. 6 is also not present across any of the three rows above. So its value will also be 3.
9. Read the third column of 4th row, which is 14. 14 is present in first row only but not in the second or third rows, so it skipped two rows. So print a value of 2 for this.
10. Read forth column of 4th row, which is 15. It is not present in first row. Fine. It is present in second row and not in third row. It basically skipped one IMMEDIATE row which is the third. We don't really care for the first row here. All we worry about is the number of times a value skipped after it appeared in the input file. So, the value for 15 will be 1.
11. Read last column of 4th row, which is 17. It is not present in any of the three rows above. So, we will print 3. Basically, if a value is not present across all top rows of the current row being considered, we HYPOTHESIZE that this value was PRESENT before the first line of the input. That is the reason we are printing 3 for the values that are not seen in any of the rows above row number 4.

So far, the output looks like this

Code:
07 10 14 20 21 0-0-0-0-0
03 15 27 30 32 0-0-0-0-0
01 10 11 19 30 2-1-2-2-0
02 06 14 15 17 3-3-2-1-3
01 06 20 25 29

Logic Continued:
12. Now the last row's first value which is 01. This iss present in third row and skipped the 4th. So, its value will be 1. If you see a value present in any row above the current row, then you DONT have to move any way further up because you have already seen that value.
13. Second column of last row, which is 06. This is present in 4th row. So, the value will be zero and DO NOT check any lines above because a value has been encountered.
14. Third column of last row, which is 20. It is present in first row but not in second, third or fourth rows. So, it skipped three rows. Print a value of 3 for this.
15. Forth column of last row, which is 25. This is not present anywhere. "Remember our hypothesis - this value occurred before the first line". So, we are printing 4 for this.
16. Fifth column of last row, which is 29. Present nowhere. So, print a value of 4.

Here is the final output

Code:
07 10 14 20 21 0-0-0-0-0
03 15 27 30 32 0-0-0-0-0
01 10 11 19 30 2-1-2-2-0
02 06 14 15 17 3-3-2-1-3
01 06 20 25 29 1-0-3-4-4

I would also like to have the frequency of unique numbers in the output column like this here

Code:
0=12times
1=3times
2=4times
3=4times
4=2times

Please ask me any questions or comments in case of any doubt.

P.S:
a. My columns are always 5.
b. My input file always has 25 records only.
c. A bonus of 5000 bits will be awarded to the best working solution.

Thank You!

Last edited by jacobs.smith; 04-10-2016 at 03:03 PM.. Reason: code tags format
# 2  
Old 04-10-2016
Why is the column added to row 2 always filled with 0-0-0-0-0? Why aren't entries in that row set to 1 if the number in a given column in row 2 is not present in row 1? In the given example, why shouldn't the last field in the output for row 2 be 1-1-1-1-1?

Other than being an interesting puzzle, does this problem address some real-world issue?
# 3  
Old 04-10-2016
In the secondary output:
Code:
0=12times
1=3times
2=4times
3=4times
4=2times

where do these numbers come from?

If you're counting the number of times a digit appears in the input, 0 occurs 13 times (not 12 times) in your sample input. If you're counting the number of times a value appears in your sample input, 0 (or 00) does not appear at all???

All of your input values are two digit strings. Are we supposed to treat 01 and 1 as the same value or as distinct values? If they are the same, is 010 to be treated as an octal value (decimal 8) or as a decimal value (10)?
# 4  
Old 04-10-2016
Assuming that data values are strings (not numbers that need to be converted to a canonical format), and that you want a count of the number of times a string appears in your input file, the following awk script seems to come close to what you said you wanted:
Code:
awk '
{	for(i = 1; i <= NF; i++) {
		c[$i]++
		if(NR > 2)
			lf = ((i > 1) ? lf "-" : "") NR - lr[$i] - 1
		else	lf = (i > 1) ? lf "-0" : "0"
	}
	for(i = 1; i <= NF; i++)
		lr[$i] = NR
	print $0, lf
}
END {	cmd = "sort -t="
	printf("\n%d rows containing %d columns processed.\n\n", NR, NF)
	for(i in c)
		printf("%s=%dtime%s\n", i, c[i], (c[i] == 1) ? "" : "s") | cmd
	close(cmd)
}' file

producing the following output from your sample data:
Code:
07 10 14 20 21 0-0-0-0-0
03 15 27 30 32 0-0-0-0-0
01 10 11 19 30 2-1-2-2-0
02 06 14 15 17 3-3-2-1-3
01 06 20 25 29 1-0-3-4-4

5 rows containing 5 columns processed.

01=2times
02=1time
03=1time
06=2times
07=1time
10=2times
11=1time
14=2times
15=2times
17=1time
19=1time
20=2times
21=1time
25=1time
27=1time
29=1time
30=2times
32=1time

(although if I were specifying the output format I'd put spaces around the equal signs and before the "time" in the secondary output.

As always, if you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
This User Gave Thanks to Don Cragun For This Post:
jacobs.smith (04-11-2016)
# 5  
Old 04-11-2016
Quote:
Originally Posted by Don Cragun
Assuming that data values are strings (not numbers that need to be converted to a canonical format), and that you want a count of the number of times a string appears in your input file, the following awk script seems to come close to what you said you wanted:
Code:
awk '
{	for(i = 1; i <= NF; i++) {
		c[$i]++
		if(NR > 2)
			lf = ((i > 1) ? lf "-" : "") NR - lr[$i] - 1
		else	lf = (i > 1) ? lf "-0" : "0"
	}
	for(i = 1; i <= NF; i++)
		lr[$i] = NR
	print $0, lf
}
END {	cmd = "sort -t="
	printf("\n%d rows containing %d columns processed.\n\n", NR, NF)
	for(i in c)
		printf("%s=%dtime%s\n", i, c[i], (c[i] == 1) ? "" : "s") | cmd
	close(cmd)
}' file

producing the following output from your sample data:
Code:
07 10 14 20 21 0-0-0-0-0
03 15 27 30 32 0-0-0-0-0
01 10 11 19 30 2-1-2-2-0
02 06 14 15 17 3-3-2-1-3
01 06 20 25 29 1-0-3-4-4

5 rows containing 5 columns processed.

01=2times
02=1time
03=1time
06=2times
07=1time
10=2times
11=1time
14=2times
15=2times
17=1time
19=1time
20=2times
21=1time
25=1time
27=1time
29=1time
30=2times
32=1time

(although if I were specifying the output format I'd put spaces around the equal signs and before the "time" in the secondary output.

As always, if you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
Don, Thank you.

You are really a Don!!!!

Your counting at the end is much more comprehensive than what I had thought.

It's all biology related. Definitely real time. Thank you.

I will be sending you the bonus right away.

Thanks a lot once again.
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
sed print from last occurrence match until the end of last occurrence match Jyotshna Shell Programming and Scripting 13 12-14-2017 03:58 AM
How to mark the row based on col value.? ken6503 Shell Programming and Scripting 5 02-10-2015 01:55 AM
UNIX help to print 50 lines after every 3rd occurrence pattern till end of file NSS Shell Programming and Scripting 5 10-01-2014 04:19 PM
Read row number from 1 file and print that row of second file Abhiraj Singh Shell Programming and Scripting 3 02-28-2014 04:20 AM
Get row number from file1 and print that row of file2 Abhiraj Singh Shell Programming and Scripting 6 02-27-2014 02:15 PM
Print lines in which value in specified Col is NOT unique owwow14 Shell Programming and Scripting 7 01-30-2014 07:59 AM
Print lines that contain a value in a specific column shared by more than 1 entity in another col owwow14 Shell Programming and Scripting 2 10-31-2013 08:17 AM
Print every 5 4th column values as separate row with different first column jacobs.smith Shell Programming and Scripting 3 02-20-2013 04:20 PM
Modifying col values based on another col newbie83 Shell Programming and Scripting 9 01-25-2013 01:24 PM
Change col to row using shell script..Very Complex asavaliya Shell Programming and Scripting 1 07-18-2012 09:27 PM
how to join files with diff col # and row #? BNasir UNIX for Dummies Questions & Answers 1 09-14-2011 02:30 PM
Print line based on highest value of col (B) and repetion of values in col (A) imahmoud UNIX for Advanced & Expert Users 4 07-25-2011 12:04 PM
Match col 1 of File 1 with col 1 File 2 and create a 3rd file sogi Ubuntu 7 06-30-2009 11:13 PM
Awk to print distinct col values anduzzi Shell Programming and Scripting 22 08-18-2008 05:09 PM
Print starting 3rd line until end of the file. smc3 Shell Programming and Scripting 1 06-19-2008 01:46 PM