Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk code to ignore the first occurence unknown number of rows in a data column

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk code to ignore the first occurence unknown number of rows in a data column
# 1  
Old 05-19-2013
awk code to ignore the first occurence unknown number of rows in a data column
Hello experts,

Shown below is the 2 column sample data(there are many data columns in actual input file),
Code:
Key, Data
A, 1
A, 2
A, 2
A, 3
A, 1
A, 1
A, 1

I need the below output.
Code:
Key, Data
A, 2
A, 2
A, 3
A, 1
A, 1
A, 1

So if any data values gets repeated then ignore the first occurrence and consider only the 2nd occurrence. The big difficulty to use head or awk 'NR>x' etc. options is I don't have how many rows I need to delete for each unique key.
In reality I have 1000's of rows for each unique key of which some of the rows (always at the start) need to be deleted.

Please let me know if my problem statement is not clear.

Thank you so much.

Sidda
# 2  
Old 05-20-2013
Based on your requirement The last the A, 1 rows should not be there in the example. Or I don't get what you want.
# 3  
Old 05-20-2013
Find the attachment which has input and output data
Hi Jim,
Thanks for your quick reply.
Find the attachment which clearly explains input, condition and the output.

Thanks
Sidda
awk code to ignore the first occurence unknown number of rows in a data column-input_and_outputpng
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to ignore multiple rows based on a condition

All, I have a text file(Inputfile.csv) with millions of rows and 100 columns. Check the sample for 2 columns below. Key,Check A,1 A,2 A, A,4 B,0 B,1 B,2 B,3 B,4 .... million rows. My requirement is to delete all the rows corresponding to all the keys which ever has at least one... (4 Replies)
Discussion started by: ks_reddy
4 Replies

2. Shell Programming and Scripting

Print Unknown Number of User Inputs in awk

Hello, I am new to awk and I am trying to figure out how to print an output based on user input. For example: ubuntu:~/scripts$ steps="step1, step2, step3" ubuntu:~/scripts$ echo $steps step1, step2, step3 I am playing around and I got this pattern that I want: ... (3 Replies)
Discussion started by: tattoostreet
3 Replies

3. Shell Programming and Scripting

Split column into two on first occurence of any number

Input : abc def 1 xyz zzz bca cde 2 yyy xxx Expected output : abc def |1 xyz zzz bca cde |2 yyy xxx I have tried the command below and losing the number. Any help is greatly appreciated 1. sed 's//|/' num.txt Result: abc def | xyz zzz bca cde |... (7 Replies)
Discussion started by: kbsuryadev
7 Replies

4. Shell Programming and Scripting

Match pattern and print the line number of occurence using awk

Hi, I have a simple problem but i guess stupid enough to figure it out. i have thousands rows of data. and i need to find match patterns of two columns and print the number of rows. for example: inputfile abd abp 123 abc abc 325 ndc ndc 451 mjk lkj... (3 Replies)
Discussion started by: redse171
3 Replies

5. Shell Programming and Scripting

awk command to print only selected rows in a particular column specified by column name

Dear All, I have a data file input.csv like below. (Only five column shown here for example.) Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 3,2,4,5,6 5,3,5,5,6 From this I want the below output Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 where the second column... (4 Replies)
Discussion started by: ks_reddy
4 Replies

6. UNIX for Dummies Questions & Answers

how to count number of rows and sum of column using awk

Hi All, I have the following input which i want to process using AWK. Rows,NC,amount 1,1202,0.192387 2,1201,0.111111 3,1201,0.123456 i want the following output count of rows = 3 ,sum of amount = 0.426954 Many thanks (2 Replies)
Discussion started by: pistachio
2 Replies

7. Shell Programming and Scripting

Script to find the average of a given column and also for specified number of rows??

Hi friends I have 100 files in my directory. Each file look like this.. Temp1 Temp2 Temp3 MAS 1 2 3 MAS 4 5 6 MAS 7 8 9 Delhi 10 11 12 Delhi 13 14 15 Delhi 16 17 ... (4 Replies)
Discussion started by: ks_reddy
4 Replies

8. Shell Programming and Scripting

awk out an unknown column ?

I need a solution to awk out an unknown column. I am unable to say '{print $x}' because the location changes. I would like to find a perl or awk solution to this. I do not know either very well but am trying to delve deeper into both. I am looking for the version of pkg8 in this example. Please... (17 Replies)
Discussion started by: i9300
17 Replies

9. Shell Programming and Scripting

awk, ignore first x number of lines.

Is there a way to tell awk to ignore the first 11 lines of a file?? example, I have a csv file with all the heading information in the first lines. I want to split the file into 5-6 different files but I want to retain the the first 11 lines of the file. As it is now I run this command: ... (8 Replies)
Discussion started by: trey85stang
8 Replies

10. Shell Programming and Scripting

Requesting an AWK code to generate averaged rows in a column

Hello, I request your kind help in solving this problem... I have a file with two columns and "n" number of rows. For the first column, it won't be change. For the second column, I want to take the average of the first three rows. Then assign the averaged value to the first three rows. This... (8 Replies)
Discussion started by: solracq
8 Replies
Login or Register to Ask a Question