Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Number Grouped Rows in File

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Number Grouped Rows in File
# 1  
Old 03-03-2008
Number Grouped Rows in File
I have a file containing 750,000 records and have managed to sort them by related columns and now i'd like to add an ID number to the front of each line of the records that are grouped together. Which probably makes no sense so i have provided some example data and desired result.

Given data.txt sorted by the command sort -t':' -k1 -k2 -k3 ~/Desktop/data.txt > ~/Desktop/outdata.txt
Code:
98-03-13:BRR:2:7:-:-
98-03-13:BRR:2:7::
98-03-13:BRR:2:7::
98-03-13:BRR:2:7::
98-03-13:BRR:2:7::
98-03-13:BRR:2:7:Y:
98-03-13:BRR:2:7:Y:
98-03-13:BRR:6:1::
98-03-20:BRR:5:2:-:-
98-03-20:BRR:5:2:-:-
98-03-27:BRR:2:1::
98-10-13:MOHO:10:1::
98-10-13:MOHO:10:1::
98-11-11:PGA:4:7:-:-
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-20:PGA:1:3::
98-11-20:PGA:1:3::
98-11-20:PGA:1:3::

The Desired Output
Code:
100:98-03-13:BRR:2:7:-:-
100:98-03-13:BRR:2:7::
100:98-03-13:BRR:2:7::
100:98-03-13:BRR:2:7::
100:98-03-13:BRR:2:7::
100:98-03-13:BRR:2:7:Y:
100:98-03-13:BRR:2:7:Y:
101:98-03-13:BRR:6:1::
102:98-03-20:BRR:5:2:-:-
102:98-03-20:BRR:5:2:-:-
103:98-03-27:BRR:2:1::
104:98-10-13:MOHO:10:1::
104:98-10-13:MOHO:10:1::
105:98-11-11:PGA:4:7:-:-
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
106:98-11-20:PGA:1:3::
106:98-11-20:PGA:1:3::
106:98-11-20:PGA:1:3::

Could i do this with AWK? However, i cannot figure out what i would use as the record separator between groups of data.

Any help or direction would be appreciated.
# 2  
Old 03-03-2008
Code:
awk '!x[$3,$4]++{c++}$0=c FS$0' c=99 OFS=":" FS=":"

or

Code:
awk '$0=(!x[$3,$4]++?++c:c)FS$0' c=99 OFS=":" FS=":"


Use nawk or /usr/xpg4/bin/awk on Solaris.
Last edited by radoulov; 03-03-2008 at 12:40 PM.. Reason: added another version
# 3  
Old 03-03-2008
Brilliant simple solution! Did exactly what i needed it to do.

Thanks for your help!
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split File based on number of rows

Hi I have a requirement, where i will receive multiple files in a folder (say: /fol1/fol2/). There will be at least 14 to 16 files. The size of the files will different, some may be 80GB or 90GB, some may be less than 5 GB (and the size of the files are very unpredictable). But the names of the... (10 Replies)
Discussion started by: kpk_ds
10 Replies

2. Shell Programming and Scripting

extracting number of rows

if > wc -l data.txt > 100 (meaning there are 100 rows) i want to assgn 100 as n so that if i do >echo n it would give me 100 Thanks (5 Replies)
Discussion started by: johnkim0806
5 Replies

3. UNIX for Dummies Questions & Answers

Write the total number of rows in multiple files into another file

Hello Friends, I know you all are busy and inteligent too... I am stuck with one small issue if you can help me then it will be really great. My problem is I am having some files i.e. Input.txt1 Input.txt2 Input.txt3 Now my task is I need to check the total number of rows in... (4 Replies)
Discussion started by: malaya kumar
4 Replies

4. UNIX for Dummies Questions & Answers

How to separate a single column file into files of the same size (i.e. number of rows)?

I have a text file with 1,000,000 rows (It is a single column text file of numbers). I would like to separate the text file into 100 files of equal size (i.e. number of rows). The first file will contain the first 10,000 rows, the second row will contain the second 10,000 rows (rows 10,001-20,000)... (2 Replies)
Discussion started by: evelibertine
2 Replies

5. UNIX for Advanced & Expert Users

merge two tab delimited file with exact same number of rows in unix/linux

Hi I have two tab delimited file with different number of columns but same number of rows. I need to combine these two files in such a way that row 1 in file 2 comes adjacent to row 1 in file 1. For example: The content of file1: field1 field2 field3 a1 a2 a3 b1 b2 b3... (2 Replies)
Discussion started by: mary271
2 Replies

6. UNIX Desktop Questions & Answers

Need to ftp some files, and check the number of rows in the transferd file

1)I need to write a script which ftps 3 files to a unix box, 2)once the files are ftped i need to check the number of rows in each file and compare it with the data (no of rows) coming in a manifest file, if the number of rows in each file matches the data coming in manifest file, then i need to... (3 Replies)
Discussion started by: imran_affu
3 Replies

7. Shell Programming and Scripting

awk number rows from 1 to 3 in file

I have a file with the following format, i need to change de field9 each 3 rows to renumber field9 with gpo1, gpo2, gpo3. I need to use awk Original file field1 field2 field3 field4 field5 field6 field7 field8 gpo3 field1 field2 field3 field4 field5 ... (3 Replies)
Discussion started by: robonet
3 Replies

8. Shell Programming and Scripting

How to add the number of events grouped by field

Help i need to add to the end of the line the number of event grouped by id, my input file: IN FILE: ID DATE 10123 01-04-2008 10123 06-05-2008 10123 01-09-2009 10123 11-12-2009 10345 01-22-2008 10345 06-15-2008 10127 02-22-2010 OUT FILE: ID DATE EVENT_NUMBER 10123 01-04-2008 1... (3 Replies)
Discussion started by: juelillo
3 Replies

9. Shell Programming and Scripting

how to add the number of row and count number of rows

Hi experts a have a very large file and I need to add two columns: the first one numbering the incidence of records and the another with the total count The input file: 21 2341 A 21 2341 A 21 2341 A 21 2341 C 21 2341 C 21 2341 C 21 2341 C 21 4567 A 21 4567 A 21 4567 C ... (6 Replies)
Discussion started by: juelillo
6 Replies

10. Shell Programming and Scripting

Splitting file based on number of rows

Hi, I'm, new to shell scripting, I have a requirement where I have to split an incoming file into separate files each containing a maximum of 3 million rows. For e.g: if my incoming file say In.txt has 8 mn rows then I need to create 3 files, in which two will 3 mn rows and one will contain 2... (2 Replies)
Discussion started by: wahi80
2 Replies
Login or Register to Ask a Question