Number Grouped Rows in File Post: 302172323

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Number Grouped Rows in File Post 302172323 by RacerX on Monday 3rd of March 2008 11:04:52 AM

03-03-2008

Registered User

Number Grouped Rows in File

I have a file containing 750,000 records and have managed to sort them by related columns and now i'd like to add an ID number to the front of each line of the records that are grouped together. Which probably makes no sense so i have provided some example data and desired result.

Given data.txt sorted by the command sort -t':' -k1 -k2 -k3 ~/Desktop/data.txt > ~/Desktop/outdata.txt

Code:

98-03-13:BRR:2:7:-:-
98-03-13:BRR:2:7::
98-03-13:BRR:2:7::
98-03-13:BRR:2:7::
98-03-13:BRR:2:7::
98-03-13:BRR:2:7:Y:
98-03-13:BRR:2:7:Y:
98-03-13:BRR:6:1::
98-03-20:BRR:5:2:-:-
98-03-20:BRR:5:2:-:-
98-03-27:BRR:2:1::
98-10-13:MOHO:10:1::
98-10-13:MOHO:10:1::
98-11-11:PGA:4:7:-:-
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-11:PGA:4:7::
98-11-20:PGA:1:3::
98-11-20:PGA:1:3::
98-11-20:PGA:1:3::

The Desired Output

Code:

100:98-03-13:BRR:2:7:-:-
100:98-03-13:BRR:2:7::
100:98-03-13:BRR:2:7::
100:98-03-13:BRR:2:7::
100:98-03-13:BRR:2:7::
100:98-03-13:BRR:2:7:Y:
100:98-03-13:BRR:2:7:Y:
101:98-03-13:BRR:6:1::
102:98-03-20:BRR:5:2:-:-
102:98-03-20:BRR:5:2:-:-
103:98-03-27:BRR:2:1::
104:98-10-13:MOHO:10:1::
104:98-10-13:MOHO:10:1::
105:98-11-11:PGA:4:7:-:-
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
105:98-11-11:PGA:4:7::
106:98-11-20:PGA:1:3::
106:98-11-20:PGA:1:3::
106:98-11-20:PGA:1:3::

Could i do this with AWK? However, i cannot figure out what i would use as the record separator between groups of data.

Any help or direction would be appreciated.

RacerX

View Public Profile for RacerX

Find all posts by RacerX

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting file based on number of rows

Hi, I'm, new to shell scripting, I have a requirement where I have to split an incoming file into separate files each containing a maximum of 3 million rows. For e.g: if my incoming file say In.txt has 8 mn rows then I need to create 3 files, in which two will 3 mn rows and one will contain 2...

2. Shell Programming and Scripting

how to add the number of row and count number of rows

Hi experts a have a very large file and I need to add two columns: the first one numbering the incidence of records and the another with the total count The input file: 21 2341 A 21 2341 A 21 2341 A 21 2341 C 21 2341 C 21 2341 C 21 2341 C 21 4567 A 21 4567 A 21 4567 C ...

3. Shell Programming and Scripting

How to add the number of events grouped by field

Help i need to add to the end of the line the number of event grouped by id, my input file: IN FILE: ID DATE 10123 01-04-2008 10123 06-05-2008 10123 01-09-2009 10123 11-12-2009 10345 01-22-2008 10345 06-15-2008 10127 02-22-2010 OUT FILE: ID DATE EVENT_NUMBER 10123 01-04-2008 1...

4. Shell Programming and Scripting

awk number rows from 1 to 3 in file

I have a file with the following format, i need to change de field9 each 3 rows to renumber field9 with gpo1, gpo2, gpo3. I need to use awk Original file field1 field2 field3 field4 field5 field6 field7 field8 gpo3 field1 field2 field3 field4 field5 ...

5. UNIX Desktop Questions & Answers

Need to ftp some files, and check the number of rows in the transferd file

1)I need to write a script which ftps 3 files to a unix box, 2)once the files are ftped i need to check the number of rows in each file and compare it with the data (no of rows) coming in a manifest file, if the number of rows in each file matches the data coming in manifest file, then i need to...

6. UNIX for Advanced & Expert Users

merge two tab delimited file with exact same number of rows in unix/linux

Hi I have two tab delimited file with different number of columns but same number of rows. I need to combine these two files in such a way that row 1 in file 2 comes adjacent to row 1 in file 1. For example: The content of file1: field1 field2 field3 a1 a2 a3 b1 b2 b3...

7. UNIX for Dummies Questions & Answers

How to separate a single column file into files of the same size (i.e. number of rows)?

I have a text file with 1,000,000 rows (It is a single column text file of numbers). I would like to separate the text file into 100 files of equal size (i.e. number of rows). The first file will contain the first 10,000 rows, the second row will contain the second 10,000 rows (rows 10,001-20,000)...

8. UNIX for Dummies Questions & Answers

Write the total number of rows in multiple files into another file

Hello Friends, I know you all are busy and inteligent too... I am stuck with one small issue if you can help me then it will be really great. My problem is I am having some files i.e. Input.txt1 Input.txt2 Input.txt3 Now my task is I need to check the total number of rows in...

9. Shell Programming and Scripting

extracting number of rows

if > wc -l data.txt > 100 (meaning there are 100 rows) i want to assgn 100 as n so that if i do >echo n it would give me 100 Thanks

10. Shell Programming and Scripting

Split File based on number of rows

Hi I have a requirement, where i will receive multiple files in a folder (say: /fol1/fol2/). There will be at least 14 to 16 files. The size of the files will different, some may be 80GB or 90GB, some may be less than 5 GB (and the size of the files are very unpredictable). But the names of the...

LEARN ABOUT PHP

px_get_info

PX_GET_INFO(3)															    PX_GET_INFO(3)

px_get_info - Return lots of information about a paradox file

SYNOPSIS

       array px_get_info (resource  $pxdoc)

DESCRIPTION

PARAMETERS

	      o $pxdoc
		- Resource identifier of the paradox database as returned by px_new(3).

RETURN VALUES

	Returns an associated array with lots of information about a paradox file. This array is likely to be extended in the future.

	      o fileversion
		-Version of file multiplied by 10, e.g. 70.

	      o tablename
		-Name  of  table  as  stored in the file. If the database was created by pxlib, then this will be the name of the file without the
		extension.

	      o numrecords
		-Number of records in this table.

	      o numfields
		-Number of fields in this table.

	      o headersize
		-Number of bytes used for the header. This is usually 0x800.

	      o recordsize
		-Number of bytes used for each record. This is the sum of all field sizes (available since version 1.4.2).

	      o maxtablesize
		- This value multiplied by 0x400 is the size of a data block in bytes. The maximum number of records in a datablock is the integer
		part of (maxtablesize * 0x400 - 8) / recordsize.

	      o numdatablocks
		- The number of data blocks in the file. Each data block contains a certain number of records which depends on the record size and
		the data block size (maxtablesize). Data blocks may not necessarily be completely filled.

	      o numindexfields
		- Number of fields used for the primary index. The fields do always start with field number 1.

	      o codepage
		- The DOS codepage which was used for encoding fields with character data. If the target encoding is not set with px_set_targeten-
		coding(3)   this  will	be  the  encoding  for	character  fields  when  records  are  being  accessed	with  px_get_record(3)	or
		px_retrieve_record(3).

SEE ALSO

       px_numfields(3), px_numrecords(3).

PHP Documentation Group 													    PX_GET_INFO(3)