Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Fetching record based on Uniq Key from huge file.
Top Forums Shell Programming and Scripting Fetching record based on Uniq Key from huge file. Post 302717371 by lathigara on Thursday 18th of October 2012 05:04:21 AM
Old 10-18-2012
output will be 100k records.

The one which i pinged is one record and i want to fetch 100k record from a file having 1000k file.


Code:
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
                              EMDS/CREDIT SERVICES                     RunDate: 10/25/2011                    REC#      1
             SEQUENCE NUMBER: 06000000001           L90 KEY :060882D85800E201684C0000            CID KEY:000000000000000000
DEBT RATIO=  0%  INQ W/I 6=  0, W/I 12=  0, W/I 18=  0, W/I 24=  0 ,PRM W/I 6=0; Trades RPTed W/I 24 =  0, No. of Trades =   3
*** FACT FRAUD CODE =  ,FACT ALERT DATA CODE = 
XXXXXX                    XXXXX                                                                   SSN-XXX-XX-XXXX C
XXXXXX,XXXXX.                                                                            FAD:  0/00/0000  FILE SINCE:  0/00/0000
XXXXX XXXXXXXX XX  XXXXXXXCA,92236
,,,,,,,,0
,,,,,,,,0
,,,,,,,,0
 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
                              EMDS/CREDIT SERVICES                     RunDate: 10/25/2011                    REC#      2
             SEQUENCE NUMBER: 06000000002           L90 KEY :060882D85800E201684C0000            CID KEY:000000000000000000
DEBT RATIO=  0%  INQ W/I 6=  0, W/I 12=  0, W/I 18=  0, W/I 24=  0 ,PRM W/I 6=0; Trades RPTed W/I 24 =  0, No. of Trades =   3
*** FACT FRAUD CODE =  ,FACT ALERT DATA CODE = 
XXXXXX                    XXXXX                                                                   SSN-XXX-XX-XXXX C
XXXXXX,XXXXX.                                                                            FAD:  0/00/0000  FILE SINCE:  0/00/0000
XXXXX XXXXXXXX XX  XXXXXXXCA,92236
,,,,,,,,0
,,,,,,,,0
,,,,,,,,0

like 100k.

thanks,
prashant
Last edited by Scrutinizer; 10-18-2012 at 06:45 AM.. Reason: code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Parsing out records from one huge record

Hi, I have one huge record and know that each record in the file is 550 bytes long. How do I parse out individual records from the single huge record. Thanks, (4 Replies)
Discussion started by: bwrynz1
4 Replies

2. Shell Programming and Scripting

Compare 2 huge files wrt to a key using awk

Hi Folks, I need to compare two very huge file ( i.e the files would contain a minimum of 70k records each) using awk or sed. The comparison needs to be done with respect to a 'key'. For example : File1 ********** 1234|TONY|Y75634|20/07/2008 1235|TINA|XCVB56|30/07/2009... (13 Replies)
Discussion started by: Ranjani
13 Replies

3. Shell Programming and Scripting

Logic for file fetching based on date

Dear friends, I receive the following files into a FTP location on a daily basis -rw-r----- 1 guest ftp1 5021 Aug 19 09:03 CHECK_TEST_Extracts_20080818210000.zip -rw-r----- 1 guest ftp1 2437 Aug 20 05:15 CHECK_TEST_Extracts_20080819210000.zip -rw-r----- 1 guest ... (2 Replies)
Discussion started by: sureshg_sampat
2 Replies

4. Shell Programming and Scripting

filter the uniq record problem

Anyone can help for filter the uniq record for below example? Thank you very much Input file 20090503011111|test|abc 20090503011112|tet1|abc|def 20090503011112|test1|bcd|def 20090503011131|abc|abc 20090503011131|bbc|bcd 20090503011152|bcd|abc 20090503011151|abc|abc... (8 Replies)
Discussion started by: bleach8578
8 Replies

5. Shell Programming and Scripting

Keep the last uniq record only

Hi folks, Below is the content of a file 'tmp.dat', and I want to keep the uniq record (key by first column). However, the uniq record should be the last record. 302293022|2|744124889|744124889 302293022|3|744124889|744124889 302293022|4|744124889|744124889 302293022|5|744124889|744124889... (4 Replies)
Discussion started by: ChicagoBlues
4 Replies

6. Shell Programming and Scripting

Need help splitting huge single record file

I was given a data file that I need to split into multiple lines/records based on a key word. The problem is that it is 2.5GB or bigger and everything I try in perl or sed causes a Segmentation fault. Can someone give me some other ideas. The data is of the form:... (5 Replies)
Discussion started by: leolson
5 Replies

7. Shell Programming and Scripting

Removing Dupes from huge file- awk/perl/uniq

Hi, I have the following command in place nawk -F, '!a++' file > file.uniq It has been working perfectly as per requirements, by removing duplicates by taking into consideration only first 3 fields. Recently it has started giving below error: bash-3.2$ nawk -F, '!a++'... (17 Replies)
Discussion started by: makn
17 Replies

8. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

9. Shell Programming and Scripting

Fetching values in CSV file based on column name

input.csv: Field1,Field2,Field3,Field4,Field4 abc ,123 ,xyz ,000 ,pqr mno ,123 ,dfr ,111 ,bbb output: Field2,Field4 123 ,000 123 ,111 how to fetch the values of Field4 where Field2='123' I don't want to fetch the values based on column position. Instead want to... (10 Replies)
Discussion started by: bharathbangalor
10 Replies

10. Shell Programming and Scripting

EBCDIC File Split Based On Record Key

I was wondering if anyone could explain to me how to split a variable length EBCDIC file into seperate files based on the record key. I have the COBOL layout, and so I need to split the file into 13 different EBCDIC files so that I can run each one through a C++ converter I have, and get the... (11 Replies)
Discussion started by: hanshot1stx
11 Replies

LEARN ABOUT MOJAVE

datetime::locale::pa_arab_pk

DateTime::Locale::pa_Arab_PK(3) 			User Contributed Perl Documentation			   DateTime::Locale::pa_Arab_PK(3)

NAME

       DateTime::Locale::pa_Arab_PK

SYNOPSIS

	 use DateTime;

	 my $dt = DateTime->now( locale => 'pa_Arab_PK' );
	 print $dt->month_name();

DESCRIPTION

       This is the DateTime locale package for Punjabi Arabic Pakistan.

DATA

       This locale inherits from the DateTime::Locale::pa_Arab locale.

       It contains the following data.

   Days
       Wide (format)

	 XXX
	 XXXX
	 XXXX
	 XXXXXX
	 XXXX
	 XXXX
	 XXXXX

       Abbreviated (format)

	 XXX
	 XXXX
	 XXXX
	 XXXXXX
	 XXXX
	 XXXX
	 XXXXX

       Narrow (format)

	 XX
	 XX
	 XXX
	 XX
	 XXXX
	 XX
	 X

       Wide (stand-alone)

	 XXX
	 XXXX
	 XXXX
	 XXXXXX
	 XXXX
	 XXXX
	 XXXXX

       Abbreviated (stand-alone)

	 XXX
	 XXXX
	 XXXX
	 XXXXXX
	 XXXX
	 XXXX
	 XXXXX

       Narrow (stand-alone)

	 XX
	 XX
	 XXX
	 XX
	 XXXX
	 XX
	 X

   Months
       Wide (format)

	 XXXXX
	 XXXXX
	 XXXX
	 XXXXX
	 XX
	 XXX
	 XXXXXX
	 XXXX
	 XXXXX
	 XXXXXX
	 XXXXX
	 XXXXX

       Abbreviated (format)

	 XXXXX
	 XXXXX
	 XXXX
	 XXXXX
	 XX
	 XXX
	 XXXXXX
	 XXXX
	 XXXXX
	 XXXXXX
	 XXXXX
	 XXXXX

       Narrow (format)

	 X
	 X
	 XX
	 X
	 X
	 XX
	 XX
	 X
	 X
	 X
	 X
	 X

       Wide (stand-alone)

	 XXXXX
	 XXXXX
	 XXXX
	 XXXXX
	 XX
	 XXX
	 XXXXXX
	 XXXX
	 XXXXX
	 XXXXXX
	 XXXXX
	 XXXXX

       Abbreviated (stand-alone)

	 1
	 2
	 3
	 4
	 5
	 6
	 7
	 8
	 9
	 10
	 11
	 12

       Narrow (stand-alone)

	 X
	 X
	 XX
	 X
	 X
	 XX
	 XX
	 X
	 X
	 X
	 X
	 X

   Quarters
       Wide (format)

	 XXXXXX XXXXX
	 XXXXXX XXXX
	 XXXXXX XXXX
	 XXXXXX XXXXX

       Abbreviated (format)

	 XXXXXX XXXXX
	 XXXXXX XXXX
	 XXXXXX XXXX
	 XXXXXX XXXXX

       Narrow (format)

	 1
	 2
	 3
	 4

       Wide (stand-alone)

	 XXXXXX XXXXX
	 XXXXXX XXXX
	 XXXXXX XXXX
	 XXXXXX XXXXX

       Abbreviated (stand-alone)

	 XXXXXX XXXXX
	 XXXXXX XXXX
	 XXXXXX XXXX
	 XXXXXX XXXXX

       Narrow (stand-alone)

	 1
	 2
	 3
	 4

   Eras
       Wide

	 XXXXXXXX
	 XX

       Abbreviated

	 XXXXXXXX
	 XX

       Narrow

	 XXXXXXXX
	 XX

   Date Formats
       Full

	  2008-02-05T18:30:30 = XXXX, 05 XXXXX 2008
	  1995-12-22T09:05:02 = XXXX, 22 XXXXX 1995
	 -0010-09-15T04:44:23 = XXXX, 15 XXXXX -10

       Long

	  2008-02-05T18:30:30 = 5 XXXXX 2008
	  1995-12-22T09:05:02 = 22 XXXXX 1995
	 -0010-09-15T04:44:23 = 15 XXXXX -10

       Medium

	  2008-02-05T18:30:30 = 5 XXXXX 2008
	  1995-12-22T09:05:02 = 22 XXXXX 1995
	 -0010-09-15T04:44:23 = 15 XXXXX -10

       Short

	  2008-02-05T18:30:30 = 05/02/2008
	  1995-12-22T09:05:02 = 22/12/1995
	 -0010-09-15T04:44:23 = 15/09/-010

       Default

	  2008-02-05T18:30:30 = 5 XXXXX 2008
	  1995-12-22T09:05:02 = 22 XXXXX 1995
	 -0010-09-15T04:44:23 = 15 XXXXX -10

   Time Formats
       Full

	  2008-02-05T18:30:30 = 6:30:30 XXXX UTC
	  1995-12-22T09:05:02 = 9:05:02 XXXXX UTC
	 -0010-09-15T04:44:23 = 4:44:23 XXXXX UTC

       Long

	  2008-02-05T18:30:30 = 6:30:30 XXXX UTC
	  1995-12-22T09:05:02 = 9:05:02 XXXXX UTC
	 -0010-09-15T04:44:23 = 4:44:23 XXXXX UTC

       Medium

	  2008-02-05T18:30:30 = 6:30:30 XXXX
	  1995-12-22T09:05:02 = 9:05:02 XXXXX
	 -0010-09-15T04:44:23 = 4:44:23 XXXXX

       Short

	  2008-02-05T18:30:30 = 6:30 XXXX
	  1995-12-22T09:05:02 = 9:05 XXXXX
	 -0010-09-15T04:44:23 = 4:44 XXXXX

       Default

	  2008-02-05T18:30:30 = 6:30:30 XXXX
	  1995-12-22T09:05:02 = 9:05:02 XXXXX
	 -0010-09-15T04:44:23 = 4:44:23 XXXXX

   Datetime Formats
       Full

	  2008-02-05T18:30:30 = XXXX, 05 XXXXX 2008 6:30:30 XXXX UTC
	  1995-12-22T09:05:02 = XXXX, 22 XXXXX 1995 9:05:02 XXXXX UTC
	 -0010-09-15T04:44:23 = XXXX, 15 XXXXX -10 4:44:23 XXXXX UTC

       Long

	  2008-02-05T18:30:30 = 5 XXXXX 2008 6:30:30 XXXX UTC
	  1995-12-22T09:05:02 = 22 XXXXX 1995 9:05:02 XXXXX UTC
	 -0010-09-15T04:44:23 = 15 XXXXX -10 4:44:23 XXXXX UTC

       Medium

	  2008-02-05T18:30:30 = 5 XXXXX 2008 6:30:30 XXXX
	  1995-12-22T09:05:02 = 22 XXXXX 1995 9:05:02 XXXXX
	 -0010-09-15T04:44:23 = 15 XXXXX -10 4:44:23 XXXXX

       Short

	  2008-02-05T18:30:30 = 05/02/2008 6:30 XXXX
	  1995-12-22T09:05:02 = 22/12/1995 9:05 XXXXX
	 -0010-09-15T04:44:23 = 15/09/-010 4:44 XXXXX

       Default

	  2008-02-05T18:30:30 = 5 XXXXX 2008 6:30:30 XXXX
	  1995-12-22T09:05:02 = 22 XXXXX 1995 9:05:02 XXXXX
	 -0010-09-15T04:44:23 = 15 XXXXX -10 4:44:23 XXXXX

   Available Formats
       d (d)

	  2008-02-05T18:30:30 = 5
	  1995-12-22T09:05:02 = 22
	 -0010-09-15T04:44:23 = 15

       EEEd (d EEE)

	  2008-02-05T18:30:30 = 5 XXXX
	  1995-12-22T09:05:02 = 22 XXXX
	 -0010-09-15T04:44:23 = 15 XXXX

       HHmmss (HH:mm:ss)

	  2008-02-05T18:30:30 = 18:30:30
	  1995-12-22T09:05:02 = 09:05:02
	 -0010-09-15T04:44:23 = 04:44:23

       Hm (H:mm)

	  2008-02-05T18:30:30 = 18:30
	  1995-12-22T09:05:02 = 9:05
	 -0010-09-15T04:44:23 = 4:44

       hm (h:mm a)

	  2008-02-05T18:30:30 = 6:30 XXXX
	  1995-12-22T09:05:02 = 9:05 XXXXX
	 -0010-09-15T04:44:23 = 4:44 XXXXX

       Hms (H:mm:ss)

	  2008-02-05T18:30:30 = 18:30:30
	  1995-12-22T09:05:02 = 9:05:02
	 -0010-09-15T04:44:23 = 4:44:23

       hms (h:mm:ss a)

	  2008-02-05T18:30:30 = 6:30:30 XXXX
	  1995-12-22T09:05:02 = 9:05:02 XXXXX
	 -0010-09-15T04:44:23 = 4:44:23 XXXXX

       M (L)

	  2008-02-05T18:30:30 = 2
	  1995-12-22T09:05:02 = 12
	 -0010-09-15T04:44:23 = 9

       Md (d/M)

	  2008-02-05T18:30:30 = 5/2
	  1995-12-22T09:05:02 = 22/12
	 -0010-09-15T04:44:23 = 15/9

       MEd (E, M-d)

	  2008-02-05T18:30:30 = XXXX, 2-5
	  1995-12-22T09:05:02 = XXXX, 12-22
	 -0010-09-15T04:44:23 = XXXX, 9-15

       MMM (LLL)

	  2008-02-05T18:30:30 = 2
	  1995-12-22T09:05:02 = 12
	 -0010-09-15T04:44:23 = 9

       MMMd (MMM d)

	  2008-02-05T18:30:30 = XXXXX 5
	  1995-12-22T09:05:02 = XXXXX 22
	 -0010-09-15T04:44:23 = XXXXX 15

       MMMEd (E MMM d)

	  2008-02-05T18:30:30 = XXXX XXXXX 5
	  1995-12-22T09:05:02 = XXXX XXXXX 22
	 -0010-09-15T04:44:23 = XXXX XXXXX 15

       MMMMd (MMMM d)

	  2008-02-05T18:30:30 = XXXXX 5
	  1995-12-22T09:05:02 = XXXXX 22
	 -0010-09-15T04:44:23 = XXXXX 15

       MMMMEd (E MMMM d)

	  2008-02-05T18:30:30 = XXXX XXXXX 5
	  1995-12-22T09:05:02 = XXXX XXXXX 22
	 -0010-09-15T04:44:23 = XXXX XXXXX 15

       mmss (mm:ss)

	  2008-02-05T18:30:30 = 30:30
	  1995-12-22T09:05:02 = 05:02
	 -0010-09-15T04:44:23 = 44:23

       ms (mm:ss)

	  2008-02-05T18:30:30 = 30:30
	  1995-12-22T09:05:02 = 05:02
	 -0010-09-15T04:44:23 = 44:23

       y (y)

	  2008-02-05T18:30:30 = 2008
	  1995-12-22T09:05:02 = 1995
	 -0010-09-15T04:44:23 = -10

       yM (y-M)

	  2008-02-05T18:30:30 = 2008-2
	  1995-12-22T09:05:02 = 1995-12
	 -0010-09-15T04:44:23 = -10-9

       yMEd (EEE, y-M-d)

	  2008-02-05T18:30:30 = XXXX, 2008-2-5
	  1995-12-22T09:05:02 = XXXX, 1995-12-22
	 -0010-09-15T04:44:23 = XXXX, -10-9-15

       yMMM (y MMM)

	  2008-02-05T18:30:30 = 2008 XXXXX
	  1995-12-22T09:05:02 = 1995 XXXXX
	 -0010-09-15T04:44:23 = -10 XXXXX

       yMMMEd (EEE, y MMM d)

	  2008-02-05T18:30:30 = XXXX, 2008 XXXXX 5
	  1995-12-22T09:05:02 = XXXX, 1995 XXXXX 22
	 -0010-09-15T04:44:23 = XXXX, -10 XXXXX 15

       yMMMM (y MMMM)

	  2008-02-05T18:30:30 = 2008 XXXXX
	  1995-12-22T09:05:02 = 1995 XXXXX
	 -0010-09-15T04:44:23 = -10 XXXXX

       yQ (y Q)

	  2008-02-05T18:30:30 = 2008 1
	  1995-12-22T09:05:02 = 1995 4
	 -0010-09-15T04:44:23 = -10 3

       yQQQ (y QQQ)

	  2008-02-05T18:30:30 = 2008 XXXXXX XXXXX
	  1995-12-22T09:05:02 = 1995 XXXXXX XXXXX
	 -0010-09-15T04:44:23 = -10 XXXXXX XXXX

       yyMMM (MMM yy)

	  2008-02-05T18:30:30 = XXXXX 08
	  1995-12-22T09:05:02 = XXXXX 95
	 -0010-09-15T04:44:23 = XXXXX -10

       yyQ (Q yy)

	  2008-02-05T18:30:30 = 1 08
	  1995-12-22T09:05:02 = 4 95
	 -0010-09-15T04:44:23 = 3 -10

   Miscellaneous
       Prefers 24 hour time?

       No

       Local first day of the week

       XXXXX

SUPPORT

       See DateTime::Locale.

AUTHOR

       Dave Rolsky <autarch@urth.org>

COPYRIGHT

       Copyright (c) 2008 David Rolsky. All rights reserved. This program is free software; you can redistribute it and/or modify it under the
       same terms as Perl itself.

       This module was generated from data provided by the CLDR project, see the LICENSE.cldr in this distribution for details on the CLDR data's
       license.

perl v5.18.2							    2017-10-06					   DateTime::Locale::pa_Arab_PK(3)
All times are GMT -4. The time now is 09:45 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy