Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Fetching record based on Uniq Key from huge file.
Top Forums Shell Programming and Scripting Fetching record based on Uniq Key from huge file. Post 302717371 by lathigara on Thursday 18th of October 2012 05:04:21 AM
Old 10-18-2012
output will be 100k records.

The one which i pinged is one record and i want to fetch 100k record from a file having 1000k file.


Code:
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
                              EMDS/CREDIT SERVICES                     RunDate: 10/25/2011                    REC#      1
             SEQUENCE NUMBER: 06000000001           L90 KEY :060882D85800E201684C0000            CID KEY:000000000000000000
DEBT RATIO=  0%  INQ W/I 6=  0, W/I 12=  0, W/I 18=  0, W/I 24=  0 ,PRM W/I 6=0; Trades RPTed W/I 24 =  0, No. of Trades =   3
*** FACT FRAUD CODE =  ,FACT ALERT DATA CODE = 
XXXXXX                    XXXXX                                                                   SSN-XXX-XX-XXXX C
XXXXXX,XXXXX.                                                                            FAD:  0/00/0000  FILE SINCE:  0/00/0000
XXXXX XXXXXXXX XX  XXXXXXXCA,92236
,,,,,,,,0
,,,,,,,,0
,,,,,,,,0
 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
                              EMDS/CREDIT SERVICES                     RunDate: 10/25/2011                    REC#      2
             SEQUENCE NUMBER: 06000000002           L90 KEY :060882D85800E201684C0000            CID KEY:000000000000000000
DEBT RATIO=  0%  INQ W/I 6=  0, W/I 12=  0, W/I 18=  0, W/I 24=  0 ,PRM W/I 6=0; Trades RPTed W/I 24 =  0, No. of Trades =   3
*** FACT FRAUD CODE =  ,FACT ALERT DATA CODE = 
XXXXXX                    XXXXX                                                                   SSN-XXX-XX-XXXX C
XXXXXX,XXXXX.                                                                            FAD:  0/00/0000  FILE SINCE:  0/00/0000
XXXXX XXXXXXXX XX  XXXXXXXCA,92236
,,,,,,,,0
,,,,,,,,0
,,,,,,,,0

like 100k.

thanks,
prashant
Last edited by Scrutinizer; 10-18-2012 at 06:45 AM.. Reason: code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Parsing out records from one huge record

Hi, I have one huge record and know that each record in the file is 550 bytes long. How do I parse out individual records from the single huge record. Thanks, (4 Replies)
Discussion started by: bwrynz1
4 Replies

2. Shell Programming and Scripting

Compare 2 huge files wrt to a key using awk

Hi Folks, I need to compare two very huge file ( i.e the files would contain a minimum of 70k records each) using awk or sed. The comparison needs to be done with respect to a 'key'. For example : File1 ********** 1234|TONY|Y75634|20/07/2008 1235|TINA|XCVB56|30/07/2009... (13 Replies)
Discussion started by: Ranjani
13 Replies

3. Shell Programming and Scripting

Logic for file fetching based on date

Dear friends, I receive the following files into a FTP location on a daily basis -rw-r----- 1 guest ftp1 5021 Aug 19 09:03 CHECK_TEST_Extracts_20080818210000.zip -rw-r----- 1 guest ftp1 2437 Aug 20 05:15 CHECK_TEST_Extracts_20080819210000.zip -rw-r----- 1 guest ... (2 Replies)
Discussion started by: sureshg_sampat
2 Replies

4. Shell Programming and Scripting

filter the uniq record problem

Anyone can help for filter the uniq record for below example? Thank you very much Input file 20090503011111|test|abc 20090503011112|tet1|abc|def 20090503011112|test1|bcd|def 20090503011131|abc|abc 20090503011131|bbc|bcd 20090503011152|bcd|abc 20090503011151|abc|abc... (8 Replies)
Discussion started by: bleach8578
8 Replies

5. Shell Programming and Scripting

Keep the last uniq record only

Hi folks, Below is the content of a file 'tmp.dat', and I want to keep the uniq record (key by first column). However, the uniq record should be the last record. 302293022|2|744124889|744124889 302293022|3|744124889|744124889 302293022|4|744124889|744124889 302293022|5|744124889|744124889... (4 Replies)
Discussion started by: ChicagoBlues
4 Replies

6. Shell Programming and Scripting

Need help splitting huge single record file

I was given a data file that I need to split into multiple lines/records based on a key word. The problem is that it is 2.5GB or bigger and everything I try in perl or sed causes a Segmentation fault. Can someone give me some other ideas. The data is of the form:... (5 Replies)
Discussion started by: leolson
5 Replies

7. Shell Programming and Scripting

Removing Dupes from huge file- awk/perl/uniq

Hi, I have the following command in place nawk -F, '!a++' file > file.uniq It has been working perfectly as per requirements, by removing duplicates by taking into consideration only first 3 fields. Recently it has started giving below error: bash-3.2$ nawk -F, '!a++'... (17 Replies)
Discussion started by: makn
17 Replies

8. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

9. Shell Programming and Scripting

Fetching values in CSV file based on column name

input.csv: Field1,Field2,Field3,Field4,Field4 abc ,123 ,xyz ,000 ,pqr mno ,123 ,dfr ,111 ,bbb output: Field2,Field4 123 ,000 123 ,111 how to fetch the values of Field4 where Field2='123' I don't want to fetch the values based on column position. Instead want to... (10 Replies)
Discussion started by: bharathbangalor
10 Replies

10. Shell Programming and Scripting

EBCDIC File Split Based On Record Key

I was wondering if anyone could explain to me how to split a variable length EBCDIC file into seperate files based on the record key. I have the COBOL layout, and so I need to split the file into 13 different EBCDIC files so that I can run each one through a C++ converter I have, and get the... (11 Replies)
Discussion started by: hanshot1stx
11 Replies

LEARN ABOUT SUNOS

script

script(1)							   User Commands							 script(1)

NAME

       script - make record of a terminal session

SYNOPSIS

       script [-a] [filename]

DESCRIPTION

       The  script  utility  makes a record of everything printed on your screen. The record is written to filename. If no file name is given, the
       record is saved in the file typescript. See WARNINGS.

       The script command forks and creates a sub-shell, according to the value of $SHELL, and records the text from this session. The script ends
       when the forked shell exits or when Control-d is typed.

OPTIONS

       The following option is supported:

       -a	Appends the session record to filename, rather than overwriting it.

NOTES

       script places everything that appears on the screen in filename, including prompts.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWcsu			   |
       +-----------------------------+-----------------------------+
       |CSI			     |Enabled			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       attributes(5)

WARNINGS

       script  can  pose  a  security risk when used in directories that are writable by other users (for example, /tmp), especially when run by a
       privileged user, that is, root. Be sure that typescript is not a link before running script.

SunOS 5.10							    30 Jan 2004 							 script(1)
All times are GMT -4. The time now is 09:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy