Sponsored Content
Top Forums Shell Programming and Scripting Fetching record based on Uniq Key from huge file. Post 302718047 by lathigara on Friday 19th of October 2012 04:16:32 AM
Old 10-19-2012
Hi sorry if you are not understanding or I interpreted wrongly my qst.

Please I am repeating my qst.
The file is like as below.
There is one file which contains almost 1000K records.
All records are SEQUENCE NUMBER separated.
All SEQUENCE NUMBER is Uniq for that file.
There might be 0 lines or 1000 line in-between two SEQUENCE NUMBER means one SEQUENCE NUMBER: can contain some line of sub records inside it or no line of sub records.

My requirement is I want to fetch some SEQUENCE NUMBER: from that file like from SEQUENCE NUMBER: 56000000001 to 56000000005 (fetching 5 records or fetching 100K records based on SEQUENCE NUMBER).

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
1 xxxxxxxxxxxxx xxxxxxxx#1000215 xxxxx REC# 3 PAGE: 1
xxxxxxx xxxxxxA SEQUENCE NUMBER: 56000000001 CID NUMBER: 06000000001
1 xxxxxxxxxxxxx xxxxxxxx#1000215 xxxxx REC# 3 PAGE: 2
SEQUENCE NUMBER: 56000000001(CONT)
0*************************************************************************************************** *********************************
* **** xxxxxxxxxxxxxxxxxxxxxx **** *
**************************************************************************************************** ********************************
1 xxxxxxxxxxxxx xxxxxxxx#1000215 xxxxx REC# 3 PAGE: 2
xxxxxxx xxxxxxA SEQUENCE NUMBER: 56000000002 CID NUMBER: 06000000002
1 xxxxxxxxxxxxx xxxxxxxx#1000215 xxxxxx REC# 3 PAGE: 2
SEQUENCE NUMBER: 56000000002(CONT)
0*************************************************************************************************** *********************************
* **** xxxxxxxxxxxxxxxxxxxxxx **** *
**************************************************************************************************** ********************************
1 xxxxxxxxxxxxx xxxxxxxx#1000215 xxxxx REC# 3 PAGE: 2
xxxxxxx xxxxxxA SEQUENCE NUMBER: 56000000003 CID NUMBER: 06000000003
1 xxxxxxxxxxxxx xxxxxxxx#1000215 xxxxx REC# 3 PAGE: 2
SEQUENCE NUMBER: 56000000003(CONT)
0*************************************************************************************************** *********************************
* **** xxxxxxxxxxxxxxxxxxxxxx **** *
**************************************************************************************************** ********************************
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Parsing out records from one huge record

Hi, I have one huge record and know that each record in the file is 550 bytes long. How do I parse out individual records from the single huge record. Thanks, (4 Replies)
Discussion started by: bwrynz1
4 Replies

2. Shell Programming and Scripting

Compare 2 huge files wrt to a key using awk

Hi Folks, I need to compare two very huge file ( i.e the files would contain a minimum of 70k records each) using awk or sed. The comparison needs to be done with respect to a 'key'. For example : File1 ********** 1234|TONY|Y75634|20/07/2008 1235|TINA|XCVB56|30/07/2009... (13 Replies)
Discussion started by: Ranjani
13 Replies

3. Shell Programming and Scripting

Logic for file fetching based on date

Dear friends, I receive the following files into a FTP location on a daily basis -rw-r----- 1 guest ftp1 5021 Aug 19 09:03 CHECK_TEST_Extracts_20080818210000.zip -rw-r----- 1 guest ftp1 2437 Aug 20 05:15 CHECK_TEST_Extracts_20080819210000.zip -rw-r----- 1 guest ... (2 Replies)
Discussion started by: sureshg_sampat
2 Replies

4. Shell Programming and Scripting

filter the uniq record problem

Anyone can help for filter the uniq record for below example? Thank you very much Input file 20090503011111|test|abc 20090503011112|tet1|abc|def 20090503011112|test1|bcd|def 20090503011131|abc|abc 20090503011131|bbc|bcd 20090503011152|bcd|abc 20090503011151|abc|abc... (8 Replies)
Discussion started by: bleach8578
8 Replies

5. Shell Programming and Scripting

Keep the last uniq record only

Hi folks, Below is the content of a file 'tmp.dat', and I want to keep the uniq record (key by first column). However, the uniq record should be the last record. 302293022|2|744124889|744124889 302293022|3|744124889|744124889 302293022|4|744124889|744124889 302293022|5|744124889|744124889... (4 Replies)
Discussion started by: ChicagoBlues
4 Replies

6. Shell Programming and Scripting

Need help splitting huge single record file

I was given a data file that I need to split into multiple lines/records based on a key word. The problem is that it is 2.5GB or bigger and everything I try in perl or sed causes a Segmentation fault. Can someone give me some other ideas. The data is of the form:... (5 Replies)
Discussion started by: leolson
5 Replies

7. Shell Programming and Scripting

Removing Dupes from huge file- awk/perl/uniq

Hi, I have the following command in place nawk -F, '!a++' file > file.uniq It has been working perfectly as per requirements, by removing duplicates by taking into consideration only first 3 fields. Recently it has started giving below error: bash-3.2$ nawk -F, '!a++'... (17 Replies)
Discussion started by: makn
17 Replies

8. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

9. Shell Programming and Scripting

Fetching values in CSV file based on column name

input.csv: Field1,Field2,Field3,Field4,Field4 abc ,123 ,xyz ,000 ,pqr mno ,123 ,dfr ,111 ,bbb output: Field2,Field4 123 ,000 123 ,111 how to fetch the values of Field4 where Field2='123' I don't want to fetch the values based on column position. Instead want to... (10 Replies)
Discussion started by: bharathbangalor
10 Replies

10. Shell Programming and Scripting

EBCDIC File Split Based On Record Key

I was wondering if anyone could explain to me how to split a variable length EBCDIC file into seperate files based on the record key. I have the COBOL layout, and so I need to split the file into 13 different EBCDIC files so that I can run each one through a C++ converter I have, and get the... (11 Replies)
Discussion started by: hanshot1stx
11 Replies
XOSD(1) 						      General Commands Manual							   XOSD(1)

NAME
osd_cat - X on-screen file displayer SYNOPSIS
osd_cat [OPTION] [FILE]... osd_cat -b percentage|slider [OPTION] DESCRIPTION
Display FILE, or standard input, on X screen. -p, --pos=POS This option tells osd_cat where to display the text. POS can be top, middle, or bottom. The default is top. -o, --offset=OFFSET This option specifies the offset from the top or bottom of screen the text is displayed. The default is 0. -A, --align=ALIGN This option tells osd_cat where to display the text. ALIGN can be left, right or center. The default is left. -i, --indent=OFFSET This option specifies the INDENT from the left of screen the text is displayed. The default is 0. -f, --font=FONT This option specifies the FONT to be used for displaying the text. The default is fixed. -c, --color=COLOR This option specifies the COLOR to be used for displaying the text. The default is red. -d, --delay=TIME This option specifies the number of seconds the text is displayed. The default is 5 seconds. -l, --lines=LINES This option specifies the number of LINES to scroll the display over. The default is 5. -s, --shadow=OFFSET This option specifies the OFFSET of the text shadow. The default is 0, which means no text shadow is created. -S, --shadowcolour=COLOUR This option specifies the COLOUR of the text shadow. The default is black. -O, --outline=WIDTH This option specifies the WIDTH of the text outline. The default is 0, which is no outline. -u, --outlinecolour=COLOUR This option specifies the COLOUR of the text outline. The default is black. -a, --age[=SCROLL_AGE] This option affects screen redrawing. If SCROLL_AGE seconds pass before a new line is ready (for example, you're reading from a pipe), all lines are cleared at once instead of being scrolled off as new lines replace old lines. The default is 0. When no SCROLL_AGE is explicitly given, the current value from DELAY is used. -w, --wait This option also affects screen redrawing. When there is data ready to be put on screen, this option will cause osd_cat to wait until the display is clear. An alternative to scrolling. -b, --barmode=TYPE Lets you display a percentage or slider bar instead of just text. TYPE may be percentage or slider. In this mode no text is read from any file, but the following options can be used: -P, --percentage=PERCENTAGE This option specified the position of the percentage / slider bar. PERCENTAGE may be in the range from 0 to 100, the default is 50. -T, --text=TEXT This option specifies an optional TEXT which gets displayed above the percentage bar. The default is empty, so no additional text is displayed. -h, --help display help (which is often more up to date) and exit With no FILE, or when FILE is -, read standard input. AUTHOR
Martijn van de Streek <martijn@foodfight.org>, Some patching done by Malcolm Valentine <farkit@iprimus.com.au> and Tim Wright <tim@ignavus.net>. xosd was written by Andre Renaud <andre@ignavus.net> and is maintained by Tim Wright <tim@ignavus.net> SEE ALSO
More information on the X OSD Library and its author can be found on http://www.ignavus.net/software.html <http://www.ignavus.net/software.html> COPYRIGHT
It is distributed under the GNU General Public License. X OSD cat January 2001 XOSD(1)
All times are GMT -4. The time now is 06:10 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy