Sponsored Content
Top Forums Shell Programming and Scripting Want to extract certain lines from big file Post 302965023 by mad man on Friday 22nd of January 2016 10:25:54 AM
Old 01-22-2016
Want to extract certain lines from big file

Hi All,

I am trying to get some lines from a file i did it with while-do-loop. since the files are huge it is taking much time. now i want to make it faster.

The requirement is the file will be having 1 million lines.
The format is like below.
Code:
##transaction, , , ,blah, blah
%%blah~trannum~blah~blah~blah
0000content01
0001content02
.
.
0010contentnn
0000EOT
##transaction, , , ,blah, blah
%%blah~trannum~blah~blah~blah
0000content01
0001content02
.
.
0010contentnn
0000EOT
##transaction, , , ,blah, blah
%%blah~transnum~blah~blah~blah
0000content01
0001content02
.
.
0010contentnn
0000EOT

What i know from the file is transnum in a set. I want to copy the ##transaction to till the next EOT for the particular transnum.
Also my requirement is from that file i want to copy only one set because my process will know only one transnum only.
So my output file will have only 10 to 15 lines (Only 1 transaction)
So please help me thanks.

Last edited by vbe; 01-22-2016 at 11:58 AM.. Reason: code tags please!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies

2. UNIX for Dummies Questions & Answers

How big is too big a config.log file?

I have a 5000 line config.log file with several "maybe" errors. Any reccomendations on finding solvable problems? (2 Replies)
Discussion started by: NeedLotsofHelp
2 Replies

3. Shell Programming and Scripting

Print #of lines after search string in a big file

I have a command which prints #lines after and before the search string in the huge file nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r;print;c=a}b{r=$0}' b=0 a=10 s="STRING1" FILE The file is 5 gig big. It works great and prints 10 lines after the lines which contains search string in... (8 Replies)
Discussion started by: prash184u
8 Replies

4. Shell Programming and Scripting

Re: Deleting lines from big file.

Hi, I have a big (2.7 GB) text file. Each lines has '|' saperator to saperate each columns. I want to delete those lines which has text like '|0|0|0|0|0' I tried: sed '/|0|0|0|0|0/d' test.txt Unfortunately, it scans the file but does nothing. file content sample:... (4 Replies)
Discussion started by: dipeshvshah
4 Replies

5. Shell Programming and Scripting

Extract some lines from one file and add those lines to current file

hi, i have two files. file1.sh echo "unix" echo "linux" file2.sh echo "unix linux forums" now the output i need is $./file2.sh unix linux forums (3 Replies)
Discussion started by: snreddy_gopu
3 Replies

6. UNIX for Advanced & Expert Users

Delete first 100 lines from a BIG File

Hi, I need a unix command to delete first n (say 100) lines from a log file. I need to delete some lines from the file without using any temporary file. I found sed -i is an useful command for this but its not supported in my environment( AIX 6.1 ). File size is approx 100MB. Thanks in... (18 Replies)
Discussion started by: unohu
18 Replies

7. Shell Programming and Scripting

Extract certain entries from big file:Request to check

Hi all I have a big file which I have attached here. And, I have to fetch certain entries and arrange in 5 columns Name Drug DAP ID disease approved or notIn the attached file data is arranged with tab separated columns in this way: and other data is... (2 Replies)
Discussion started by: manigrover
2 Replies

8. Shell Programming and Scripting

Extract certain columns from big data

The dataset I'm working on is about 450G, with about 7000 colums and 30,000,000 rows. I want to extract about 2000 columns from the original file to form a new file. I have the list of number of the columns I need, but don't know how to extract them. Thanks! (14 Replies)
Discussion started by: happypoker
14 Replies

9. UNIX for Beginners Questions & Answers

How to copy only some lines from very big file?

Dear all, I have stuck with this problem for some days. I have a very big file, this file can not open by vi command. There are 200 loops in this file, in each loop will have one line like this: GWA quasiparticle energy with Z factor (eV) And I need 98 lines next after this line. Is... (6 Replies)
Discussion started by: phamnu
6 Replies

10. Shell Programming and Scripting

Extract Big and continuous regions

Hi all, I have a file like this I want to extract only those regions which are big and continous chr1 3280000 3440000 chr1 3440000 3920000 chr1 3600000 3920000 # region coming within the 3440000 3920000. so i don't want it to be printed in output chr1 3920000 4800000 chr1 ... (2 Replies)
Discussion started by: amrutha_sastry
2 Replies
Wx::Perl::DataWalker(3pm)				User Contributed Perl Documentation				 Wx::Perl::DataWalker(3pm)

NAME
Wx::Perl::DataWalker - Perl extension for blah blah blah SYNOPSIS
# alternatively, see the dataWalker.pl script in this distribution! use Wx::Perl::DataWalker; my $data_to_browse = { # complex data structure. # Good test: \%:: }; package MyWalkerDialog; our @ISA = qw(Wx::App); sub OnInit { my $self = shift; my $frame = Wx::Perl::DataWalker->new( {data => $data_to_browse}, undef, -1, "dataWalker", ); $self->SetTopWindow($frame); $frame->Show(1); $frame->SetSize(500,500); return 1; } package main; my $app = MyWalkerDialog->new(); $app->MainLoop(); DESCRIPTION
"Wx::Perl::DataWalker" implements a "Wx::Frame" subclass that shows a relatively simple Perl data structure browser. After opening such a frame and supplying it with a reference to an essentially arbitrary data structure, you can visually browse it by double-clicking references. So far, there is no tree view but only a display of the current level of the data structure. You can traverse back up the structure with a back button. Optionally, "Wx::Perl::DataWalker" displays the (approximate!) size of the data structure using "Devel::Size". SEE ALSO
Wx, Devel::Size Padre::Plugin::DataWalker AUTHOR
Steffen Mueller, <smueller@cpan.org> COPYRIGHT AND LICENSE
Copyright (C) 2009 by Steffen Mueller This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8 or, at your option, any later version of Perl 5 you may have available. perl v5.10.0 2009-03-29 Wx::Perl::DataWalker(3pm)
All times are GMT -4. The time now is 02:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy