How to copy only some lines from very big file?

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers How to copy only some lines from very big file?
# 1  
Old 02-03-2016
How to copy only some lines from very big file?

Dear all,

I have stuck with this problem for some days.
I have a very big file, this file can not open by vi command.
There are 200 loops in this file, in each loop will have one line like this:

Code:
GWA quasiparticle energy with Z factor (eV)

And I need 98 lines next after this line.

Is any way I can do this with UNIX commands?

Thanks

Last edited by Don Cragun; 02-03-2016 at 06:52 AM.. Reason: Add CODE tags.
# 2  
Old 02-03-2016
If you have latest grep you have below options in grep
Context Line Control
Quote:
-A NUM, --after-context=NUM
Print NUM lines of trailing context after matching lines. Places a line containing a group separator (--) between contiguous
groups of matches. With the -o or --only-matching option, this has no effect and a warning is given.
-B NUM, --before-context=NUM
Print NUM lines of leading context before matching lines. Places a line containing a group separator (--) between contiguous
groups of matches. With the -o or --only-matching option, this has no effect and a warning is given.
-C NUM, -NUM, --context=NUM
Print NUM lines of output context. Places a line containing a group separator (--) between contiguous groups of matches. With
the -o or --only-matching option, this has no effect and a warning is given.
or you can use awk


awk 'c-->0;/pattern/{c=98}' Infile
This User Gave Thanks to vidyadhar85 For This Post:
# 3  
Old 02-03-2016
The grep command did not work in this case.

I have tried the awk command as you suggested:

a
Code:
wk 'c-->0;/GWA quasiparticle energy with Z factor(eV)/{c=98}' ISYS.out >test.txt

but there is nothing in test.txt file.
Could you please check the awk command for me?
Did I write anything wrong?

Last edited by Don Cragun; 02-03-2016 at 06:53 AM.. Reason: Add CODE and ICODE tags.
# 4  
Old 02-03-2016
Hello phamnu,

Could you please try this and let me know if this helps.
Code:
awk '/GWA quasiparticle energy with Z factor \(eV\)/{A=1;while(A<=98){print;getline;A++}}'  Input_file

Above will print the matching line also, if you want to leave that matching line then following may help you in same.
Code:
awk -vnum=98 '/GWA quasiparticle energy with Z factor \(eV\)/{A=1;while(A<98){getline;print;A++}}'  Input_file

Also considering that there are 98 line present after the match into the Input_file. Let me know on same.

Thanks,
R. Singh

Last edited by RavinderSingh13; 02-03-2016 at 01:46 AM..
This User Gave Thanks to RavinderSingh13 For This Post:
# 5  
Old 02-03-2016
Dear Singh,

I have tried both of the commands but still get no results.
My file is something like this:
Code:
....
...
.
.
GWA quasiparticle energy with Z factor (eV)
(I need 98 lines after the above line)
...
...
*many lines after, than again*
GWA quasiparticle energy with Z factor (eV)

Do you have any suggestion?

Thanks,
Nu

Last edited by Don Cragun; 02-03-2016 at 06:54 AM.. Reason: Add CODE tags.
# 6  
Old 02-03-2016
Hello phamnu,

Those above commands worked for me successfully, I tried them in bash environment. There are few questions/suggestions here as follows.
i- It is always good to mention your O.S details into post, in case you are on a Solaris/SunOS system, change awkto /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk.
ii- If it is possible for you to paste sample Input into code tags (you could remove confidential data if any in it).

Let me know if you have any queries on same.

Thanks,
R. Singh
# 7  
Old 02-03-2016
As no samples are available, I have to guess there is something special with the input file. Did you test the suggestions with a simplified (abbreviated) dummy input file, possibly created copying a section of the original? Please post a (small but representative) hexdump of the input. Should look like
Code:
od -tx1c file
0000000  23  65  6d  70  6e  6f  0a  65  6e  61  6d  65  0a  6a  6f  62
          #   e   m   p   n   o  \n   e   n   a   m   e  \n   j   o   b

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copy lines from x to y to another file

OS : RHEL 7.2 Shell : bash I have a file which has lines like below I want to copy from 2nd line to the 6th line and copy(redirect) those lines to another file. $ cat patterns.txt hello world hello asia hello europe hello africa hello america hello antartica hello... (9 Replies)
Discussion started by: omega3
9 Replies

2. Shell Programming and Scripting

Want to extract certain lines from big file

Hi All, I am trying to get some lines from a file i did it with while-do-loop. since the files are huge it is taking much time. now i want to make it faster. The requirement is the file will be having 1 million lines. The format is like below. ##transaction, , , ,blah, blah... (38 Replies)
Discussion started by: mad man
38 Replies

3. UNIX for Dummies Questions & Answers

Copy Files From a big list

Requirement: When I do ls -ltr /home/data/orders I get a huge list of files, I need to copy that last 50 to another directory say /home/work/ later, I will do my ETL process and then again I need to copy from 51 to 100 and so on. What is the command to copy files specifying 1 to 50... (5 Replies)
Discussion started by: eskay
5 Replies

4. UNIX for Advanced & Expert Users

Delete first 100 lines from a BIG File

Hi, I need a unix command to delete first n (say 100) lines from a log file. I need to delete some lines from the file without using any temporary file. I found sed -i is an useful command for this but its not supported in my environment( AIX 6.1 ). File size is approx 100MB. Thanks in... (18 Replies)
Discussion started by: unohu
18 Replies

5. Shell Programming and Scripting

How to copy lines that starts with either 3 or 4 into new file?

Hi Guys, I have an awk script that would search the input file for line that starts with a number 3 and copies into a new text file. I want to extend this script to find the lines that either starts with 3 or a or b and copy all those lines into the new file. Here is what I have so far:... (1 Reply)
Discussion started by: Amith821
1 Replies

6. Shell Programming and Scripting

Re: Deleting lines from big file.

Hi, I have a big (2.7 GB) text file. Each lines has '|' saperator to saperate each columns. I want to delete those lines which has text like '|0|0|0|0|0' I tried: sed '/|0|0|0|0|0/d' test.txt Unfortunately, it scans the file but does nothing. file content sample:... (4 Replies)
Discussion started by: dipeshvshah
4 Replies

7. Shell Programming and Scripting

Print #of lines after search string in a big file

I have a command which prints #lines after and before the search string in the huge file nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r;print;c=a}b{r=$0}' b=0 a=10 s="STRING1" FILE The file is 5 gig big. It works great and prints 10 lines after the lines which contains search string in... (8 Replies)
Discussion started by: prash184u
8 Replies

8. UNIX for Dummies Questions & Answers

How big is too big a config.log file?

I have a 5000 line config.log file with several "maybe" errors. Any reccomendations on finding solvable problems? (2 Replies)
Discussion started by: NeedLotsofHelp
2 Replies

9. Shell Programming and Scripting

Copy only the initial 10 lines from a file to another

Hi all, I'm new to shell scripting. I want to copy initial few lines(say first 10 lines) from a file to another file. There is no "head" command in our embedded system. sed & awk is there which I believe will do that, but I dont know how to. This is linux 2.6 (embedded) So please help me.... (5 Replies)
Discussion started by: jockey007
5 Replies

10. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies
Login or Register to Ask a Question