Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to copy only some lines from very big file?
Top Forums UNIX for Beginners Questions & Answers How to copy only some lines from very big file? Post 302965779 by RavinderSingh13 on Wednesday 3rd of February 2016 02:05:10 AM
Old 02-03-2016
Hello phamnu,

Those above commands worked for me successfully, I tried them in bash environment. There are few questions/suggestions here as follows.
i- It is always good to mention your O.S details into post, in case you are on a Solaris/SunOS system, change awkto /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk.
ii- If it is possible for you to paste sample Input into code tags (you could remove confidential data if any in it).

Let me know if you have any queries on same.

Thanks,
R. Singh
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies

2. Shell Programming and Scripting

Copy only the initial 10 lines from a file to another

Hi all, I'm new to shell scripting. I want to copy initial few lines(say first 10 lines) from a file to another file. There is no "head" command in our embedded system. sed & awk is there which I believe will do that, but I dont know how to. This is linux 2.6 (embedded) So please help me.... (5 Replies)
Discussion started by: jockey007
5 Replies

3. UNIX for Dummies Questions & Answers

How big is too big a config.log file?

I have a 5000 line config.log file with several "maybe" errors. Any reccomendations on finding solvable problems? (2 Replies)
Discussion started by: NeedLotsofHelp
2 Replies

4. Shell Programming and Scripting

Print #of lines after search string in a big file

I have a command which prints #lines after and before the search string in the huge file nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r;print;c=a}b{r=$0}' b=0 a=10 s="STRING1" FILE The file is 5 gig big. It works great and prints 10 lines after the lines which contains search string in... (8 Replies)
Discussion started by: prash184u
8 Replies

5. Shell Programming and Scripting

Re: Deleting lines from big file.

Hi, I have a big (2.7 GB) text file. Each lines has '|' saperator to saperate each columns. I want to delete those lines which has text like '|0|0|0|0|0' I tried: sed '/|0|0|0|0|0/d' test.txt Unfortunately, it scans the file but does nothing. file content sample:... (4 Replies)
Discussion started by: dipeshvshah
4 Replies

6. Shell Programming and Scripting

How to copy lines that starts with either 3 or 4 into new file?

Hi Guys, I have an awk script that would search the input file for line that starts with a number 3 and copies into a new text file. I want to extend this script to find the lines that either starts with 3 or a or b and copy all those lines into the new file. Here is what I have so far:... (1 Reply)
Discussion started by: Amith821
1 Replies

7. UNIX for Advanced & Expert Users

Delete first 100 lines from a BIG File

Hi, I need a unix command to delete first n (say 100) lines from a log file. I need to delete some lines from the file without using any temporary file. I found sed -i is an useful command for this but its not supported in my environment( AIX 6.1 ). File size is approx 100MB. Thanks in... (18 Replies)
Discussion started by: unohu
18 Replies

8. UNIX for Dummies Questions & Answers

Copy Files From a big list

Requirement: When I do ls -ltr /home/data/orders I get a huge list of files, I need to copy that last 50 to another directory say /home/work/ later, I will do my ETL process and then again I need to copy from 51 to 100 and so on. What is the command to copy files specifying 1 to 50... (5 Replies)
Discussion started by: eskay
5 Replies

9. Shell Programming and Scripting

Want to extract certain lines from big file

Hi All, I am trying to get some lines from a file i did it with while-do-loop. since the files are huge it is taking much time. now i want to make it faster. The requirement is the file will be having 1 million lines. The format is like below. ##transaction, , , ,blah, blah... (38 Replies)
Discussion started by: mad man
38 Replies

10. Shell Programming and Scripting

Copy lines from x to y to another file

OS : RHEL 7.2 Shell : bash I have a file which has lines like below I want to copy from 2nd line to the 6th line and copy(redirect) those lines to another file. $ cat patterns.txt hello world hello asia hello europe hello africa hello america hello antartica hello... (9 Replies)
Discussion started by: omega3
9 Replies

LEARN ABOUT SUNOS

largefile

largefile(5)						Standards, Environments, and Macros					      largefile(5)

NAME

       largefile - large file status of utilities

DESCRIPTION

       A  large file is a regular file whose size is greater than or equal to 2 Gbyte ( 2**31 bytes). A small file is a regular file whose size is
       less than 2 Gbyte.

   Large file aware utilities
       A utility is called large file aware if it can process large files in the same manner as it does small files. A utility that is large  file
       aware is able to handle large files as input and generate as output large files that are being processed. The exception is where additional
       files  are used as system configuration files or support files that can augment the processing. For example, the file utility supports  the
       -m  option  for	an  alternative "magic" file and the -f option for a support file that can contain a list of file names. It is unspecified
       whether a utility that is large file aware will accept configuration or support files that are large files. If a large file  aware  utility
       does  not accept configuration or support files that are large files, it will cause no data loss or corruption upon encountering such files
       and will return an appropriate error.

       The following /usr/bin utilities are large file aware:

       adb	     awk	   bdiff	 cat	       chgrp
       chmod	     chown	   cksum	 cmp	       compress
       cp	     csh	   csplit	 cut	       dd
       dircmp	     du 	   egrep	 fgrep	       file
       find	     ftp	   getconf	 grep	       gzip
       head	     join	   jsh		 ksh	       ln
       ls	     mdb	   mkdir	 mkfifo        more
       mv	     nawk	   page 	 paste	       pathchck
       pg	     rcp	   remsh	 rksh	       rm
       rmdir	     rsh	   sed		 sh	       sort
       split	     sum	   tail 	 tar	       tee
       test	     touch	   tr		 uncompress    uudecode
       uuencode      wc 	   zcat

       The following /usr/xpg4/bin utilities are large file aware:

       awk	     cp 	   chgrp	 chown	       du
       egrep	     fgrep	   file 	 grep	       ln
       ls	     more	   mv		 rm	       sed
       sh	     sort	   tail 	 tr

       The following /usr/xpg6/bin utilities are large file aware:

       getconf	     ls 	   tr

       The following /usr/sbin utilities are large file aware:

       install	     mkfile	   mknod	 mvdir	       swap

       See the USAGE section of the swap(1M) manual page for limitations of swap on block devices greater than 2 Gbyte on a 32-bit operating  sys-
       tem.

       The following /usr/ucb utilities are large file aware:

       chown	     from	   ln		 ls	       sed
       sum	     touch

       The /usr/bin/cpio and /usr/bin/pax utilities are large file aware, but cannot archive a file whose size exceeds 8 Gbyte - 1 byte.

       The /usr/bin/truss utilities has been modified to read a dump file and display information relevant to large files, such as offsets.

   cachefs file systems
       The following /usr/bin utilities are large file aware for cachefs file systems:

       cachefspack	cachefsstat

       The following /usr/sbin utilities are large file aware for cachefs file systems:

       cachefslog	cachefswssize	cfsadmin	 fsck
       mount		umount

   nfs file systems
       The following utilities are large file aware for nfs file systems:

       /usr/lib/autofs/automountd    /usr/sbin/mount
       /usr/lib/nfs/rquotad

   ufs file systems
       The following /usr/bin utility is large file aware for ufs file systems:

	      df

       The following /usr/lib/nfs utility is large file aware for ufs file systems:

	      rquotad

       The following /usr/xpg4/bin utility is large file aware for ufs file systems:

	      df

       The following /usr/sbin utilities are large file aware for ufs file systems:

       clri	     dcopy	   edquota	 ff	       fsck
       fsdb	     fsirand	   fstyp	 labelit       lockfs
       mkfs	     mount	   ncheck	 newfs	       quot
       quota	     quotacheck    quotaoff	 quotaon       repquota
       tunefs	     ufsdump	   ufsrestore	 umount

   Large file safe utilities
       A  utility  is called large file safe if it causes no data loss or corruption when it encounters a large file. A utility that is large file
       safe is unable to process properly a large file, but returns an appropriate error.

       The following /usr/bin utilities are large file safe:

       audioconvert	audioplay    audiorecord    comm	  diff
       diff3		diffmk	     ed 	    lp		  mail

       mailcompat	mailstats    mailx	    pack	  pcat
       red		rmail	     sdiff	    unpack	  vi
       view

       The following /usr/xpg4/bin utilities are large file safe:

       ed	     vi 	   view

       The following /usr/xpg6/bin utility is large file safe:

       ed

       The following /usr/sbin utilities are large file safe:

       lpfilter      lpforms

       The following /usr/ucb utilities are large file safe:

       Mail	     lpr

       The following /usr/lib utility is large file safe:

	      sendmail

SEE ALSO

       lf64(5), lfcompile(5), lfcompile64(5)

SunOS 5.10							    7 Nov 2003							      largefile(5)
All times are GMT -4. The time now is 01:40 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy