Selection of a range of lines from a huge file

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to quickly substitute pattern within certain range of a huge file?

I have big files (some are >300GB!) that need substitution for some patterns, for example, change Multiple Spaces into Tab. I used this oneliner:sed '1,18s/ \{1,\}/\t/g' infile_big.sam > outfile_big.sambut it seems very slow as the job is still running after 24 hours! In this example, only the...

2. Shell Programming and Scripting

splitting a huge line of file into multiple lines with fixed number of columns

Hi, I have a huge file with a single line. But I want to break that line into lines of with each line having five columns. My file is like this: code: "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you." I want it like this: code:...

3. Shell Programming and Scripting

how to split a huge file by every 100 lines

into small files. i need to add a head.txt and tail.txt into small files at the begin and end, and give a name as q1.xml q2.xml q3.xml .... thank you very much.

4. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to...

5. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ...

6. Shell Programming and Scripting

NAWK array to store lines from huge file

Hi, I would like to clarify about the NAWK array to store multiple lines from huge file. The file is having an unique REF.NO, I wants to store the lines (it may be 100+ lines) till I found the new REF.NO. How can I apply NAWK - arrays for the above? Rgds, sharif.

7. UNIX for Dummies Questions & Answers

copy and paste certain many lines of huge file in linux

Dear All, I am working with windoes OS but remote a linux machine. I wonder the way to copy an paste some part of a huge file in linux machine. the contain of file like as follow: ... dump annealling all custom 10 anneal_*.dat id type x y z q timestep 0.02 run 200000 Memory...

8. Shell Programming and Scripting

Random lines selection form a file.

>cat data.dat 0001 Robbert 0002 Nick 0003 Mark ....... 1000 Jarek

9. Shell Programming and Scripting

Delete lines from huge file

I have to delete 1st 7000 lines of a file which is 12GB large. As it is so large, i can't open in vi and delete these lines. Also I found one post here which gave solution using perl, but I don't have perl installed. Also some solutions were redirecting the o/p to a different file and renaming it....

10. Shell Programming and Scripting

how to extract a range of lines from a file

I am reading a file that contains over 5000 lines and I want to assign it to a shell variable array (which has a restriction of 1024 rows). I had an idea that if I could grab 1000 record hunks of the file, and pipe the records out, that I could perform a loop until I got to the end and process 1000...

LEARN ABOUT X11R4

free_hugepages

ALLOC_HUGEPAGES(2)					     Linux Programmer's Manual						ALLOC_HUGEPAGES(2)

NAME

       alloc_hugepages, free_hugepages - allocate or free huge pages

SYNOPSIS

       void *alloc_hugepages(int key, void *addr, size_t len,
			     int prot, int flag);

       int free_hugepages(void *addr);

DESCRIPTION

       The  system calls alloc_hugepages() and free_hugepages() were introduced in Linux 2.5.36 and removed again in 2.5.54.  They existed only on
       i386 and ia64 (when built with CONFIG_HUGETLB_PAGE).  In Linux 2.4.20, the syscall numbers exist, but the calls fail with the error ENOSYS.

       On i386 the memory management hardware knows about ordinary pages (4 KiB) and huge pages (2 or 4 MiB).  Similarly  ia64	knows  about  huge
       pages of several sizes.	These system calls serve to map huge pages into the process's memory or to free them again.  Huge pages are locked
       into memory, and are not swapped.

       The key argument is an identifier.  When zero the pages are private, and not inherited by children.  When positive  the	pages  are  shared
       with other applications using the same key, and inherited by child processes.

       The addr argument of free_hugepages() tells which page is being freed: it was the return value of a call to alloc_hugepages().  (The memory
       is first actually freed when all users have released it.)  The addr argument of alloc_hugepages() is a hint, that the kernel may or may not
       follow.	Addresses must be properly aligned.

       The len argument is the length of the required segment.	It must be a multiple of the huge page size.

       The prot argument specifies the memory protection of the segment.  It is one of PROT_READ, PROT_WRITE, PROT_EXEC.

       The  flag  argument  is	ignored, unless key is positive.  In that case, if flag is IPC_CREAT, then a new huge page segment is created when
       none with the given key existed.  If this flag is not set, then ENOENT is returned when no segment with the given key exists.

RETURN VALUE

       On success, alloc_hugepages() returns the allocated virtual address, and free_hugepages() returns zero.	On  error,  -1	is  returned,  and
       errno is set appropriately.

ERRORS

       ENOSYS The system call is not supported on this kernel.

FILES

       /proc/sys/vm/nr_hugepages
	      Number of configured hugetlb pages.  This can be read and written.

       /proc/meminfo
	      Gives  info  on  the  number  of	configured hugetlb pages and on their size in the three variables HugePages_Total, HugePages_Free,
	      Hugepagesize.

CONFORMING TO

       These calls are specific to Linux on Intel processors, and should not be used in programs intended to be portable.

NOTES

       These system calls are gone; they existed only in Linux 2.5.36 through to 2.5.54.  Now the hugetlbfs filesystem can be used instead.   Mem-
       ory backed by huge pages (if the CPU supports them) is obtained by using mmap(2) to map files in this virtual filesystem.

       The maximal number of huge pages can be specified using the hugepages= boot parameter.

COLOPHON

       This  page is part of release 4.15 of the Linux man-pages project.  A description of the project, information about reporting bugs, and the
       latest version of this page, can be found at https://www.kernel.org/doc/man-pages/.

Linux								    2017-09-15							ALLOC_HUGEPAGES(2)

UNIX for Beginners Questions & Answers