10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have big files (some are >300GB!) that need substitution for some patterns, for example, change Multiple Spaces into Tab. I used this oneliner:sed '1,18s/ \{1,\}/\t/g' infile_big.sam > outfile_big.sambut it seems very slow as the job is still running after 24 hours! In this example, only the... (8 Replies)
Discussion started by: yifangt
8 Replies
2. Shell Programming and Scripting
Hi,
I have a huge file with a single line.
But I want to break that line into lines of with each line having five columns.
My file is like this:
code:
"hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you."
I want it like this:
code:... (1 Reply)
Discussion started by: rajsharma
1 Replies
3. Shell Programming and Scripting
into small files. i need to add a head.txt and tail.txt into small files at the begin and end, and give a name as q1.xml q2.xml q3.xml ....
thank you very much. (2 Replies)
Discussion started by: dtdt
2 Replies
4. UNIX for Advanced & Expert Users
Hi All,
I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space.
I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies
5. Shell Programming and Scripting
Hi,
I have one file, say file 1, that has data like below where 19900107 is the date,
19900107 12 144 129 0.7380047
19900108 12 168 129 0.3149017
19900109 12 192 129 3.2766666E-02
... (3 Replies)
Discussion started by: Wynner
3 Replies
6. Shell Programming and Scripting
Hi,
I would like to clarify about the NAWK array to store multiple lines from huge file.
The file is having an unique REF.NO, I wants to store the lines (it may be
100+ lines) till I found the new REF.NO.
How can I apply NAWK - arrays for the above?
Rgds,
sharif. (1 Reply)
Discussion started by: sharif
1 Replies
7. UNIX for Dummies Questions & Answers
Dear All,
I am working with windoes OS but remote a linux machine. I wonder the way to copy an paste some part of a huge file in linux machine.
the contain of file like as follow:
...
dump annealling all custom 10 anneal_*.dat id type x y z q
timestep 0.02
run 200000
Memory... (2 Replies)
Discussion started by: ariesto
2 Replies
8. Shell Programming and Scripting
>cat data.dat
0001 Robbert
0002 Nick
0003 Mark
.......
1000 Jarek (3 Replies)
Discussion started by: McLan
3 Replies
9. Shell Programming and Scripting
I have to delete 1st 7000 lines of a file which is 12GB large. As it is so large, i can't open in vi and delete these lines. Also I found one post here which gave solution using perl, but I don't have perl installed. Also some solutions were redirecting the o/p to a different file and renaming it.... (3 Replies)
Discussion started by: rahulrathod
3 Replies
10. Shell Programming and Scripting
I am reading a file that contains over 5000 lines and I want to assign it to a shell variable array (which has a restriction of 1024 rows). I had an idea that if I could grab 1000 record hunks of the file, and pipe the records out, that I could perform a loop until I got to the end and process 1000... (5 Replies)
Discussion started by: beilstwh
5 Replies
ALLOC_HUGEPAGES(2) Linux Programmer's Manual ALLOC_HUGEPAGES(2)
NAME
alloc_hugepages, free_hugepages - allocate or free huge pages
SYNOPSIS
void *alloc_hugepages(int key, void *addr, size_t len,
int prot, int flag);
int free_hugepages(void *addr);
DESCRIPTION
The system calls alloc_hugepages() and free_hugepages() were introduced in Linux 2.5.36 and removed again in 2.5.54. They existed only on
i386 and ia64 (when built with CONFIG_HUGETLB_PAGE). In Linux 2.4.20 the syscall numbers exist, but the calls fail with the error ENOSYS.
On i386 the memory management hardware knows about ordinary pages (4 KiB) and huge pages (2 or 4 MiB). Similarly ia64 knows about huge
pages of several sizes. These system calls serve to map huge pages into the process's memory or to free them again. Huge pages are locked
into memory, and are not swapped.
The key argument is an identifier. When zero the pages are private, and not inherited by children. When positive the pages are shared
with other applications using the same key, and inherited by child processes.
The addr argument of free_hugepages() tells which page is being freed: it was the return value of a call to alloc_hugepages(). (The memory
is first actually freed when all users have released it.) The addr argument of alloc_hugepages() is a hint, that the kernel may or may not
follow. Addresses must be properly aligned.
The len argument is the length of the required segment. It must be a multiple of the huge page size.
The prot argument specifies the memory protection of the segment. It is one of PROT_READ, PROT_WRITE, PROT_EXEC.
The flag argument is ignored, unless key is positive. In that case, if flag is IPC_CREAT, then a new huge page segment is created when
none with the given key existed. If this flag is not set, then ENOENT is returned when no segment with the given key exists.
RETURN VALUE
On success, alloc_hugepages() returns the allocated virtual address, and free_hugepages() returns zero. On error, -1 is returned, and
errno is set appropriately.
ERRORS
ENOSYS The system call is not supported on this kernel.
FILES
/proc/sys/vm/nr_hugepages Number of configured hugetlb pages. This can be read and written.
/proc/meminfo Gives info on the number of configured hugetlb pages and on their size in the three variables HugePages_Total,
HugePages_Free, Hugepagesize.
CONFORMING TO
These calls are specific to Linux on Intel processors, and should not be used in programs intended to be portable.
NOTES
These system calls are gone; they existed only in Linux 2.5.36 through to 2.5.54. Now the hugetlbfs file system can be used instead. Mem-
ory backed by huge pages (if the CPU supports them) is obtained by using mmap(2) to map files in this virtual file system.
The maximal number of huge pages can be specified using the hugepages= boot parameter.
COLOPHON
This page is part of release 3.27 of the Linux man-pages project. A description of the project, and information about reporting bugs, can
be found at http://www.kernel.org/doc/man-pages/.
Linux 2007-05-31 ALLOC_HUGEPAGES(2)