Sponsored Content
Top Forums Shell Programming and Scripting Breaking large file into small files Post 302937408 by gandolf989 on Thursday 5th of March 2015 11:09:17 AM
Old 03-05-2015
If each entry is a line and you want 10,000 lines try using the
head and tail commands to chop up the file in 10,000 line chunks.
You can probably put it in a for loop with the wc command to see
how many lines are in the file, hence the maximum number to extract.

Code:
head -10000 file_name.txt               > file1.txt
head -20000 file_name.txt | tail -10000 > file2.txt
head -30000 file_name.txt | tail -20000 > file2.txt
head -40000 file_name.txt | tail -30000 > file2.txt

This User Gave Thanks to gandolf989 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting large file into small files

Hi, I need to split a large file into small files based on a string. At different palces in the large I have the string ^Job. I need to split the file into different files starting from ^Job to the last character before the next ^Job. Also all the small files should be automatically named.... (4 Replies)
Discussion started by: dncs
4 Replies

2. Shell Programming and Scripting

Split a file into 16 small files

Hi I want to split a file that has 'n' number of records into 16 small files. Can some one suggest me how to do this using Unix script? Thanks rrkk (10 Replies)
Discussion started by: rrkks
10 Replies

3. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies

4. Shell Programming and Scripting

script to splite large file to number of small files

Dear All, Could you please help me to split a file contain around 240,000,000 line to 4 files all equally likely , note that we need to maintain that the end of each file should started by start flage (MSISDN) and ended by end flag (End), also the number of the line between the... (10 Replies)
Discussion started by: ahmed.gad
10 Replies

5. Shell Programming and Scripting

Breaking one file into many files based on first column?

Hi, I have a file that looks like this (tab deliminited). MAT1 YKR2 3 MAT1 YMR1 2 MAT1 YFG2 2 MAT2 YLM4 4 MAT2 YHL2 1 BAR1 YKR2 3 BAR1 YFR1 4 BAR1 YMR1 1 What I want to do is break this file down into multiple files. So the result will look like this: File 1... (2 Replies)
Discussion started by: kylle345
2 Replies

6. Shell Programming and Scripting

Breaking the files as 10k recs. per file

Hi, I have a code as given below Set -A _Category="A\ B\ C" for _cat in ${_Category} do sed -e "s:<TABLE_NAME>:${_cat}:g" \ -e "s:<date>:${_dt}:g" \ ${_home}/skl/sq1.sql >> ${_dest}/del_${_dt}.sql fi ... (4 Replies)
Discussion started by: mr_manii
4 Replies

7. UNIX for Advanced & Expert Users

Splitting a file into small files

Hi Folks, Please help me in solving the problem. I want to write script in order to split a file into small pieces and send it automatically through mail. Ex. The file name is CALM*.txt . It is around 50 MB. I want to split the file into 20 MB 2-3 smaller files and send (like uuencode) it... (6 Replies)
Discussion started by: piyushbhashkar
6 Replies

8. UNIX for Dummies Questions & Answers

Breaking a fasta formatted file into multiple files containing each gene separately

Hey, I've been trying to break a massive fasta formatted file into files containing each gene separately. Could anyone help me? I've tried to use the following code but i've recieved errors every time: for i in *.rtf.out do awk '/^>/{f=++d".fasta"} {print > $i.out}' $i done (1 Reply)
Discussion started by: Ann Mc Cartney
1 Replies

9. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies

10. UNIX for Beginners Questions & Answers

Split large file into 24 small files on one hour basis

I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55 can any one help me on this.! ... (20 Replies)
Discussion started by: Raghuram717
20 Replies
GET_ROBUST_LIST(2)						Linux System Calls						GET_ROBUST_LIST(2)

NAME
get_robust_list, set_robust_list - get/set list of robust futexes SYNOPSIS
#include <linux/futex.h> #include <sys/types.h> #include <syscall.h> long get_robust_list(int pid, struct robust_list_head **head_ptr, size_t *len_ptr); long set_robust_list(struct robust_list_head *head, size_t len); Note: There are no glibc wrappers for these system calls; see NOTES. DESCRIPTION
The robust futex implementation needs to maintain per-thread lists of robust futexes which are unlocked when the thread exits. These lists are managed in user space; the kernel is notified about only the location of the head of the list. The get_robust_list() system call returns the head of the robust futex list of the thread whose thread ID is specified in pid. If pid is 0, the head of the list for the calling thread is returned. The list head is stored in the location pointed to by head_ptr. The size of the object pointed to by **head_ptr is stored in len_ptr. The set_robust_list() system call requests the kernel to record the head of the list of robust futexes owned by the calling thread. The head argument is the list head to record. The len argument should be sizeof(*head). RETURN VALUE
The set_robust_list() and get_robust_list() system calls return zero when the operation is successful, an error code otherwise. ERRORS
The set_robust_list() system call can fail with the following error: EINVAL len does not match the size of structure struct robust_list_head expected by kernel. The get_robust_list() system call can fail with the following errors: EPERM The calling process does not have permission to see the robust futex list of the thread with the thread ID pid, and does not have the CAP_SYS_PTRACE capability. ESRCH No thread with the thread ID pid could be found. EFAULT The head of the robust futex list can't be stored at the location head. VERSIONS
These system calls were added in Linux 2.6.17. NOTES
These system calls are not needed by normal applications. No support for them is provided in glibc. In the unlikely event that you want to call them directly, use syscall(2). A thread can have only one robust futex list; therefore applications that wish to use this functionality should use the robust mutexes pro- vided by glibc. SEE ALSO
futex(2) Documentation/robust-futexes.txt and Documentation/robust-futex-ABI.txt in the Linux kernel source tree Linux 2015-01-22 GET_ROBUST_LIST(2)
All times are GMT -4. The time now is 04:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy