Visit The New, Modern Unix Linux Community

Split the text file into two

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split the text file into two
# 1  
Split the text file into two

OS : RHEL 7.3

I have a file like below. I want to move (cut and paste) the first 7 lines of file1 to another file (file2).
How can I do this ? In my real life scenario, I will be moving first 12 millions lines of file1 to file2

$ cat file1.txt
      7369|SMITH     |CLERK    |      7902|17-DEC-80|       800|          |        20
      7499|ALLEN     |SALESMAN |      7698|20-FEB-81|      1600|       300|        30
      7521|WARD      |SALESMAN |      7698|22-FEB-81|      1250|       500|        30
      7566|JONES     |MANAGER  |      7839|02-APR-81|      2975|          |        20
      7654|MARTIN    |SALESMAN |      7698|28-SEP-81|      1250|      1400|        30
      7698|BLAKE     |MANAGER  |      7839|01-MAY-81|      2850|          |        30
      7788|SCOTT     |ANALYST  |      7566|19-APR-87|      3000|          |        20
      7839|KING      |PRESIDENT|          |17-NOV-81|      5000|          |        10
      7844|TURNER    |SALESMAN |      7698|08-SEP-81|      1500|         0|        30
      7876|ADAMS     |CLERK    |      7788|23-MAY-87|      1100|          |        20
      7900|JAMES     |CLERK    |      7698|03-DEC-81|       950|          |        30
      7902|FORD      |ANALYST  |      7566|03-DEC-81|      3000|          |        20
      8839|KATE      |ANALYST  |      7566|17-FEB-79|      6000|          |        40

# 2  
Did you consider the split command?
# 3  
Hello kraljic,

Could you please try following and let me know if this helps you.
awk -v count=1 -v line=7 'NR>line &&!a{close("file"count);count++;a++} {print > "file"count}' FiLe1.txt

You have to change variable named line's value to as per your lines which you want in file1, also this logic will only create 2 files named file1(where it will put those many lines from starting of file1.txt which you will give in line variable) and file2 will have remaining lines, kindly do give a try to it and let me know if this helps you.

R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
# 4  
If you want to deploy awk, try
awk -vLN=7 '{print > ("file" 2-(NR>LN))}' file

These 2 Users Gave Thanks to RudiC For This Post:
# 5  
Hello kraljic,

I have a few to questions pose in response first:-
  • What have you tried so far?
  • What output/errors do you get?
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)
Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.

Did split help, it's probably the right tool for this, unless you want to do other processing at the same time. With split you can break up your file based on the number of bytes, lines, etc. and define output filenames and suffix length to make it suitable. It will not easily take the literal name file1 and generate a file2, but you can force it to be something sensible and then rename the output as needed.

If you generate 5 files of output and want to be separating out just the content of the first, it is straightforward to rename that one to the desired name and either delete the remainder or cat them back together (in the right order) to overwrite the original file, thereby removing the lines you have just extracted. You would then need to tidy the temporary files up too to save space.

We're all here to learn and getting the relevant information will help us all.

Have a go with split as my learned member RudiC suggests and let us know if it helps or you get stuck. I'm sure we can get a working solution that you can support and/or reuse.

kind regards,
This User Gave Thanks to rbatte1 For This Post:
# 6  
Originally Posted by RudiC
If you want to deploy awk, try
awk -vLN=7 '{print > ("file" 2-(NR>LN))}' file

Well, that's a neater solution than the temporary split files I was thinking of. I will save that in my own hints & tips file, thank you Smilie Smilie

Many thanks,

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #751
Difficulty: Medium
The first IBM 5150 PCs had two 5.25-inch 360 KB single sided double density (SSDD) floppy disk drives
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to target certain delimiter to split text file?

Hi, all. I have an input file. I would like to generate 3 types of output files. Input: LG10_PM_map_19_LEnd_1000560 LG10_PM_map_6-1_27101856 LG10_PM_map_71_REnd_20597718 LG12_PM_map_5_chr_118419232 LG13_PM_map_121_24341052 LG14_PM_1a_456799 LG1_MM_scf_5a_opt_abc_9029993 ... (5 Replies)
Discussion started by: huiyee1
5 Replies

2. Shell Programming and Scripting

Split a text file into multiple pages based on pattern

Hi, I have a text file (attached the sample). I have also, attached the way the way the files need to be split. We get this file, that will either have 24 Jurisdictions, or will miss some and retain some. Like in the attached sample file, there are only Jurisdictions 03,11,14,15, 20 and 30.... (3 Replies)
Discussion started by: ebsus
3 Replies

3. Shell Programming and Scripting

Split a text file into multiple text files?

I have a text file with entries like 1186 5556 90844 7873 7722 12 7890.6 78.52 6679 3455 9867 1127 5642 ..N so many records like this. I want to split this file into multiple files like cluster1.txt, cluster2.txt, cluster3.txt, ..... clusterN.txt. (4 Replies)
Discussion started by: sammy777
4 Replies

4. Shell Programming and Scripting

Split column using awk in a text file

chr1 412573 . A C 2758.77 . AC=2;AF=1.00;AN=2;DP=71;Dels=0.00;FS=0.000;HaplotypeScore=2.8822;MLEAC=2;MLEAF=1.00;MQ=58.36;MQ0=0;QD=38.86;resource.EFF=INTERGENIC(MODIFIER||||||||) GT:AD:DP:GQ:PL 1/1:0,71:71:99:2787,214,0 GATKSAM chr1 602567 rs21953190 A ... (9 Replies)
Discussion started by: mehar
9 Replies

5. Shell Programming and Scripting

Split text file

Hi all, I am very new to shell scripting and some help is greatly appreciated. I have 10 column based text files, i would like to split each of them into 6 files ; the 1st one having columns 1, 2 ,3,4 , the second one having columns 1,2,8,9 etc. Is there a way I could get 60 files out my... (3 Replies)
Discussion started by: shreymuk
3 Replies

6. Shell Programming and Scripting

How to split a fixed width text file into several ones based on a column value?

Hi, I have a fixed width text file without any header row. One of the columns contains a date in YYYYMMDD format. If the original file contains 3 dates, I want my shell script to split the file into 3 small files with data for each date. I am a newbie and need help doing this. (14 Replies)
Discussion started by: bhanja_trinanja
14 Replies

7. Shell Programming and Scripting

Improper wrapping of text in split of file

Hi I am using shell script where I am calling SQLPLUS and executing one PL/SQL block. This PL/SQL block generates the spool file for example splfile.txt. After successful generation of spool file I use nawk to split this file into 2 different files. Till here no issues. nawk... (1 Reply)
Discussion started by: shekharjchandra
1 Replies

8. Shell Programming and Scripting

Perl Split for text in file

Hi all I have written Perl script to swap the strings in the second a third column from a text file. My input file format is : the|empty|the|det lake|empty|lake|conj_and was|empty|was|auxpass drained|empty|drained|conj_and birds|empty|bird|s|nn The expected output file format is... (11 Replies)
Discussion started by: my_Perl
11 Replies

9. Shell Programming and Scripting

efficiently split a 2GB text file into two

Can an expert kindly write an efficient Linux ksh script that will split a large 2 GB text file into two? Here is a couple of sample record from that text file: "field1","field2","field3",11,22,33,44 "TG","field2b","field3b",1,2,3,4 The above rows are delimited by commas. This script is to... (2 Replies)
Discussion started by: ihot
2 Replies

10. UNIX for Dummies Questions & Answers

Split text file by pages

Hello! Firts of all, I'm sorry for my English. My problem: I have text file with few Form Feed symbols (FF, ASCII code =12) inside (for example - some report, consists of some pages for printing). I want to split this text by pages - each page (until FF symbol) in single file. I... (2 Replies)
Discussion started by: ranri
2 Replies

Featured Tech Videos