Sponsored Content
Top Forums Shell Programming and Scripting Split a .csv File into Multiple Files Post 302886678 by azherkn3 on Monday 3rd of February 2014 07:17:45 AM
Old 02-03-2014
Split a .csv File into Multiple Files

Hi guys,

I have a requirement where i need to split a .csv file into multiple files.
Say for example i have data.csv file and i have splitted that into multiple files based on some conditions i.e first file should have 100, last file 50 and other files 1000 each. Am passing the values in command line arguments say ./samp.sh 100 50 1000..
I need to check the following conditions in which i got stuck

1) If the file has 1000 records in it will not split further
2) Any file greater than 1000 records will be split

how to achieve this, pls guide me through

Thanks in advance.

Moderator's Comments:
Mod Comment This is your third infraction for not thinking about your subject text. Please follow forum rules and use a well thought out, descriptive subject test. Also, do not cut-and-paste from other forums and sites without removing the formatting tags from those sites. Thank you.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split a file into multiple files

I have a file ehich has multiple create statements as create abc 123 one two create xyz 456 four five create nnn 666 six four I want to separte each create statement in seperate files (3 Replies)
Discussion started by: glamo_2312
3 Replies

2. UNIX for Dummies Questions & Answers

split a file into multiple files

Hi All, I have a file ABC.txt and I need to split this file on every 250 rows. And the file name should be ABC1.txt , ABC2.txt and so on. I tried with split command split -l 250 <filename> '<filename>' but the file name returned was ABC.txtaa ABC.txtab. Please... (8 Replies)
Discussion started by: kumar66
8 Replies

3. Shell Programming and Scripting

Split a file into multiple files

Hi, i have a file like this: 1|2|3|4|5| 1|2|8|4|6| Trailer1||||| 1|2|3| Trailer2||| 3|4|5|6| 3|4|5|7| 3|4|5|8| Trailer2||| I want to generate 3 files out of this based on the trailer record. Trailer record string can be different for each file or it may be same for one or two. No... (24 Replies)
Discussion started by: pparthji
24 Replies

4. Shell Programming and Scripting

Split file into multiple files

Hi I have a file that has multiple sequences; the sequence name is the line starting with '>'. It looks like below: infile.txt: >HE_ER tttggtgccttgactcggattgggggacctcccttgggagatcaatcccctgtcctcctgctctttgctc cgtgaaaaggatccacctatgacctctagtcctcagacccaccagcccaaggaacatctcaccaatttca >M7B_Ho_sap... (2 Replies)
Discussion started by: jdhahbi
2 Replies

5. Shell Programming and Scripting

split a csv file into specified number of files (not lines)

hi, i really need it ...it's not simple to explain but as it's part of a crontab i can't split the file manually...and the file can change every day so the lines are not a good base. example: how to split 1 csv file in 15 files? thank you very much regards :b: (4 Replies)
Discussion started by: 7stars
4 Replies

6. Shell Programming and Scripting

split file into multiple files

Hi, I have a file of the following syntax that has around 120K records that are tab separated. input.txt abc def klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango unix_0000103; abc def klm 415 439 . + . klm_mango unix_00001043; I am looking for an awk oneliner... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

7. Shell Programming and Scripting

Split file in unix into multiple files

Hi Gurus I have to split the incoming source file into multiple file. File contains some unwanted XML tags also . Files looks like some XML tags FILEHEADERABC 12 -- --- ---- EOF some xml tags xxxFILEHEADERABC 13 -- --- ---- EOF I have to ignore XML tags and only split file... (6 Replies)
Discussion started by: manish2608
6 Replies

8. Shell Programming and Scripting

Split a file into multiple files with an extension

Hi I have a file with 100 million rows. I want to split them into 1000 subfiles and name them from 1.xls to 1000.xls.. Can I do it in awk? Thanks, (8 Replies)
Discussion started by: Diya123
8 Replies

9. Shell Programming and Scripting

Split file into multiple files using delimiter

Hi, I have a file which has many URLs delimited by space. Now i want them to move to separate files each one holding 10 URLs per file. http://3276.e-printphoto.co.uk/guardian http://abdera.apache.org/ http://abdera.apache.org/docs/api/index.html I have used the below code to arrange... (6 Replies)
Discussion started by: vel4ever
6 Replies

10. Shell Programming and Scripting

Split file into multiple files using awk

I have following file: FHEAD0000000001RTLG20161205110959201612055019 THEAD...... TCUST..... TITEM.... TTEND... TTAIL... THEAD...... TCUST..... TITEM.... TITEM..... TTEND... TTAIL... FTAIL<number of lines in file- 10 digits;prefix 0><number of lines in file-2 - 10 digits- perfix 0>... (6 Replies)
Discussion started by: amitdaf
6 Replies
split(1)						      General Commands Manual							  split(1)

NAME
split - Splits a file into pieces SYNOPSIS
Current syntax split [-l line_count] [-a suffix_length] [file | -] [prefix] split -b n [k|m] [-a suffix_length] [file | -] [prefix] Obsolescent syntax split [-number] [-a suffix_length] [file | -] [prefix] STANDARDS
Interfaces documented on this reference page conform to industry standards as follows: split: XCU5.0 Refer to the standards(5) reference page for more information about industry standards and associated tags. OPTIONS
Uses suffix_length letters to form the suffix portion of the file names of the split file. If -a is not specified, the default suffix length is two letters. If the sum of the prefix and the suffix arguments would create a file name exceeding NAME_MAX bytes, an error occurs. In this case, split exits with a diagnostic message and no files are created. Split a file into pieces n bytes in size. Split a file into pieces n kilobytes (1024 bytes) in size. Split a file into pieces n megabytes (1048576 bytes) in size. Specifies the number of lines in each output file. The line_count argument is an unsigned decimal integer. The default value is 1000. If the input does not end with a newline character, the partial line is included in the last output file. Specifies the number of lines in each output file. The default is 1000 lines per output file. If the input does not end with a newline character, the partial line is included in the last output file. (Obsolescent) OPERANDS
The pathname of the file to be split. If you do not specify an input file, or if you specify -, the standard input is used. DESCRIPTION
The split command reads file and writes it in number-line pieces (default 1000 lines) to a set of output files. The size of the output files can be modified by using the -b or -l options. Each output file is created with a unique suffix consisting of exactly suffix lowercase letters from the POSIX locale. The letters of the suffix are used as if they were a base-26 digit system, with the first suffix to be created consisting of all a characters, the second with b replacing the last a etc., until a name of all zs is cre- ated. By default, the names of the output files are x, followed by a two-character suffix from the character set as described above, starting with aa, ab, ac, etc., and continuing until the suffix zz, for a maximum of 676 files. The value of prefix cannot be longer than the value of NAME_MAX from <limits.h> minus two. If the number of files required is greater than the maximum allowed by the effective suffix length (such that the last allowable file would be larger than the requested size), split fails after creating the last possible file with a valid suffix. The split command will not delete the files it created with valid suffixes. If the file limit is not exceeded, the last file created contains the remainder of the input file and thus might be smaller than the requested size. EXIT STATUS
The following exit values are returned: Successful completion. An error occurred. EXAMPLES
To split a file into 1000-line segments, enter: split book This splits book into 1000-line segments named xaa, xab, xac, and so forth. To split a file into 50-line segments and specify the file name prefix, enter: split -l50 book sect This splits book into 50-line segments named sectaa, sectab, sectac, and so forth. ENVIRONMENT VARIABLES
The following environment variables affect the execution of split: Provides a default value for the internationalization variables that are unset or null. If LANG is unset or null, the corresponding value from the default locale is used. If any of the internationalization vari- ables contain an invalid setting, the utility behaves as if none of the variables had been defined. If set to a non-empty string value, overrides the values of all the other internationalization variables. Determines the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multibyte characters in arguments and input files). Determines the locale for the format and contents of diagnostic messages written to standard error. Determines the location of message catalogues for the processing of LC_MESSAGES. SEE ALSO
Commands: bfs(1), csplit(1) Standards: standards(5) split(1)
All times are GMT -4. The time now is 04:43 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy