Sponsored Content
Top Forums Shell Programming and Scripting Splitting the Huge file into several files... Post 302404372 by jim mcnamara on Tuesday 16th of March 2010 10:46:53 AM
Old 03-16-2010
you can use awk:
Code:
awk -F'|' { file=sprintf("%s_%s", FILENAME, substr($5,5,2)); print $0 >> file}' inputfile

will produce files names inputfile_01 inputfile_02 ... inputfile_12 if there is data for all months in inputfile.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting huge XML Files into fixsized wellformed parts

Hi, I need to split xml-files with sizes greater than 2 gb into smaler chunks. As I dont want to end up with billions of files, I want those splitted files to have configurable sizes like 250 MB. Each file should be well formed having an exact copy of the header (and footer as the closing of the... (0 Replies)
Discussion started by: Malapha
0 Replies

2. Shell Programming and Scripting

splitting huge xml into multiple files

hi all i have a some huge html files (500MB to 1GB). Each file has multiple <html></html> tags <html> ................. .................... .................... </html> <html> ................. .................... .................... </html> <html> .................... (5 Replies)
Discussion started by: uttamhoode
5 Replies

3. Shell Programming and Scripting

Help on splitting this huge file

Hi , i have files coming in my system which are very huge in MB and GBs, all these files are in a single line, there is no newline character. I need to get only last 700 bytes of these files, of this i am splitting the files by "split -b 700 filename" but this gives all the splitted... (2 Replies)
Discussion started by: Prateek007
2 Replies

4. Shell Programming and Scripting

Splitting files from one file

Hi, I have an input file like: 111 abcdefgh asdfghjk dfghjkl 222 aaaaaaa bbbbbb 333 djfhfgjktitjhgfkg 444 djdhfjkhfjkghjkfg hsbfjksdbhjkgherjklg fjkhfjklsahjgh fkrjkgnj I want to read this input file and make separate output files with the header as numric value like "111"... (9 Replies)
Discussion started by: saltysumi
9 Replies

5. Shell Programming and Scripting

Splitting file into 2 files ?

Hi extending to one of my previous posted query .... I am using nawk -v invar1="$aa" '{print > ("ABS\_"((/\|/)?"A\_":"B\_")invar1"\_NETWORKID.txt")}' spfile.txt to get 2 different files based on split condition i.e. "|" Similar to invar1 variable in nawk I also need one more variable... (18 Replies)
Discussion started by: shekharjchandra
18 Replies

6. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

7. Shell Programming and Scripting

splitting a huge line of file into multiple lines with fixed number of columns

Hi, I have a huge file with a single line. But I want to break that line into lines of with each line having five columns. My file is like this: code: "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you." I want it like this: code:... (1 Reply)
Discussion started by: rajsharma
1 Replies

8. Shell Programming and Scripting

Need help splitting huge single record file

I was given a data file that I need to split into multiple lines/records based on a key word. The problem is that it is 2.5GB or bigger and everything I try in perl or sed causes a Segmentation fault. Can someone give me some other ideas. The data is of the form:... (5 Replies)
Discussion started by: leolson
5 Replies

9. UNIX for Dummies Questions & Answers

File comparison of huge files

Hi all, I hope you are well. I am very happy to see your contribution. I am eager to become part of it. I have the following question. I have two huge files to compare (almost 3GB each). The files are simulation outputs. The format of the files are as below For clear picture, please see... (9 Replies)
Discussion started by: kaaliakahn
9 Replies

10. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies
TCA_CORRECT(1)							       HUGIN							    TCA_CORRECT(1)

NAME
tca_correct - Calculate transverse chromatic aberration SYNOPSIS
tca_correct [options] inputfile <inputfile> is the base name of 4 image files: <inputfile> Colour file to compute TCA parameters red_<inputfile> Red channel of <inputfile> green_<inputfile> Green channel of <inputfile> blue_<inputfile> Blue channel of <inputfile> The channel images must be colour images with 3 identical channels. If any of -R, -G, or -B is given, this file name is used instead of the derived name. Output: commandline arguments for fulla DESCRIPTION
Generates fulla command-line parameters for correction of transverse chromatic aberration (TCA) from sample photos. Ideal sample photos would have a lot of contrast over the entire image area, little saturated colour and no blown highlights. OPTIONS
-h Display help -l input file is PTO file instead of image -m method optimization method (0 normal, 1 newfit) -o optvars string of variables to optimize ("abcvde") -r Reset values (this will zero a,b,c,d,e params and set v to 10) makes sense only with -l option -s <scale> Scale for corner detection -n <number> number of points per grid cell (default: 10) -g <number> divide image in <number>x<number> grid cells (default: 10) -t num Remove all control points with an error higher than num pixels (default: 1.5) -v Verbose -w filename write PTO file -R <r> Use this file as red channel -G <g> Use this file as green channel -B <b> Use this file as blue channel AUTHORS
Written by Pablo d'Angelo and others. "Version: 2011.4.0" 2011-12-02 TCA_CORRECT(1)
All times are GMT -4. The time now is 04:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy