Sponsored Content
Full Discussion: Splitting data file
Top Forums Shell Programming and Scripting Splitting data file Post 59985 by zazzybob on Thursday 6th of January 2005 10:51:38 AM
Old 01-06-2005
There's probably an easier way to do it, but I'd do something like this with Perl...
Code:
#!/usr/bin/perl

# Just modify the following two variables
$NUM_LINES=10;
$HEADER_STRING="This is my header";

$count=0;
$file_num=0;

while (<>) {
  if ( $count == 0 ) {
     $filename = join("","file",$file_num);
     open( FILE, ">> $filename" );
     print( FILE "$HEADER_STRING\n" );
     print( FILE "$_" );
     $count++;
  } elsif ( $count == $NUM_LINES ) {
     close( FILE );
     $count = 0;
     $file_num++;
  } else {
     # just write the line!
     print( FILE "$_" );
     $count++;
  }
}

Then make it executable and call with
$ my_perl_script.pl my_large_file

This will split your large file into as many 10 data line files (i.e. 11 lines including the header) as required.

Just modify the $NUM_LINES and $HEADER_STRING variables to suit your needs.

Cheers
ZB
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting the data and storing it into 2 variables

Hi, I have a file in the given format: |aaa |hhh |fat hat chat |make sure I need to get store in a variable,say 'error' the value of the string 'fat hat chat' In short, echo $error should give the result fat hat chat Is this possible using awk? Also, can this be split.For... (10 Replies)
Discussion started by: jisha
10 Replies

2. Shell Programming and Scripting

Data Splitting into two files from one file

I have a file as: I/P File: Ground Car 2009 Lib 2008 Lib 2003 Ground Car 2009 Ground Car 2003 Car 2005 Car 2003 Car 2005 Sita 2900 2006 Car 2007 I have to split the file into two: - one for names and second for years. O/p1 (Names): Ground Car (3 Replies)
Discussion started by: karumudi7
3 Replies

3. UNIX for Dummies Questions & Answers

Splitting Data in File

I have a file with the below Data 1,nj@ny@pa@caa 2,ct 3,ca@vaa@txI want the output to be 1,nj 1,ny 1,pa 1,caa 2,ct 3,ca 3,vaa 3,tx I need to split the second column based on @ as delimiter The number of delimiters is unknown (4 Replies)
Discussion started by: traininfa
4 Replies

4. Shell Programming and Scripting

Splitting the data in a column into several columns

Hi, I have the following input file 32895901-d17f-414c-ac93-3e7e0f5ec240 AND @GDF_INPUT 73b129e1-1fa9-4c0d-b95b-4682e5389612 AUS @GDF_INPUT 40f82e88-d1ff-4ce2-9b8e-d827ddb39447 BEL @GDF_INPUT 36e9c3f1-042a-43a4-a80e-4a3bc2513d01 BGR @GDF_INPUT I want to split column 3 into two columns:... (1 Reply)
Discussion started by: ramky79
1 Replies

5. Shell Programming and Scripting

perl : splitting the data into 2 different variables

I have a perl variable which contains the below value. $var1 = "2% / 51%" Now I would like to split the data into 2 different variables. For example $part1 = 2 $part2 = 51 Could anyone please help me in this regard ? Regards, GS (4 Replies)
Discussion started by: giridhar276
4 Replies

6. Shell Programming and Scripting

Help me pls : splitting single file in unix into different files based on data

I have a file in unix with sample data as follows : -------------------------------------------------------------- -------------------------------------------------------------- {30001002|XXparameter|Layout|$ I want this file to be splitted into different files and corresponding to the sample... (54 Replies)
Discussion started by: Ravindra Swan
54 Replies

7. UNIX for Dummies Questions & Answers

Splitting data into new records

Hi, My file is seperated with ";" delimiter, after 13 delimiter i want to put the data in new line... eg: My current file:- a;b;c;d;e;f;g;h;e;f;h;s;t;a;i;o;q;t;q;r;yu;f;sz;f;t;r........... i want o/p as:- a;b;c;d;e;f;g;h;e;f;h;s;t a;i;o;q;t;q;r;yu;f;sz;f;t;r How to achieve ths,... (2 Replies)
Discussion started by: gnnsprapa
2 Replies

8. UNIX for Dummies Questions & Answers

Extracting data from one file, based on another file (splitting)

Dear All, I have two files but want to extract data from one based on another... can you please help me file 1 David Tom Ellen and file 2 David|0010|testnamez|resultsz David|0004|testnamex|resultsx Tom|0010|testnamez|resultsz Tom|0004|testnamex|resultsx Ellen|0010|testnamez|resultsz... (12 Replies)
Discussion started by: A-V
12 Replies

9. Shell Programming and Scripting

Splitting the Data using awk

Hello All, I have a comma delimiter file with 10 columns. I took the desired data but from $4 I need to split into two columns as 3+7 bytes. awk -F"," -v OFS=',' '{print $2,$3,$4}' foo.txt 42366,11/10/2014,5012418769 42366,11/10/2014,2046955672 42366,11/10/2014,2076802951 ... (3 Replies)
Discussion started by: karumudi7
3 Replies

10. Shell Programming and Scripting

awk issue splitting a fixed-width file containing line feed in data

Hi Forum. I have the following script that splits a large fixed-width file into smaller multiple fixed-width files based on input segment type. The main command in the script is: awk -v search_col_pos=$search_col_pos -v search_str_len=$search_str_len -v segment_type="$segment_type"... (8 Replies)
Discussion started by: pchang
8 Replies
FLOCKFILE(3)						   BSD Library Functions Manual 					      FLOCKFILE(3)

NAME
flockfile, ftrylockfile, funlockfile -- stdio locking functions LIBRARY
Standard C Library (libc, -lc) SYNOPSIS
#include <stdio.h> void flockfile(FILE *file); int ftrylockfile(FILE *file); void funlockfile(FILE *file); DESCRIPTION
These functions provide explicit application-level locking of stdio FILE objects. They can be used to avoid output from multiple threads being interspersed, input being dispersed among multiple readers, and to avoid the overhead of locking the object for each operation. The flockfile() function acquires an exclusive lock on the specified object. If another thread has already locked the object, flockfile() will block until the lock is released. The ftrylockfile() function is a non-blocking version of flockfile(); if the lock cannot be acquired immediately, ftrylockfile() returns non- zero instead of blocking. The funlockfile() function releases the lock on an object acquired by an earlier call to flockfile() or ftrylockfile(). These functions behave as if there is a lock count associated with each object. Each time flockfile() is called on the object, the count is incremented, and each time funlockfile() is called on the object, the count is decremented. The lock is only actually released when the count reaches zero. RETURN VALUES
The flockfile() and funlockfile() functions return no value. The ftrylockfile() function returns zero if the object was successfully locked, non-zero otherwise. SEE ALSO
getc_unlocked(3), putc_unlocked(3) STANDARDS
The flockfile(), ftrylockfile(), and funlockfile() functions conform to IEEE Std 1003.1-2001 (``POSIX.1''). BSD
January 10, 2003 BSD
All times are GMT -4. The time now is 03:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy