Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Splitting a file into chunks of 1TB
Top Forums Shell Programming and Scripting Splitting a file into chunks of 1TB Post 302842607 by bombcan on Friday 9th of August 2013 02:28:53 PM
Old 08-09-2013
Splitting a file into chunks of 1TB
Hi


I have a file with different filesystems with there sizes. I need to split them in chucks of 1TB.

The file looks like

Code:
vf_MTLHQNASF07_Wkgp2 187428400  10601AW1
vf_MTLHQNASF07_Wkgp2 479504596  10604AW1
vf_MTLHQNASF07_Wkgp2 19940      10605AID
vf_MTLHQNASF07_Wkgp2 1242622044 10605AW1
vf_MTLHQNASF07_Wkgp2 412696     10605AWP
vf_MTLHQNASF07_Wkgp2 87813204   10607AW1
vf_MTLHQNASF07_Wkgp2 31712      10607AW2
vf_MTLHQNASF07_Wkgp2 305420     ADMIN
vf_MTLHQNASF07_Wkgp2 8396       ANNUAL
vf_MTLHQNASF07_Wkgp2 94668      BO_Reports
vf_MTLHQNASF07_Wkgp2 4896484    Board Material
vf_MTLHQNASF07_Wkgp2 4992       CMONGEAU
vf_MTLHQNASF07_Wkgp2 64944      CORR
vf_MTLHQNASF07_Wkgp2 185218932  CS4AW1
vf_MTLHQNASF07_Wkgp2 15423520   CS4AW2
vf_MTLHQNASF07_Wkgp2 63368560   CS4AW3
vf_MTLHQNASF07_Wkgp2 2735968    CS4AW4

What I have is

Code:
cat /tmp/file |awk '{print; sum+=$2;if (sum>=1024000000){t=int(sum/1024000000)*1024000000; printf RS}}'

However it does not split them correctly

Code:
vf_MTLHQNASF07_Wkgp2 187428400  10601AW1
vf_MTLHQNASF07_Wkgp2 479504596  10604AW1
vf_MTLHQNASF07_Wkgp2 19940      10605AID
vf_MTLHQNASF07_Wkgp2 1242622044 10605AW1

vf_MTLHQNASF07_Wkgp2 412696     10605AWP
vf_MTLHQNASF07_Wkgp2 87813204   10607AW1
vf_MTLHQNASF07_Wkgp2 31712      10607AW2
vf_MTLHQNASF07_Wkgp2 305420     ADMIN
vf_MTLHQNASF07_Wkgp2 8396       ANNUAL
vf_MTLHQNASF07_Wkgp2 94668      BO_Reports
vf_MTLHQNASF07_Wkgp2 4896484    Board Material
vf_MTLHQNASF07_Wkgp2 4992       CMONGEAU
vf_MTLHQNASF07_Wkgp2 64944      CORR
vf_MTLHQNASF07_Wkgp2 185218932  CS4AW1

vf_MTLHQNASF07_Wkgp2 15423520   CS4AW2
vf_MTLHQNASF07_Wkgp2 63368560   CS4AW3
vf_MTLHQNASF07_Wkgp2 2735968    CS4AW4

Any help please
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove chunks of text from file

All, So, I have an ldif file that contains about 6500 users worth of data. Some users have a block of text I'd like to remove, while some don't. Example (block of text in question is the block starting with "authAuthority: ;Kerberosv5"): User with text block: # username, users,... (7 Replies)
Discussion started by: staze
7 Replies

2. Shell Programming and Scripting

Split file into chunks of low & high byte

Hi guys, i have a question about spliting a binary file into 2 chunks. First chunk with all high bytes and the second one with all low bytes. What unix tools can i use? And how can this be performed? I looked in manpages of split and dd but this does not help. Thanks (2 Replies)
Discussion started by: basta
2 Replies

3. Solaris

Multiple Backups to USB 1TB Drives using dd

First of all, great web site! I have been using it for a while but just registered today. It's been a great resource for me. Now, on to my issue.;) I'm geographically separated from six (Sun v245s) development servers that I have been asked to backup and restore as development is done and... (11 Replies)
Discussion started by: ShawnD41
11 Replies

4. AIX

Can't backup more than ~1TB.

Hello all, I create a backup of a file system with hostA # find . | backup -iqvf /backup/hostA.fsA.backup After a while I get the following error message: backup medium write error: File too large Check backup media and rerun the backup /backup is a NFS mount and is the backup device... (5 Replies)
Discussion started by: petervg
5 Replies

5. UNIX for Dummies Questions & Answers

Awk: Print out overlapping chunks of file - rows 0-20,10-30,20-40 etc.

First time poster, but the forum has saved my bacon more times than... Lots. Anyway, I have a text file, and wanted to use Awk (or any other sensible program) to print out overlapping sections, or arbitrary length. To describe by example, for file 1 2 3 4 5 etc... I want the out put... (3 Replies)
Discussion started by: matfald
3 Replies

6. Shell Programming and Scripting

awk for splitting file in constant chunks

Hi gurus, I wanted to split main file in 20 files with 2500 lines in each file. My main file conatins total 2500*20 lines. Following awk I made, but it is breaking with error. awk '{ for (i = 1; i <= 20; i++) { starts=2500*$i-1; ends=2500*$i; NR>=starts && NR<=ends {f=My$i".txt"; print >> f;... (10 Replies)
Discussion started by: mukesh.lalwani
10 Replies

7. Shell Programming and Scripting

Reverse sort on delimited chunks within a file

Hello, I have a large file in which data of names is sorted according to their homographs. The database has the following structure:Each set of homographs with their corresponding equivalents in Devanagari is separated out from the next set by a hard return. An example will make this... (12 Replies)
Discussion started by: gimley
12 Replies

8. Shell Programming and Scripting

Deleting duplicated chunks in a file using awk/sed

Hi all, I'd always appreciate all helps from this site. I would like to delete duplicated chunks of strings on the same row(?). One chunk is comprised of four lines such as: path name starting point ending point voltage number I would like to delete duplicated chunks on the same... (5 Replies)
Discussion started by: jypark22
5 Replies

9. Shell Programming and Scripting

Modification of perl script to split a large file into chunks of 5000 chracters

I have a perl script which splits a large file into chunks.The script is given below use strict; use warnings; open (FH, "<monolingual.txt") or die "Could not open source file. $!"; my $i = 0; while (1) { my $chunk; print "process part $i\n"; open(OUT, ">part$i.log") or die "Could... (4 Replies)
Discussion started by: gimley
4 Replies

10. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies

LEARN ABOUT CENTOS

locale::codes::langfam

Locale::Codes::LangFam(3)				User Contributed Perl Documentation				 Locale::Codes::LangFam(3)

NAME

       Locale::Codes::LangFam - standard codes for language extension identification

SYNOPSIS

	  use Locale::Codes::LangFam;

	  $lext = code2langfam('apa');		       # $lext gets 'Apache languages'
	  $code = langfam2code('Apache languages');    # $code gets 'apa'

	  @codes   = all_langfam_codes();
	  @names   = all_langfam_names();

DESCRIPTION

       The "Locale::Codes::LangFam" module provides access to standard codes used for identifying language families, such as those as defined in
       ISO 639-5.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 639-5
       language family codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying language families. A code set may be specified using either a name, or a
       constant that is automatically exported by this module.

       For example, the two are equivalent:

	  $lext = code2langfam('apa','alpha');
	  $lext = code2langfam('apa',LOCALE_LANGFAM_ALPHA);

       The codesets currently supported are:

       alpha
	   This is the set of three-letter (lowercase) codes from ISO 639-5 such as 'apa' for Apache languages.

	   This is the default code set.

ROUTINES

       code2langfam ( CODE [,CODESET] )
       langfam2code ( NAME [,CODESET] )
       langfam_code2code ( CODE ,CODESET ,CODESET2 )
       all_langfam_codes ( [CODESET] )
       all_langfam_names ( [CODESET] )
       Locale::Codes::LangFam::rename_langfam  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Codes::LangFam::add_langfam  ( CODE ,NAME [,CODESET] )
       Locale::Codes::LangFam::delete_langfam  ( CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_alias  ( NAME ,NEW_NAME )
       Locale::Codes::LangFam::delete_langfam_alias  ( NAME )
       Locale::Codes::LangFam::rename_langfam_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::delete_langfam_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.loc.gov/standards/iso639-5/id.php
	   ISO 639-5 .

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 2011-2013 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.16.3							    2013-02-27						 Locale::Codes::LangFam(3)
All times are GMT -4. The time now is 07:36 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy