Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

split based on the number of characters

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting split based on the number of characters
# 1  
Old 06-12-2008
split based on the number of characters
Hello,

if i have file like this:
010000890306932455804 05306977653873 0520080417010520ISMS SMT ZZZZZZZZZZZZZOC30693599000 30971360000 ZZZZZZZZZZZZZZZZZZZZ202011302942311 010000890306946317387 05306977313623 0520080417010520ISMS SMT ZZZZZZZZZZZZZOC306942190000 30971360000 ZZZZZZZZZZZZZZZZZZZZ202010300391748 010000890306945153336 05306977918990 0520080417010521ISMS SMT ZZZZZZZZZZZZZOC306942190000 30971360000 ZZZZZZZZZZZZZZZZZZZZ202011304607230 010000890306948068406 05306977404213 0520080417010523ISMS SMT ZZZZZZZZZZZZZOC306942190000 30971360000 ZZZZZZZZZZZZZZZZZZZZ202010000717971 010000890306998573372

How can i perform a split based on the number of characters?
Foa example i want in array[0] to be stored the 70 first characters of the file and in array[1] the next 70 charactets etc...

How can i do this?
# 2  
Old 06-12-2008
A 70 character split can be done as:

Code:
sed -e 's/.\{69\}/&\n/g' <file>

//Jadu
Last edited by jaduks; 06-12-2008 at 12:11 PM.. Reason: Remove " with '
# 3  
Old 06-12-2008
Thank you...

What about if i want to perform the split in perl always using 'size' as a limit
# 4  
Old 07-05-2008
I know this isn't exactly what you wanted, but this might come in handy -

Code:
split -b 60 filename.txt

Would split a file into multiple 60 byte (character) text files.

(It returns files in the format of xaa, xab, xac, xad, etc, each file having the specified number of bytes)
Last edited by AndrewTheArt; 07-05-2008 at 10:26 PM..
# 5  
Old 07-05-2008
Quote:
Originally Posted by chriss_58
Thank you...

What about if i want to perform the split in perl always using 'size' as a limit
Here is a small sample to give you an idea.

Code:
 
#!/usr/bin/perl
$teststring = "1234567890abcdefghij0987654321ABCDEFGHIJlmnop";
@chunks = split /(.{10})/, $teststring;
foreach (@chunks) {
  printf "%s\n", $_;
}

In this case, I'm using 10 characters as the size of the pieces to extract. The pattern used with split is for the delimiter/separator. Here, we say match any 10 characters as the separator. If it matches every 10 characters as a separator, then it is returning null strings for the split fields. Normally, the separator is not returned but we want the separator because these will be the actual values of interest. The parentheses that are included in the pattern tell perl to also return the separators.

If you execute this, you get the following:
Hostname:> testscript3.sh

1234567890

abcdefghij

0987654321

ABCDEFGHIJ
lmnop
Hostname:>

This is because you have null strings interspersed with the separators. There is no null string before the last 5-character substring because we did not have a full 10 characters to match.

I'll leave it for you as an exercise to remove the null strings or otherwise decide how you will skip/ignore them. How exactly you end up incorporating this into your code will also be dependent on your data file. From your description, I could not tell if records spanned lines or not.
# 6  
Old 07-06-2008
Hi.

Here is another method:
Code:
#!/usr/bin/perl

# @(#) p2       Demonstrate perl unpack to break apart a long line.

use warnings;
use strict;

my ($debug);
$debug = 0;
$debug = 1;

my ( @a, $i, $nc, $nv );
my ($lines) = 0;

while (<>) {
  $lines++;
  chomp;
  @a  = unpack( "(a70)*", $_ );
  $nc = length($_);
  $nv = scalar(@a);
  print " Unpacked $nv strings from line $lines (length $nc characters)\n";
  for ( $i = 0; $i < $nv; $i++ ) {
    print "$i: $a[$i]\n";
  }
}

print STDERR " ( Lines read: $lines )\n";

exit(0);

Producing:
Code:
% ./p2 data1
 Unpacked 9 strings from line 1 (length 572 characters)
0: 010000890306932455804 05306977653873 0520080417010520ISMS SMT ZZZZZZZZ
1: ZZZZZOC30693599000 30971360000 ZZZZZZZZZZZZZZZZZZZZ202011302942311 010
2: 000890306946317387 05306977313623 0520080417010520ISMS SMT ZZZZZZZZZZZ
3: ZZOC306942190000 30971360000 ZZZZZZZZZZZZZZZZZZZZ202010300391748 01000
4: 0890306945153336 05306977918990 0520080417010521ISMS SMT ZZZZZZZZZZZZZ
5: OC306942190000 30971360000 ZZZZZZZZZZZZZZZZZZZZ202011304607230 0100008
6: 90306948068406 05306977404213 0520080417010523ISMS SMT ZZZZZZZZZZZZZOC
7: 306942190000 30971360000 ZZZZZZZZZZZZZZZZZZZZ202010000717971 010000890
8: 306998573372
 ( Lines read: 1 )

For your data in file data1:
Code:
% wc data1
  1  29 573 data1

Eliminating the newline, 70 * 8 -> 560, + 12 => 572 ... cheers, drl
# 7  
Old 07-06-2008
Using fold:

Code:
fold -b70 input

This User Gave Thanks to radoulov For This Post:
drl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split File based on number of rows

Hi I have a requirement, where i will receive multiple files in a folder (say: /fol1/fol2/). There will be at least 14 to 16 files. The size of the files will different, some may be 80GB or 90GB, some may be less than 5 GB (and the size of the files are very unpredictable). But the names of the... (10 Replies)
Discussion started by: kpk_ds
10 Replies

2. UNIX for Dummies Questions & Answers

Split file based on number of blank lines

Hello All , I have a file which needs to split based on the blank lines Name ABC Address London Age 32 (4 blank new line) Name DEF Address London Age 30 (4 blank new line) Name DEF Address London (8 Replies)
Discussion started by: Pratik4891
8 Replies

3. Shell Programming and Scripting

How to split a file based on pattern line number?

Hi i have requirement like below M <form_name> sdasadasdMklkM D ...... D ..... M form_name> sdasadasdMklkM D ...... D ..... D ...... D ..... M form_name> sdasadasdMklkM D ...... M form_name> sdasadasdMklkM i want split file based on line number by finding... (10 Replies)
Discussion started by: bhaskar v
10 Replies

4. Shell Programming and Scripting

[Solved] How to separate one line to mutiple line based on certain number of characters?

hi Gurus, I need separate a file which is one huge line to multiple lines based on certain number of charactors. for example: abcdefghi high abaddffdd I want to separate the line to multiple lines for every 4 charactors. the result should be abcd efgh i hi gh a badd ffdd Thanks in... (5 Replies)
Discussion started by: ken6503
5 Replies

5. Shell Programming and Scripting

Split a big file into multiple files based on first four characters

I have a requirement to split a huge file to smaller text files based on first four characters which look like ABCD 1234 DFGH RREX : : : : : 0000 Each of these records are OF EQUAL bytes with a different internal layout based on the above first digit identifier.. Any help to start... (5 Replies)
Discussion started by: etldev
5 Replies

6. UNIX for Dummies Questions & Answers

Command to split the files based on the number of lines in it

Hello Friends, Can anyone help me for the below requirement. I am having a file called Input.txt. My requirement is first check the count that is wc -l input.txt If the result of the wc -l Input.txt is less than 10 then don't split the Input.txt file. Where as if Input.txt >= 10 the split... (12 Replies)
Discussion started by: malaya kumar
12 Replies

7. Shell Programming and Scripting

Script to split files based on number of lines

I am getting a few gzip files into a folder by doing ftp to another server. Once I get them I move them to another location .But before that I need to make sure each gzip is not more than 5000 lines and split it up . The files I get are anywhere from 500 lines to 10000 lines in them and is in gzip... (4 Replies)
Discussion started by: gubbu
4 Replies

8. Shell Programming and Scripting

Split single file into multiple files based on the number in the column

Dear All, I would like to split a file of the following format into multiple files based on the number in the 6th column (numbers 1, 2, 3...): ATOM 1 N GLY A 1 -3.198 27.537 -5.958 1.00 0.00 N ATOM 2 CA GLY A 1 -2.199 28.399 -6.617 1.00 0.00 ... (3 Replies)
Discussion started by: tomasl
3 Replies

9. Shell Programming and Scripting

Split File Based on Line Number Pattern

Hello all. Sorry, I know this question is similar to many others, but I just can seem to put together exactly what I need. My file is tab delimitted and contains approximately 1 million rows. I would like to send lines 1,4,& 7 to a file. Lines 2, 5, & 8 to a second file. Lines 3, 6, & 9 to... (11 Replies)
Discussion started by: shankster
11 Replies

10. UNIX for Advanced & Expert Users

split a string into 3 characters

Hi, I have a requirement like this. String=abcdefghijkl I need to split this string into a substrign each 3 characters and write output into a file. Output: abc def ghi jkl Can any one please help me how can i acheive this Thanks (3 Replies)
Discussion started by: ukatru
3 Replies
Login or Register to Ask a Question