KSH: Split String into smaller substrings based on count


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting KSH: Split String into smaller substrings based on count
# 1  
Old 09-19-2012
KSH: Split String into smaller substrings based on count

KSH
HP-SOL-Lin
Cannot use xAWK

I have several strings that are quite long and i want to break them down into smaller substrings.

What I have
Code:
String = "word1 word2 word3 word4 .....wordx"

What I want
Code:
String1="word1 word2"
String2="word 3 word4"
String3="word4 word5"
Stringx="wordx wordx+1"
etc.....

How Can i break this up to where if my string is longer than x words, break into smaller strings no longer than x?

Some strings are longer than 2000 words, so i cannot use a shell array as there is a max of 1024 fields.

ideas?
# 2  
Old 09-19-2012
If it's too large for a shell variable, too large for an array, to the point that you're trying to split it between multiple separate variables, that's a subtle hint, written in mile-high flashing neon letters, that it's a bad idea to try and load enormous amounts of text into a shell variable. You should be trying to make a loop of some sort, handle it piecemeal, instead of loading it whole.

Where did all that text come from? Are you loading it from file, or could you load it from file? What does or would the file look like?
# 3  
Old 09-19-2012
Its not too big for a shell variable....Its in a shell variable.

I just want it smaller so i can put it into an array.
# 4  
Old 09-19-2012
I can't give you code that I know for a fact will work given the text is so huge that it's too many elements for an array.

If it's large to the point that you're trying to cram things in two-by-two into multiple shell variables to make it fit, it's a subtle hint, written in mile-high flashing neon letters, that it's a bad idea to try and load enormous amounts of text into a shell variable. You should approach the problem from another direction instead of trying to load it whole. That's how shell code usually works. It also makes for code that works in other shells, not just yours. Someone who wrote this for BASH would not have your problem and might not imagine it could ever cause you trouble...

So again, why do you want it in an array? What are you actually trying to do? Not the way you're hellbent on doing so -- the actual goal of having the data in an array.
# 5  
Old 09-19-2012
does it matter why I want to put 2000 words into array? NO
The fact is I have a string that i need to put into an array and its too big, so I want to split it up. PERIOD.

If you dont want to help, then dont reply.
# 6  
Old 09-19-2012
Quote:
If you dont want to help, then dont reply.
Your programming habits are going to paint you into a corner someday, but I'll try and give you what you want.

It won't be elegant because it can't be. I still think you're going about this the wrong way.

Code:
N=0

set --

echo "$ENORMOUSSTRING" | tr -s ' ' '\n' | while read STRING
do
        set -- $1 "$@"

        if [ "$#" -gt 1 ]
        then
                read `print "VARNAME%04d" $N` <<EOF
$1 $2
EOF
                shift 2
                N=$((N+1))
        fi
done

if [ "$#" -gt 1 ]
then
        read `print "VARNAME%04d" $N` <<EOF
$1
EOF
        shift
fi

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split files based on row delimiter count

I have a huge file (around 4-5 GB containing 20 million rows) which has text like: <EOFD>11<EOFD>22<EORD>2<EOFD>2222<EOFD>3333<EORD>3<EOFD>44<EOFD>55<EORD>66<EOFD>888<EOFD>9999<EORD> Actually above is an extracted file from a Sql Server with each field delimited by <EOFD> and each row ends... (8 Replies)
Discussion started by: amvip
8 Replies

2. Shell Programming and Scripting

awk - split data based on the count

Greetings Experts, I am generating a validation query through awk and facing an issue, which I need to overcome by splitting the data based on the pattern matching count in the value of an array. File1 -- Table11@column1@date@Table21@column1@varchar(10)@d;... (4 Replies)
Discussion started by: chill3chee
4 Replies

3. Shell Programming and Scripting

How to Split File based on String?

hi , The scenario is like this, i have a large text files (max 5MB , about 5000 file per day ), Inside almost each line of this file there is a tag 3100.2.22.1 (represent Call_Type) , i need to generate many filess , each one with distinct (3100.2.22.1 Call_Type ) , and one more file to... (3 Replies)
Discussion started by: OTNA
3 Replies

4. Shell Programming and Scripting

A command to split a file into two based on a string

Hello What command can i use to split a tab delimited txt file into two files base on the occurrence of a string my file name is EDIT.txt The content of file is below XX 1234 PROCEDURES XY 1634 PROCEDURES XM 1245 CODES XZ 1256 CODES It has more than a million record If there is... (16 Replies)
Discussion started by: madrazzii
16 Replies

5. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

6. Shell Programming and Scripting

split file based on group count

Hi, can some one please help me to split the file based on groups. like in the below scenario x indicates the begining of the group and the file should be split each with 2 groups below there are 10 groups it should create 5 files. could you please help? (4 Replies)
Discussion started by: hitmansilentass
4 Replies

7. Shell Programming and Scripting

filtering out duplicate substrings, regex string from a string

My input contains a single word lines. From each line data.txt prjtestBlaBlatestBlaBla prjthisBlaBlathisBlaBla prjthatBlaBladpthatBlaBla prjgoodBlaBladpgoodBlaBla prjgood1BlaBla123dpgood1BlaBla123 Desired output --> data_out.txt prjtestBlaBla prjthisBlaBla... (8 Replies)
Discussion started by: kchinnam
8 Replies

8. Shell Programming and Scripting

How to split the String based on condition?

hi , I have a String str="/opt/ibm/lotus/ibw/latest" or ="/opt/lotus/ibw/latest" this value is dynamic..I want to split this string into 2 strings 1. /opt/ibm/lotus(/opt/lotus) this string must ends with "lotus" 2./ibw/latest can any body help me on this? Regards, sankar (2 Replies)
Discussion started by: sankar reddy
2 Replies

9. Shell Programming and Scripting

[KSH] Split string into array

Hi, Is there any way to convert a string into an array in KSH? In other words I want to split the string like this: STRING="one two three four" into an array of 4 values splitting on white space. The array should be similar to the one that would be created with the following command: ... (3 Replies)
Discussion started by: piooooter
3 Replies

10. Shell Programming and Scripting

KSH split string into variables

Hello, I am an intermediate scripter. I can usually find and adapt what I need by searching through previous postings, but I'm stumped. I have a string with the format "{Name1 Release1 Type1 Parent1} {Name2 Release2 Type2 Parent2}". It is being passed as an argument into a ksh script. I need to... (5 Replies)
Discussion started by: drd_2b
5 Replies
Login or Register to Ask a Question