Split a fixed length file bases on last occurence of string Post: 302833831

Sponsored Content

Top Forums Shell Programming and Scripting Split a fixed length file bases on last occurence of string Post 302833831 by Don Cragun on Thursday 18th of July 2013 01:59:00 AM

07-18-2013

Registered User

You didn't say what your filenames are??? You didn't say whether the patterns appear only at the start of lines or appear anywhere in lines in your input file???

Making lots of wild assumptions:

The patterns you're trying to match don't contain any characters that are "special" in a regular expression.
The patterns you're trying to match don't contain any whitespace characters.
The patterns you're trying to match don't contain any question mark characters.
The patterns you're trying to match only appear at the start of a line in your input file.
The patterns you want to match appear on the first line of a file named patterns and are separated by one or more space or tab characters.
The names of the input file and both output files will be passed to this script as operands 1, 2, and 3, respectively (and default to files named input, out1, and out2 if operands are not given to the script).
Your output filenames do not contain any whitespace characters.
The last line in your input file matching the 1st pattern appears earlier in the input file than the last line matching the 2nd pattern.

If any of these assumptions are incorrect, the following script may need to be modified to make it work. But, if all of these assumptions are correct, the following script produces the output you requested if the input file contains the sample input in your 1st messsage in this thread and the file patterns contains:

Code:

3186 9876

as the first two fields on the first line:

Code:

#!/bin/ksh
f1=${1:-input}
f2=${2:-out1}
f3=${3:-out2}
read pat1 pat2 junk < patterns
ed -s "$f1" <<-END_ED
        1;?^$pat1?ka
        1;?^$pat2?kb
        1,'aw $f2
        'a+1,'bw $f3
        q
END_ED

This was written and tested using the Korn shell, but should work fine with any other shell that recognizes basic Bourne shell syntax or the shell syntax specified by the POSIX Standards and the Single UNIX Specifications.

The awk script Scrutinizer provided makes some of the above assumptions and also assumes that your second pattern can be found on the last line in your input file (which was true in your example). The ed script above allows other lines to follow the 2nd pattern and to not copy them to the 2nd file.

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

creating a fixed length output from a variable length input

Is there a command that sets a variable length? I have a input of a variable length field but my output for that field needs to be set to 32 char. Is there such a command? I am on a sun box running ksh Thanks

2. Shell Programming and Scripting

convert fixed length file to CSV

Newbie Looking for a script to convert my input file to delimited text file. Not familier with AWK or shell programing. Below is sample record in my input file and the expected output format. My OS is HPUX 11.23. Thanks in advance for your assistance. tbtbs input file:...

3. Shell Programming and Scripting

How to print string on screen according the fixed length?

Problem: entry_name="joke:hello:yellow:blue:default" print("%d %-12s\t%-10s\t%-5s\n", $i, $entry_name....); I just want to print the output like this index entry value .... 1 joke:hello:y 0 123 567 ellow:blue:d ...

4. UNIX for Dummies Questions & Answers

What the command to find out the record length of a fixed length file?

I want to find out the record length of a fixed length file? I forgot the command. Any body know?

5. UNIX for Dummies Questions & Answers

Convert a tab delimited/variable length file to fixed length file

Hi, all. I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file: 10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783 19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657 And this is the expected...

6. Shell Programming and Scripting

Splitting fixed length file using awk

Hi, I need to split a fixed length file of 160 characters based on value of a column. Example: ABC 456780001 DGDG SDFSF BCD 444440002 SSSS TTTTT ABC 777750003 HHHH UUUUU THH 888880001 FFFF LLLLLL HHH 999990002 GGGG OOOOO I need to split this file on basis of column from...

7. UNIX for Dummies Questions & Answers

Length of a fixed width file

I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on...

8. Shell Programming and Scripting

Break one long string into multiple fixed length lines

This is actually a KSH under Unix System Services (Z/OS), but hoping I can get a standard AIX/KSH solution to work... I have a very large, single line file in Windows, that we download via FTP, with the "SITE WRAP" option, into a Z/OS file with an LRECL of 200. This essentially breaks the single...

9. Shell Programming and Scripting

Generate fixed length txt file

hi, i am using below query to generate the fixed length txt file. this sql is being called from shell script. This is supposed to be a fixed record file with the below definitions. There must be 2 byte filler after the CAT_ID AND each line should have total of 270 bytes. field ...

LEARN ABOUT DEBIAN

flow-split

flow-split(1)						      General Commands Manual						     flow-split(1)

NAME

       flow-split -- Split flow files into smaller files.

SYNOPSIS

       flow-split [-gGhn]  [-b big|little]  [-C comment]  [-d debug_level]  [-N nflows]  [-o outfile_basename]	[-T nseconds]  [-z z_level]

DESCRIPTION

       The flow-split utility will split a flow file into smaller files based on the the number of flows or the ammount of time that has passed.

OPTIONS

       -b big|little
		 Byte order of output.

       -C Comment
		 Add a comment.

       -d debug_level
		 Enable debugging.

       -g	 Split on source tag.

       -G	 Split on destination tag.

       -h	 Display help.

       -n	 Use symbols for tag field in filename.

       -N nflows Split after processing nflows.

       -o outfile_basename
		 The basename of the resulting files.

       -T nsecond
		 Split after processing an interval of nseconds flows.

       -z z_level
		 Configure compression level to  z_level.  0 is disabled (no compression), 9 is highest compression.

EXAMPLES

       Create 1 minute flow files from the flow archive in /flows/krc4.  Store the results in /flows/krc4.split

	 flow-cat /flows/krc4 | flow-split -T60 -o /flows/krc4.split/1min.

BUGS

       None known.

AUTHOR

       Mark Fullmer maf@splintered.net

SEE ALSO

       flow-tools(1)

																     flow-split(1)