Sponsored Content
Top Forums Shell Programming and Scripting Pivot variable record length file and change delimiter Post 302139520 by thomasr on Sunday 7th of October 2007 11:39:46 PM
Old 10-08-2007
Pivot variable record length file and change delimiter

Hi experts.

I got a file (500mb max) and need to pivot it (loading into ORCL) and change BLANK delimiter to PIPE |.
Sometimes there are multipel BLANKS (as a particular value may be BLANK, or simply two BLANKS instead of one BLANK).

thanks for your input!

Cheers,


Layout
<id>BLANK<record_type>BLANK<field_num>BLANK<field_value>BLANK<field_num> BLANK<field_value>BLANK ........\n

example: (005, 004 are record_type, one leading for each line, same with <id>)

1234 005 001 23 198 5 098 45 033 2.........090 12\n
1234 006 008 66 002 345 129 2 345 2 897 45 091 56788 123 56321 443 45.........100 33\n

I want to use awk to "massage" this file to this output:
<record_type> <field_num> <field_value>
<record_type> <field_num> <field_value>

Example:
1234|005|001|23\n
1234|005|198|5\n
1234|005|098|45\n
1234|005|033|2\n
...
1234|005|090|12\n
1234|006|008|66\n
1234|006|002|345\n
1234|006|129|2\n
1234|006|345|2\n
1234|006|897|45\n
1234|006|091|56788\n
1234|006|443|45\n
....
1234|006|100|33\n
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Parsing a variable length record

I need to pick a field out of a variable record - the field is always found 4 fields after a certain text string, but it can be on any line of the record and in any position across the record on a line. I have had no luck through any of the Unix editors being able to cut a field that isn't always... (17 Replies)
Discussion started by: Barb
17 Replies

2. UNIX for Dummies Questions & Answers

What the command to find out the record length of a fixed length file?

I want to find out the record length of a fixed length file? I forgot the command. Any body know? (9 Replies)
Discussion started by: tranq01
9 Replies

3. UNIX for Dummies Questions & Answers

Convert a tab delimited/variable length file to fixed length file

Hi, all. I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file: 10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783 19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657 And this is the expected... (2 Replies)
Discussion started by: Everton_Silveir
2 Replies

4. Shell Programming and Scripting

Make variable length record a fixed length

Very, very new to unix scripting and have a unique situation. I have a file of records that contain 3 records types: (H)eader Records (D)etail Records (T)railer Records The Detail records are 82 bytes in length which is perfect. The Header and Trailer records sometimes are 82 bytes in... (3 Replies)
Discussion started by: jclanc8
3 Replies

5. Shell Programming and Scripting

Adding a delimiter to a variable length file

Hi, I'm new to unix, i have a variable length file like below, 01|Test|Test1|Sample| 02|AA|BB|CC|DD| 03|AAA|BBB|CCC|DDD|EEE|RRR|TTT|SSS|YYY| I need to make this as a fixed length file. Assume that i have 10 columns in the DAT file. for ex: the first 01 record is having 4cols -... (8 Replies)
Discussion started by: Mohankumar Venu
8 Replies

6. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

7. Shell Programming and Scripting

How to change variable length field?

Hello, I have a file with a date field with various lengths. For example: m/d/yyyy hh:mm or h:mm mm/dd/yyyy hh:mm or h:mm Is there a way using sed or awk to change the field to m/d/y ? I don't need the hours and minutes in that field, just the date in the proper format. Thanks in... (6 Replies)
Discussion started by: sonnyo916
6 Replies

8. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

9. Shell Programming and Scripting

How to check record delimiter of a file ?

My requirment is for every record of a particular file I've to check for a record delimeter (e.g. "\n") and if any row doesn't have "\n" then report it in error file . Please suggest me to go through this. (4 Replies)
Discussion started by: manab86
4 Replies

10. Shell Programming and Scripting

Convert variable length record to fixed length

Hi Team, I have an issue to split the file which is having special chracter(German Char) using awk command. I have a different length records in a file. I am separating the files based on the length using awk command. The command is working fine if the record is not having any... (7 Replies)
Discussion started by: Anthuvan
7 Replies
STRFMON(3)						     Linux Programmer's Manual							STRFMON(3)

NAME
strfmon, strfmon_l - convert monetary value to a string SYNOPSIS
#include <monetary.h> ssize_t strfmon(char *s, size_t max, const char *format, ...); ssize_t strfmon_l(char *s, size_t max, locale_t locale, const char *" format , ...); DESCRIPTION
The strfmon() function formats the specified monetary amount according to the current locale and format specification format and places the result in the character array s of size max. The strfmon_l() function performs the same task, but uses the locale specified by locale. The behavior of strfmon_l() is undefined if locale is the special locale object LC_GLOBAL_LOCALE (see duplocale(3)) or is not a valid locale object handle. Ordinary characters in format are copied to s without conversion. Conversion specifiers are introduced by a '%' character. Immediately following it there can be zero or more of the following flags: =f The single-byte character f is used as the numeric fill character (to be used with a left precision, see below). When not speci- fied, the space character is used. ^ Do not use any grouping characters that might be defined for the current locale. By default, grouping is enabled. ( or + The ( flag indicates that negative amounts should be enclosed between parentheses. The + flag indicates that signs should be han- dled in the default way, that is, amounts are preceded by the locale's sign indication, for example, nothing for positive, "-" for negative. ! Omit the currency symbol. - Left justify all fields. The default is right justification. Next, there may be a field width: a decimal digit string specifying a minimum field width in bytes. The default is 0. A result smaller than this width is padded with spaces (on the left, unless the left-justify flag was given). Next, there may be a left precision of the form "#" followed by a decimal digit string. If the number of digits left of the radix charac- ter is smaller than this, the representation is padded on the left with the numeric fill character. Grouping characters are not counted in this field width. Next, there may be a right precision of the form "." followed by a decimal digit string. The amount being formatted is rounded to the specified number of digits prior to formatting. The default is specified in the frac_digits and int_frac_digits items of the current locale. If the right precision is 0, no radix character is printed. (The radix character here is determined by LC_MONETARY, and may dif- fer from that specified by LC_NUMERIC.) Finally, the conversion specification must be ended with a conversion character. The three conversion characters are % (In this case, the entire specification must be exactly "%%".) Put a '%' character in the result string. i One argument of type double is converted using the locale's international currency format. n One argument of type double is converted using the locale's national currency format. RETURN VALUE
The strfmon() function returns the number of characters placed in the array s, not including the terminating null byte, provided the string, including the terminating null byte, fits. Otherwise, it sets errno to E2BIG, returns -1, and the contents of the array is unde- fined. ATTRIBUTES
For an explanation of the terms used in this section, see attributes(7). +------------+---------------+----------------+ |Interface | Attribute | Value | +------------+---------------+----------------+ |strfmon() | Thread safety | MT-Safe locale | +------------+---------------+----------------+ |strfmon_l() | Thread safety | MT-Safe | +------------+---------------+----------------+ CONFORMING TO
POSIX.1-2001, POSIX.1-2008. EXAMPLE
The call strfmon(buf, sizeof(buf), "[%^=*#6n] [%=*#6i]", 1234.567, 1234.567); outputs [EUR **1234,57] [EUR **1 234,57] in the nl_NL locale. The de_DE, de_CH, en_AU, and en_GB locales yield [ **1234,57 EUR] [ **1.234,57 EUR] [ Fr. **1234.57] [ CHF **1'234.57] [ $**1234.57] [ AUD**1,234.57] [ L**1234.57] [ GBP**1,234.57] SEE ALSO
duplocale(3), setlocale(3), sprintf(3), locale(7) Linux 2017-09-15 STRFMON(3)
All times are GMT -4. The time now is 12:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy