10-08-2007
Pivot variable record length file and change delimiter
Hi experts.
I got a file (500mb max) and need to pivot it (loading into ORCL) and change BLANK delimiter to PIPE |.
Sometimes there are multipel BLANKS (as a particular value may be BLANK, or simply two BLANKS instead of one BLANK).
thanks for your input!
Cheers,
Layout
<id>BLANK<record_type>BLANK<field_num>BLANK<field_value>BLANK<field_num> BLANK<field_value>BLANK ........\n
example: (005, 004 are record_type, one leading for each line, same with <id>)
1234 005 001 23 198 5 098 45 033 2.........090 12\n
1234 006 008 66 002 345 129 2 345 2 897 45 091 56788 123 56321 443 45.........100 33\n
I want to use awk to "massage" this file to this output:
<record_type> <field_num> <field_value>
<record_type> <field_num> <field_value>
Example:
1234|005|001|23\n
1234|005|198|5\n
1234|005|098|45\n
1234|005|033|2\n
...
1234|005|090|12\n
1234|006|008|66\n
1234|006|002|345\n
1234|006|129|2\n
1234|006|345|2\n
1234|006|897|45\n
1234|006|091|56788\n
1234|006|443|45\n
....
1234|006|100|33\n
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I need to pick a field out of a variable record - the field is always found 4 fields after a certain text string, but it can be on any line of the record and in any position across the record on a line. I have had no luck through any of the Unix editors being able to cut a field that isn't always... (17 Replies)
Discussion started by: Barb
17 Replies
2. UNIX for Dummies Questions & Answers
I want to find out the record length of a fixed length file? I forgot the command. Any body know? (9 Replies)
Discussion started by: tranq01
9 Replies
3. UNIX for Dummies Questions & Answers
Hi, all.
I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file:
10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783
19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657
And this is the expected... (2 Replies)
Discussion started by: Everton_Silveir
2 Replies
4. Shell Programming and Scripting
Very, very new to unix scripting and have a unique situation. I have a file of records that contain 3 records types:
(H)eader Records
(D)etail Records
(T)railer Records
The Detail records are 82 bytes in length which is perfect. The Header and Trailer records sometimes are 82 bytes in... (3 Replies)
Discussion started by: jclanc8
3 Replies
5. Shell Programming and Scripting
Hi,
I'm new to unix, i have a variable length file like below,
01|Test|Test1|Sample|
02|AA|BB|CC|DD|
03|AAA|BBB|CCC|DDD|EEE|RRR|TTT|SSS|YYY|
I need to make this as a fixed length file. Assume that i have 10 columns in the DAT file.
for ex: the first 01 record is having 4cols -... (8 Replies)
Discussion started by: Mohankumar Venu
8 Replies
6. Shell Programming and Scripting
Hi,
I have a No Delimiter variable length text file with following schema -
Column Name Data length
Firstname 5
Lastname 5
age 3
phoneno1 10
phoneno2 10
phoneno3 10
sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies
7. Shell Programming and Scripting
Hello,
I have a file with a date field with various lengths. For example:
m/d/yyyy hh:mm or h:mm
mm/dd/yyyy hh:mm or h:mm
Is there a way using sed or awk to change the field to m/d/y ? I don't need the hours and minutes in that field, just the date in the proper format.
Thanks in... (6 Replies)
Discussion started by: sonnyo916
6 Replies
8. Shell Programming and Scripting
Hi,
Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV
I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters.
Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies
9. Shell Programming and Scripting
My requirment is for every record of a particular file I've to check for a record delimeter (e.g. "\n") and if any row doesn't have "\n" then report it in error file .
Please suggest me to go through this. (4 Replies)
Discussion started by: manab86
4 Replies
10. Shell Programming and Scripting
Hi Team,
I have an issue to split the file which is having special chracter(German Char) using awk command.
I have a different length records in a file. I am separating the files based on the length using awk command.
The command is working fine if the record is not having any... (7 Replies)
Discussion started by: Anthuvan
7 Replies