Split variable length and variable format CSV file
Dear all,
I have basic knowledge of Unix script and her I am trying to process variable length and variable format CSV file.
The file length will depend on the numbers of Earnings/Deductions/Direct Deposits.
And
The format will depend on whether it is Earnings/Deductions or Direct Deposits
For example, if employee has 2 earning, 2 deductions, and 2 direct deposits accounts, The record after EmployeeID will repeat twice for Earnings, repeat twice for Deductions, and repeat twice for direct deposits.
For earnings and deposits, the format are the same, they need to have earnings/deposit numbers then earnings/deposit amounts.
For direct deposits, the format is more complicated, it will require Bank Number, Account Type, Account Number, Amount,
Here is the sample data. The first line is header.
For Employee #1, he/she has 1 earning, 2 deductions, and 1 direct deposit, so his/her record will have one earning (01), two deductions (13 and 14), and one direct deposit account (010001001,ABC,123456)
For Employee #2, he/she has 2 earnings, 2 deduction,s and 2 direct deposits, so his/her record will have one earning (01 and 02), two deductions (11 and 12), and two direct deposit accounts (010001002,ABC,1212121 and 010001003,CDE,123123)
Since the numbers of Earnings/Deductions or Direct Deposits could vary, the length and format of the records in CSV file are different.
I have several options.
1. I can seperate the file into three files. One for Earning, one for deduction, and one for direct deposit. Each file will have employee id in the beginning of the record.
so for sample data will look like
Earning File
Deduction File
Direct Deposit File
or
2. I can further "normalize" each record into sepeate records in each files.
Earning File
Deduction File
Direct Deposit File
Thank you very much for your time to help me to resolve this issue.
chechun
Last edited by Franklin52; 02-05-2010 at 01:08 PM..
Reason: Please use code tags!
Unfortunately, I couldn't find a good solution in Unix shell script. I have resolved this issue using PL/SQL. I used several for loop and combination of substr and instr to resolve it. Thanks!
Hi All,
I have a file which has hundred of records with fixed number of fields. In each record there is set of 8 characters which represent the duration of that activity. I want to sum up the duration present in all the records for a report. The problem is the duration changes per record so I... (5 Replies)
Hi,
I'm new to unix, i have a variable length file like below,
01|Test|Test1|Sample|
02|AA|BB|CC|DD|
03|AAA|BBB|CCC|DDD|EEE|RRR|TTT|SSS|YYY|
I need to make this as a fixed length file. Assume that i have 10 columns in the DAT file.
for ex: the first 01 record is having 4cols -... (8 Replies)
I have the output of a process which on status change of the object being processed appends status,timestamp to the record in the text file...
so i end up with output something like:
task123,TERMINAL,glob,5,INITIAL,2012-02-27 16:48:07,PREPARING,2012-02-27 16:49:06,SCHEDULED,2012-02-27... (2 Replies)
Hi,
Can anyone help with a effective solution ?
I need to change a variable length text field (between 1 - 18 characters) to a fixed length text of 18 characters with the unused portion, at the end, filled with spaces.
The text field is actually field 10 of a .csv file however I could cut... (7 Replies)
Very, very new to unix scripting and have a unique situation. I have a file of records that contain 3 records types:
(H)eader Records
(D)etail Records
(T)railer Records
The Detail records are 82 bytes in length which is perfect. The Header and Trailer records sometimes are 82 bytes in... (3 Replies)
Hi I am new to shell scripting. I need to parse a file which contains the header and detail records and split into n of file based on dept ID, for ex.
INPUT FILE:
DEPT ID: 1
EMPNAME: XYZ
EMPAddress: XYZZZ
DEPT ID: 2
EMPNAME: ABC
EMPAddress: ABCD
DEPT ID: 1
EMPNAME: PQR
EMPAddress:... (6 Replies)
Hi, all.
I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file:
10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783
19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657
And this is the expected... (2 Replies)
Hi all,
Does anyone know a technique for creating a download file in variable length format? I have looked around but haven't found any resources on this (or, maybe I'm not sure what to Google for :) )
Thanks in advance! (2 Replies)
Any help to read the contents of a variable length csv ....??(using awk)
The csv mite look like this :
anjali,ram,rahul,mohini,sam,.... and so on ...
I need to pick up each name..
Thanks in advance
SD (3 Replies)
Is there a command that sets a variable length?
I have a input of a variable length field but my output for that field needs to be set to 32 char.
Is there such a command?
I am on a sun box running ksh
Thanks (2 Replies)