|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Split long record into csv file
Post 302314170 by wvdeijk on Thursday 7th of May 2009 02:24:10 PM
05-07-2009
Finally managed to implement the setFieldsByWidth solution, and got it working once I realised that the function is an awk function, not a shell function (which would reside in my $FPATH)
. I do however need a little more help if possible - I have a lot of fields (200+) and the awk script is now erroring out because the FIELDWIDTHS variable is (much) longer than 399 characters. Is there an easy way round this.
A second problem is that I have 2 different record type in the file, which would require 2 different FIELDWIDTHS variables. Is it possible to do this, or would it be better to split the input file into 2 separate files before parsing?
Thanks for your help.
. I do however need a little more help if possible - I have a lot of fields (200+) and the awk script is now erroring out because the FIELDWIDTHS variable is (much) longer than 399 characters. Is there an easy way round this.A second problem is that I have 2 different record type in the file, which would require 2 different FIELDWIDTHS variables. Is it possible to do this, or would it be better to split the input file into 2 separate files before parsing?
Thanks for your help.
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi
Guys
I have downloaded a table from oracle database in .csv format. it has many fields as Title, First Name, Last Name etc. I have to download distinct titles from database and now i have to check all those titles from data of First Name one by one. and then i have to delete matched record.... (1 Reply)
Discussion started by: Rajeev Agrawal
1 Replies
2. UNIX for Dummies Questions & Answers
HI ,
I am new to shell scripting ,
I have a requirement that
I send a file for data quality ( original.csv) & i will be getting 4 files daily into a particular directory in return with cleansed data .
the files may be clean.csv,
unclean.csv ,
... (2 Replies)
Discussion started by: sirik
2 Replies
3. Shell Programming and Scripting
-Hi, I have a problem with parcing/spliting a file record into two parts and assigning the split parts to two viriables. The record is as follows:
ftrn facc ttrd feed xref fsdb fcp ruldb csdb omom fordr ftxn fodb fsdc texc oxox reng ttrn ttxn fqdb ... (5 Replies)
Discussion started by: aoussenko
5 Replies
4. Shell Programming and Scripting
Hello Gurus,
We have a requirement to count the valid number of records in a comma delimited file with double quotes.
The catch here is..few records have a new line carriage within the double quotes,,say for ex:we have a file called accounts the record count is 4827..but the actual valid count... (5 Replies)
Discussion started by: ajaykk
5 Replies
5. Shell Programming and Scripting
Hi,
I have a script which creates and modifies a csv file.
I have managed to do everything I need to do apart from 1 thing. I need to append a trailer record to the file. I need this line to hold the total of an entire column of the csv file (skipping the 1st line which is a header).
Can... (2 Replies)
Discussion started by: mcclunyboy
2 Replies
6. Shell Programming and Scripting
Hello All,
I have a large file, more than 50,000 lines, and I want to split it in even 5000 records. Which I can do using
sed '1d;$d;' <filename> | awk 'NR%5000==1{x="F"++i;}{print > x}'Now I need to add one more condition that is not to break the file at 5000th record if the 5000th record... (20 Replies)
Discussion started by: ibmtech
20 Replies
7. Shell Programming and Scripting
Hi,
I have to output a new csv file from an input csv file with first unique value in the first column.
input csv file
color product id status
green 102 pass
yellow 201 hold
yellow 202 keep
green 101 ok
green 103 hold
yellow 203 ... (5 Replies)
Discussion started by: Chris LAU
5 Replies
8. Shell Programming and Scripting
riends
I have the following problem:
test.txt I have a file that has the following contents:
is a fixed-length file to the end of the number 12 has spaces, so that it is fixed length
123456789
123456789
123456789
12
This code shows me the length of each record, but in... (2 Replies)
Discussion started by: tricampeon81
2 Replies
9. Shell Programming and Scripting
I was wondering if anyone could explain to me how to split a variable length EBCDIC file into seperate files based on the record key. I have the COBOL layout, and so I need to split the file into 13 different EBCDIC files so that I can run each one through a C++ converter I have, and get the... (11 Replies)
Discussion started by: hanshot1stx
11 Replies
10. UNIX for Advanced & Expert Users
Hi,
I have received a file which is 20 GB. We would like to split the file into 4 equal parts and process it to avoid memory issues.
If the record delimiter is unix new line, I could use split command either with option l or b.
The problem is that the line terminator is |##|
How to use... (5 Replies)
Discussion started by: Ravi.K
5 Replies
LEARN ABOUT OPENDARWIN
ucblinks
ucblinks(1B) SunOS/BSD Compatibility Package Commands ucblinks(1B) NAME ucblinks - adds /dev entries to give SunOS 4.x compatible names to SunOS 5.x devices SYNOPSIS /usr/ucb/ucblinks [-e rulebase] [-r rootdir] DESCRIPTION ucblinks creates symbolic links under the /dev directory for devices whose SunOS 5.x names differ from their SunOS 4.x names. Where possi- ble, these symbolic links point to the device's SunOS 5.x name rather than to the actual /devices entry. ucblinks does not remove unneeded compatibility links; these must be removed by hand. ucblinks should be called each time the system is reconfiguration-booted, after any new SunOS 5.x links that are needed have been created, since the reconfiguration may have resulted in more compatibility names being needed. In releases prior to SunOS 5.4, ucblinks used a nawk rule-base to construct the SunOS 4.x compatible names. ucblinks no longer uses nawk for the default operation, although nawk rule-bases can still be specifed with the -e option. The nawk rule-base equivalent to the SunOS 5.4 default operation can be found in /usr/ucblib/ucblinks.awk. OPTIONS -e rulebase Specify rulebase as the file containing nawk(1) pattern-action statements. -r rootdir Specify rootdir as the directory under which dev and devices will be found, rather than the standard root directory /. FILES /usr/ucblib/ucblinks.awk sample rule-base for compatibility links ATTRIBUTES See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWscpu | +-----------------------------+-----------------------------+ SEE ALSO devlinks(1M), disks(1M), ports(1M), tapes(1M), attributes(5) SunOS 5.10 13 Apr 1994 ucblinks(1B)