07-02-2015
If the data can be sorted hierarchically first (man sort) on the key starting with rltp_id, then prod_id, then cust_id, then tnx_id... then you can write script in a single pass to convert the data (trying to give hints rather than doing your homework).
20 million records... well... takes more space than most, but very doable...
You could use perl or awk to do this (or python or ruby or whatever)... You could use shell (bourne shell script variants), but might not be as easy especially since you want it to be relatively fast and also avoid duplicates.
I'm not trying to be mean, just trying to encourage your own work.
If you get too frustrated I might whip up a bourne (pure) example just as a challenge....
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Dear all,
Can you send me a script file the changes .xml to .csv file.
Thanks,
Srinivasa (4 Replies)
Discussion started by: srinivasaphani
4 Replies
2. Shell Programming and Scripting
Dear Yogesh..."Convert XML file into TEXT file using PERL seript"... can u help me regarding this issue...plz
Thanks
Rudro (0 Replies)
Discussion started by: Rudro
0 Replies
3. UNIX for Dummies Questions & Answers
I have a .csv file in Unix box i need a UNIX script to convert the.csv files to.xls format.
Its very urgent please help me. (1 Reply)
Discussion started by: moon_friend
1 Replies
4. Shell Programming and Scripting
Hi
I have a excel file in unix machine and have to convert it into a .csv file.I have to do this from a unix script.How do we do this?
Thanks
Abhinav (3 Replies)
Discussion started by: akashtcs
3 Replies
5. Shell Programming and Scripting
I have huge xml file in server and i want to convert it to .csv with specific column ...
i have search in blog but i didn't get any usefully command.
Thanks in advance (1 Reply)
Discussion started by: pareshkp
1 Replies
6. Shell Programming and Scripting
Hi Guys,
I am new to Shell scripting and need to convert an XML files to a CSV file.
My actual problem is that XML file loading is taking hours and I have decided to convert the XML structure to row based data in a CSV file.
My XML file: Message846 can repeat within main loop and... (1 Reply)
Discussion started by: qamar.shahbaz
1 Replies
7. Shell Programming and Scripting
Hi All,
I've got in a situation where I need to convert .xlsx or .xls formatted files into simple text file or .csv file.
I've found many options but doing this using PERL script is the best way I believe.I'm in AIX box.
Perl code should have 2 params while running. i.e
perl... (1 Reply)
Discussion started by: manab86
1 Replies
8. Shell Programming and Scripting
Is there a Unix or Perl script that converts JSON files to CSV or tab delimited format? We are running AIX 6.1.
Thanks in advance! (1 Reply)
Discussion started by: warpmail
1 Replies
9. Shell Programming and Scripting
Hello, I need to get few values from a XML file and output needs to be written in another file with pipe delimited format. The Header & Footer of the Pipe Delimited file will be constant.
The below is my sample XML file. I need to pull the values in between the XML tags <Operator_info to... (15 Replies)
Discussion started by: karthi1305561
15 Replies
10. Shell Programming and Scripting
I have input data looks like this which is a part of a csv file
7,1265,76548,"0102:04"
8,1266,76545,"0112:04"
I need to make the output data should look like this and the output data will be part of text file:
7|1265000 |7654899 |A|
8|12660000 |76545999 |B|
The logic behind the... (6 Replies)
Discussion started by: RJG
6 Replies
USE(1) User Commands USE(1)
NAME
use - Frontend to the Usepackage Environment Manager
SYNOPSIS
csh and derivatives:
source /usr/share/usepackage/use.csh
bourne shell and derivatives:
source /usr/share/usepackage/use.bsh
korn shell:
. /usr/share/usepackage/use.ksh
use [-vs] [-f file] package ...
use -l
DESCRIPTION
Usepackage is an environment management program. It is based on the principle of packages - collections of executables that share a common
set of necessary environment variables, such as PATH, MANPATH or LD_LIBRARY_PATH.
For each given package, use sources the appropriate environment information into the current shell. The environment information is speci-
fied in a configuration file, see usepackage(1).
OPTIONS
-v Output verbose information to the standard error stream.
-s Silence warnings for un-matched packages. This is useful in a shell rc script when a package is known not to be available on all
architectures that the shell is used on.
-f file
Specify an alternate initial configuration file.
-l List available packages and groups.
FILES
/usr/share/usepackage/usepackage.conf
The default configuration file.
/usr/share/usepackage/use.csh
Shell setup for csh and derivatives.
/usr/share/usepackage/use.bsh
Shell setup for bourne shell and derivatives.
/usr/share/usepackage/use.ksh
Shell setup for ksh.
/usr/bin/usepackage
The underlying Usepackage executable.
ENVIRONMENT
Other than the reading and re-definition of environment variables for package setup, use also uses the following environment variables for
user configuration:
PACKAGES_PATH Colon-separated path list giving the directories to search for configuration files. Shell-style tilde (~) user-directory
escapes are expanded.
HOME If present in the environment, this is used to provide the expansion for the tilde (~) user-directory.
SHELL If present in the environment, the last path component of this is used for shell matching (see SYNTAX) and detecting the
style of environment output that should be used, see usepackage(1).
COPYRIGHT
Usepackage Environment Manager
Copyright (C) 1995-2005 Jonathan Hogg
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER-
CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation,
Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
SEE ALSO
usepackage(1), csh(1), sh(1), ksh(1)
Usepackage $Date: 2005/12/11 16:42:09 $ USE(1)