08-03-2013
Generated csv files size are more than 3GB
Hi
i have a shell script which will generate the CSV files from a group of XML files.
but the XML files are in 50K plus. so the generated CSV files are more then 4 GB in size. is there any way i can take care in the scripting itself to make the shell script efficent to write data into CSV files to save the size of generating.
Thanks & Regards,
Sunny
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I have one file stat.
Stat file contents are as follows: for example.
H50768020040913,00260100,507680,13,0000000643,0000000643,00000,0000
H50769520040808,00260100,507695,13,0000000000,0000000000,00000,0000 H50770620040611,00260100,507706,13,0000000000,0000000000,00000,0000
Now i... (1 Reply)
Discussion started by: davidpreml
1 Replies
2. UNIX for Dummies Questions & Answers
How to list all the files which are not generated today, and move all the above files to backup dir. (2 Replies)
Discussion started by: redlotus72
2 Replies
3. Shell Programming and Scripting
Hi All,
I'm working on a ftp shell script in which I'm tranfering files from one sever to another using ftp.
Some program generates files at undefined time & throughout the day.
I have to transfer the files time to time.. i.e. once the file is generated, it should be transfered at the very... (3 Replies)
Discussion started by: im_new
3 Replies
4. Programming
Helo,
my server side system is running on a redhat linux o.s.
I have c program on the server which export list file into CSV format. now,
I want that my program on server side is able to add support for
CRLF(carriage return Line feed)into csv file format. so how do i write C program whcih... (3 Replies)
Discussion started by: amitpansuria
3 Replies
5. UNIX for Advanced & Expert Users
Hi All,
I was checking some of the files and I got the following entries:-
===============
v, 664, serv, serv, version.txt, exe
L, 775, serv, serv, start.sh, eventserv
================
Could someone please tell me what does the type"v" and "L" represent to.
I have not... (2 Replies)
Discussion started by: shubhranshu
2 Replies
6. Solaris
Hi,
We have an application ASPA . The application related processes are running in /ASPA/bin directory . now whenever a process terminates abruptly , a core file should be generated (correct me if i am wrong) in the
/ASPA/bin directory . But i am not able to see any such files . The... (4 Replies)
Discussion started by: asalman.qazi
4 Replies
7. Shell Programming and Scripting
Hello,
Can any one please assist how to scirpt it:
Every day a new log file is create and I want to process only the one generated yesterday and get the data of column 3 and 6.
For example today's date is 24 then I want to get the data of log file created on 23rd.
Log Files in... (7 Replies)
Discussion started by: sureshcisco
7 Replies
8. Solaris
Hi All,
New to unix. Here is the problem. Running a script that extracts data from hyperion essbase and generates a file in unix. This script fails most of the times with a very low success rate. The data has increased a lot in the last few months resulting in the file being more than 2 gb.
... (2 Replies)
Discussion started by: noufalshaw
2 Replies
9. Shell Programming and Scripting
Hi
I have the following files generated by different users on a directory
-rw-rw-r-- 1 NAME1 database03 809 Nov 17 10:41 PCAS_CARD_TRANS_OFF.1111171041.lg
-rw-rw-r-- 1 richard ccsdba 10968411 Nov 17 10:43 load_123_RX0_0.1111171016.lg
-rw-rw-r-- 1 DEV db03 10713 Nov 17... (5 Replies)
Discussion started by: bobby1015
5 Replies
10. Shell Programming and Scripting
I would like to find the Files which are generated today in the current directory:
I use the commad ls -lrt * | egrep " `date "+%b"` * `date "+%d"`
to acheive this. Is there any better way to acquire the same.
Multiple answers will be great. Thanks (3 Replies)
Discussion started by: kusathy
3 Replies
LEARN ABOUT SUSE
xmlstarlet
XMLSTARLET(1) xmlstarlet Manual XMLSTARLET(1)
NAME
xmlstarlet - command line XML/XSLT toolkit
SYNOPSIS
xmlstarlet [<options>] [<command>] [<cmd-options>]
INTRODUCTION
XMLStarlet is a set of command line utilities (tools) which can be used to transform, query, validate, and edit XML documents and files us-
ing simple set of shell commands in similar way it is done for plain text files using UNIX grep, sed, awk, diff, patch, join, etc commands.
This set of command line utilities can be used by those who deal with many XML documents on UNIX shell command prompt as well as for auto-
mated XML processing with shell scripts.
OPTIONS
--version
Display the version of xmlstarlet.
--help Display help.
COMMANDS
Type: xml <command> --help <ENTER> for command help
Available commands include:
ed (or edit)
Edit/update XML document(s).
sel (or select)
Select data or query XML document(s) (XPATH, etc).
tr (or transform)
Transform XML documents(s) using XSLT.
val (or validate)
Validate XML document(s) (well-formed/DTD/XSD/RelaxNG).
fo (or format)
Format XML document(s).
el (or elements)
Display element structure of XML document.
c14n (or canonic)
XML canonicalization.
ls (or list)
List directory as XML.
esc (or escape)
Escape special XML characters.
unesc (or unescape)
Unescape special XML characters.
pyx (or xmln)
Convert XML into PYX format (based on ESIS - ISO 8879).
p2x (or depyx)
Convert PYX into XML.
REFERENCES
XMLStarlet is a command line toolkit to query/edit/check/transform XML documents (for more information see http://xmlstar.source-
forge.net/).
AUTHOR
Mikhail Grushinskiy.
XMLSTARLET(1)