Sponsored Content
Full Discussion: Help with Split shell script
Top Forums Shell Programming and Scripting Help with Split shell script Post 302304489 by amtrip on Monday 6th of April 2009 03:03:21 PM
Old 04-06-2009
Help with Split shell script

Hi all,

Following is the shell script which I am using for splitting the contents from a flat file to the multiple XMLs. The problem is that this script is working file when the input file is having 10000 lines. When the number of lines increases; the performance degrades drastically. Please help me ASAP for this:

Code:
 
#!/bin/bash
# This Shell Script reads the Input Master XML file line by line and splits into multiple XMLs.
        #Name of the Input Master XML file
        file=$1/MASTER_XML.out
        #Creating Directory in AFSOutput Folder
        outputdir=`date +%m%d%Y_%H%M`
        mkdir $1/$outputdir
        chmod 777 $1/$outputdir
        #Reading the filename column from the XML
        filename=($(grep -o "[0-9A-Za-z]\+\.[0-9A-Za-z]\+\.[tif]\{2,3\}" $1/MASTER_XML.out))
        x=0
        while [ $x -lt $(wc -l <$file) ]
                do
                        let x=x+1
 head -n $x $file | tail -n  1  >$1/$outputdir/${filename["$x-1"]%.*}.xml
 chmod 777 $1/$outputdir/${filename["$x-1"]%.*}.xml
 done
        exit 0

Please help

Thanks,
Amit

Last edited by jim mcnamara; 04-06-2009 at 04:25 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to split XML file

Hi, I'm experiencing difficulty in loading an XML file to an Oracle destination table.I keep running into a memory problem due to the large size of the file. I want to split the XML file into several smaller files based on the keyword(s)/tags : '' and '' and would like to use a Unix shell... (2 Replies)
Discussion started by: bayflash27
2 Replies

2. UNIX for Dummies Questions & Answers

awk Shell Script error : "Syntax Error : `Split' unexpected

hi there i write one awk script file in shell programing the code is related to dd/mm/yy to month, day year format but i get an error please can anybody help me out in this problem ?????? i give my code here including error awk ` # date-month -- convert mm/dd/yy to month day,... (2 Replies)
Discussion started by: Herry
2 Replies

3. Shell Programming and Scripting

Shell script to parse/split input string and display the tokens

Hi, How do I parse/split lines (strings) read from a file and display the individual tokens in a shell script? Given that the length of individual lines is not constant and number of tokens in each line is also not constant. The input file could be as below: ... (3 Replies)
Discussion started by: yajaykumar
3 Replies

4. Shell Programming and Scripting

SPLIT STRING in bash shell script

i need one help.... if i have a string like aaaaa,bbbbb,ccccc,aaaaa How to to split the string and check howmany times aaaaa will be in that string? Thanks (7 Replies)
Discussion started by: karthinvk
7 Replies

5. Shell Programming and Scripting

Split line to multiple files Awk/Sed/Shell Script help

Hi, I need help to split lines from a file into multiple files. my input look like this: 13 23 45 45 6 7 33 44 55 66 7 13 34 5 6 7 87 45 7 8 8 9 13 44 55 66 77 8 44 66 88 99 6 I want to split every 3 lines from this file to be written to individual files. (3 Replies)
Discussion started by: saint2006
3 Replies

6. Shell Programming and Scripting

Shell Script:split the values

Hi, I have file like below as file.txt Linux sptpp1a 2.6.18 348.1.1.el5 xxxxx 05/02/2013 %usr %sys %wio %idle 14:18:30 0 1 0 99 14:18:40 1 1 0 98 14:18:50 1 1 0 98 14:19:00 ... (5 Replies)
Discussion started by: Mani_apr08
5 Replies

7. Shell Programming and Scripting

How to split files using shell script?

solid top facet normal 0 1 0 outer loop vertex 0 1 0 vertex 1 1 1 vertex 1 1 0 endloop endfacet facet normal 0 1 0 outer loop vertex 0 1 0 vertex 0 1 1 vertex 1 1 1 endloop endfacet endsolid top solid bottom facet normal 0 -1 ... (3 Replies)
Discussion started by: linuxUser_
3 Replies

8. Shell Programming and Scripting

Split a file into two different file using shell script

could you please provide solution to split file and load into different files in shell script. cat test.txt "01"|""emp_name"":ram|""emp_sal"":600000|""emp_value"":""CREDITCARD""|""1410"":] "02"|""emp_name"":syam|""emp_sal"":500000|""emp_value"":""voucher""|""1610"":]|""1510"":]... (7 Replies)
Discussion started by: shabeena
7 Replies

9. UNIX for Beginners Questions & Answers

Shell script to split data with a delimiter having chars and special chars

Hi Team, I have a file a1.txt with data as follows. dfjakjf...asdfkasj</EnableQuotedIDs><SQL><SelectStatement modified='1' type='string'><! The delimiter string: <SelectStatement modified='1' type='string'><! dlm="<SelectStatement modified='1' type='string'><! The above command is... (7 Replies)
Discussion started by: kmanivan82
7 Replies

10. UNIX for Beginners Questions & Answers

Shell script to Split matrix file with delimiter into multiple files

I have a large semicolon delimited file with thousands of columns and many thousands of line. It looks like: ID1;ID2;ID3;ID4;A_1;B_1;C_1;A_2;B_2;C_2;A_3;B_3;C_3 AA;ax;ay;az;01;02;03;04;05;06;07;08;09 BB;bx;by;bz;03;05;33;44;15;26;27;08;09 I want to split this table in to multiple files: ... (1 Reply)
Discussion started by: trymega
1 Replies
PUPPET-DOC(8)							   Puppet manual						     PUPPET-DOC(8)

NAME
puppet-doc - Generate Puppet documentation and references SYNOPSIS
Generates a reference for all Puppet types. Largely meant for internal Puppet Labs use. USAGE
puppet doc [-a|--all] [-h|--help] [-o|--outputdir rdoc-outputdir] [-m|--mode text|pdf|rdoc] [-r|--reference reference-name] [--charset charset] [manifest-file] DESCRIPTION
If mode is not 'rdoc', then this command generates a Markdown document describing all installed Puppet types or all allowable arguments to puppet executables. It is largely meant for internal use and is used to generate the reference document available on the Puppet Labs web site. In 'rdoc' mode, this command generates an html RDoc hierarchy describing the manifests that are in 'manifestdir' and 'modulepath' configu- ration directives. The generated documentation directory is doc by default but can be changed with the 'outputdir' option. If the command is run with the name of a manifest file as an argument, puppet doc will output a single manifest's documentation on stdout. OPTIONS
--all Output the docs for all of the reference types. In 'rdoc' mode, this also outputs documentation for all resources. --help Print this help message --outputdir Used only in 'rdoc' mode. The directory to which the rdoc output should be written. --mode Determine the output mode. Valid modes are 'text', 'pdf' and 'rdoc'. The 'pdf' mode creates PDF formatted files in the /tmp direc- tory. The default mode is 'text'. In 'rdoc' mode you must provide 'manifests-path' --reference Build a particular reference. Get a list of references by running 'puppet doc --list'. --charset Used only in 'rdoc' mode. It sets the charset used in the html files produced. --manifestdir Used only in 'rdoc' mode. The directory to scan for stand-alone manifests. If not supplied, puppet doc will use the manifestdir from puppet.conf. --modulepath Used only in 'rdoc' mode. The directory or directories to scan for modules. If not supplied, puppet doc will use the modulepath from puppet.conf. --environment Used only in 'rdoc' mode. The configuration environment from which to read the modulepath and manifestdir settings, when reading said settings from puppet.conf. Due to a known bug, this option is not currently effective. EXAMPLE
$ puppet doc -r type > /tmp/type_reference.markdown or $ puppet doc --outputdir /tmp/rdoc --mode rdoc /path/to/manifests or $ puppet doc /etc/puppet/manifests/site.pp or $ puppet doc -m pdf -r configuration AUTHOR
Luke Kanies COPYRIGHT
Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License Puppet Labs, LLC June 2012 PUPPET-DOC(8)
All times are GMT -4. The time now is 03:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy