Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Splitting a file in to multiple files and passing each individual file to a command

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Splitting a file in to multiple files and passing each individual file to a command
# 1  
Old 11-17-2011
Question Splitting a file in to multiple files and passing each individual file to a command
I have an input file with contents like:
MainFile.dat:
Code:
12247689|7896|77698080
16768900|hh78|78959390
12247689|7896|77698080
16768900|hh78|78959390
12247689|7896|77698080
16768900|hh78|78959390
12247689|7896|77698080
16768900|hh78|78959390
12247689|7896|77698080
16768900|hh78|78959390

I need to split this file in to 10 with each having extension as _temp1,_temp2,_temp3 and so on

and pass each of this file to a 10 processes each to a process:
Code:
RemoveUsage File1
RemoveUsage file2
RemoveUsage file3

etc..


Can anyone help me in this Smilie

Moderator's Comments:
Mod Comment Please use code tags!
Last edited by zaxxon; 11-17-2011 at 03:03 AM.. Reason: code tags, see PM
# 2  
Old 11-17-2011
Each line to a single file? What file names to be used besides the extension? What 10 processes ".. each process"? I do not really understand what you want, sorry. Maybe describe a bit more so that others that do not know what you want to do get a chance to understand to minimize guessing, thanks.
Btw: What have you tried so far?
# 3  
Old 11-17-2011
Quote:
Originally Posted by zaxxon
Each line to a single file? What file names to be used besides the extension? What 10 processes ".. each process"? I do not really understand what you want, sorry. Maybe describe a bit more so that others that do not know what you want to do get a chance to understand to minimize guessing, thanks.
Btw: What have you tried so far?

The file is 0f about 6000 lines I want to split the file into 10 files so that each file may be having around 600 lines.

The divided files need to be passed as a argument to a tool which takes the input as filename and process it.
RemoveUsage <fileName>

as there are 10 such files I need 10 processess to run parallely with each processing a file.
# 4  
Old 11-17-2011
Try this...
Code:
split -b 230 input_file
for file in x*
do
  RemoveUsage $file &
done

230 is the size of 10 lines... it is based on the input you have pasted...

--ahamed
# 5  
Old 11-17-2011
Quote:
Originally Posted by ahamed101
Try this...
Code:
split -b 230 input_file
for file in x*
do
  RemoveUsage $file &
done

230 is the size of 10 lines... it is based on the input you have pasted...

--ahamed
The size of the line may vary here...the input file may contain different line sizes..what to do in this case :-(
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script for splitting file of records into multiple files

Hello I have a file of following format HDR 1234 abc qwerty abc def ghi jkl HDR 4567 xyz qwerty abc def ghi jkl HDR 890 mno qwerty abc def ghi jkl HDR 1234 abc qwerty abc def ghi jkl HDR 1234 abc qwerty abc def ghi jkl -Need to split this into multiple files based on tag... (8 Replies)
Discussion started by: wincrazy
8 Replies

2. Shell Programming and Scripting

Splitting file into multiple files and renaming them

Hi all, Newbie here. First of all, sorry if I made any mistakes while posting this question in terms of rules. Correct me if I am wrong. :b: I have a .dat file whose name is in the format of 20170311_abc_xyz.dat. The file consists of records whose first column consists of multiple dates in... (2 Replies)
Discussion started by: chanduris
2 Replies

3. Shell Programming and Scripting

Splitting a single file to multiple files

Hi Friends , Please guide me with the code to extract multiple files from one file . The File Looks like ( Suppose a file has 2 tables list ,column length may vary ) H..- > File Header.... H....- >Table 1 Header.... D....- > Table 1 Data.... T....- >Table 1 Trailer.... H..-> Table 2... (1 Reply)
Discussion started by: AspiringD
1 Replies

4. Shell Programming and Scripting

Using a single "find" cmd to search for multiple file types and output individual files

Hi All, I am new here but I have a scripting question that I can't seem to figure out with the "find" cmd. What I am trying to do is to only have to run a single find cmd parsing the directories and output the different file types to induvidual files and I have been running into problems.... (3 Replies)
Discussion started by: swaters
3 Replies

5. UNIX for Dummies Questions & Answers

Splitting up a text file into multiple files by columns

Hi, I have a space delimited text file with multiple columns 102 columns. I want to break it up into 100 files labelled 1.txt through 100.txt (n.txt). Each text file will contain the first two columns and in addition the nth column (that corresponds to n.txt). The third file will contain the... (1 Reply)
Discussion started by: evelibertine
1 Replies

6. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

7. Shell Programming and Scripting

splitting a file (xml) into multiple files

To split the files Hi, I'm having a xml file with multiple xml header. so i want to split the file into multiple files. Test.xml --------- <?xml version="UTF_8"> <emp: ....> <name>a</name> <age>10</age> </emp> <?xml version="UTF_8"> <emp: ....> <name>b</name> <age>10</age>... (11 Replies)
Discussion started by: sasi_u
11 Replies

8. Shell Programming and Scripting

help splitting a file into multiple files in bash

I have a file that logs multiple sessions. What I would like to do is split this file inclusive of the lines that include "starting session" and "shutting down" and ignore the data before and after the beginning of the first session and the end of the last session. The output files can be called... (2 Replies)
Discussion started by: elinenbe
2 Replies

9. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies

10. UNIX for Dummies Questions & Answers

scripting: multiple values from file passing to command

one of my colleagues has this question. he has a command, C_CMD which accepts 4 variables, $1 $2 $3 $4 he wants to load up a file with multiple rows, one row per set of variables and then iteratively execute the command based on the content of the file. example: at the command line you'd... (5 Replies)
Discussion started by: LisaS
5 Replies
Login or Register to Ask a Question