Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Help me pls : splitting single file in unix into different files based on data
Top Forums Shell Programming and Scripting Help me pls : splitting single file in unix into different files based on data Post 302712981 by Ravindra Swan on Wednesday 10th of October 2012 06:03:09 AM
Old 10-10-2012
O/P for the first command-filenames with size(101 files)
-rw-r--r-- 1 a8hh8 abiuser 286975 Oct 10 05:51 Filter_by_Expression101
-rw-r--r-- 1 a8hh8 abiuser 2932 Oct 10 05:51 Replicate87
-rw-r--r-- 1 a8hh8 abiuser 5194 Oct 10 05:51 Replicate71
-rw-r--r-- 1 a8hh8 abiuser 2403 Oct 10 05:51 Replicate70
-rw-r--r-- 1 a8hh8 abiuser 9217 Oct 10 05:51 Reformat88
-rw-r--r-- 1 a8hh8 abiuser 8119 Oct 10 05:51 reformat74
-rw-r--r-- 1 a8hh8 abiuser 8357 Oct 10 05:51 Reformat67
-rw-r--r-- 1 a8hh8 abiuser 7793 Oct 10 05:51 Partition_by_Key_and_Sort85
-rw-r--r-- 1 a8hh8 abiuser 4207 Oct 10 05:51 Lookup_File81
-rw-r--r-- 1 a8hh8 abiuser 2042 Oct 10 05:51 Lookup_File80
-rw-r--r-- 1 a8hh8 abiuser 2256 Oct 10 05:51 Input_File78
-rw-r--r-- 1 a8hh8 abiuser 2231 Oct 10 05:51 Input_File68
-rw-r--r-- 1 a8hh8 abiuser 2442 Oct 10 05:51 Gather76
-rw-r--r-- 1 a8hh8 abiuser 6647 Oct 10 05:51 Filter_by_Expression99
-rw-r--r-- 1 a8hh8 abiuser 6647 Oct 10 05:51 Filter_by_Expression98
-rw-r--r-- 1 a8hh8 abiuser 7702 Oct 10 05:51 Filter_by_Expression92
-rw-r--r-- 1 a8hh8 abiuser 9022 Oct 10 05:51 Filter_by_Expression84
-rw-r--r-- 1 a8hh8 abiuser 6322 Oct 10 05:51 Filter_by_Expression72
-rw-r--r-- 1 a8hh8 abiuser 6647 Oct 10 05:51 Filter_by_Expression100
-rw-r--r-- 1 a8hh8 abiuser 6884 Oct 10 05:51 Dedup_Sorted97
-rw-r--r-- 1 a8hh8 abiuser 7100 Oct 10 05:51 Dedup_Sorted93
-rw-r--r-- 1 a8hh8 abiuser 7104 Oct 10 05:51 Dedup_Sorted90
-rw-r--r-- 1 a8hh8 abiuser 7103 Oct 10 05:51 Dedup_Sorted83
-rw-r--r-- 1 a8hh8 abiuser 7077 Oct 10 05:51 Dedup_Sorted82
-rw-r--r-- 1 a8hh8 abiuser 11676 Oct 10 05:51 Rollup89
-rw-r--r-- 1 a8hh8 abiuser 8156 Oct 10 05:51 Reformat69
-rw-r--r-- 1 a8hh8 abiuser 8795 Oct 10 05:51 Partition_by_Key_and_Sort96
-rw-r--r-- 1 a8hh8 abiuser 7506 Oct 10 05:51 Partition_by_Key_and_Sort86
-rw-r--r-- 1 a8hh8 abiuser 2185 Oct 10 05:51 Output_File77
-rw-r--r-- 1 a8hh8 abiuser 3547 Oct 10 05:51 Output_File73
-rw-r--r-- 1 a8hh8 abiuser 1958 Oct 10 05:51 Lookup_File79
-rw-r--r-- 1 a8hh8 abiuser 1574 Oct 10 05:51 Input_File75
-rw-r--r-- 1 a8hh8 abiuser 6448 Oct 10 05:51 Filter_by_Expression95
-rw-r--r-- 1 a8hh8 abiuser 7367 Oct 10 05:51 Filter_by_Expression94
-rw-r--r-- 1 a8hh8 abiuser 7795 Oct 10 05:51 Filter_by_Expression91
-rw-r--r-- 1 a8hh8 abiuser 11019 Oct 10 05:51 Rollup65
-rw-r--r-- 1 a8hh8 abiuser 2419 Oct 10 05:51 Replicate52
-rw-r--r-- 1 a8hh8 abiuser 7492 Oct 10 05:51 Partition_by_Key_and_Sort50
-rw-r--r-- 1 a8hh8 abiuser 7911 Oct 10 05:51 Partition_by_Key_and_Sort44
-rw-r--r-- 1 a8hh8 abiuser 7928 Oct 10 05:51 Partition_by_Key_and_Sort42
-rw-r--r-- 1 a8hh8 abiuser 31225 Oct 10 05:51 Join37
-rw-r--r-- 1 a8hh8 abiuser 1690 Oct 10 05:51 Input_File60
-rw-r--r-- 1 a8hh8 abiuser 2162 Oct 10 05:51 Input_File58
-rw-r--r-- 1 a8hh8 abiuser 2169 Oct 10 05:51 Input_File56
-rw-r--r-- 1 a8hh8 abiuser 1740 Oct 10 05:51 Input_File54
-rw-r--r-- 1 a8hh8 abiuser 8111 Oct 10 05:51 Input_File48
-rw-r--r-- 1 a8hh8 abiuser 2343 Oct 10 05:51 Input_File39
-rw-r--r-- 1 a8hh8 abiuser 2779 Oct 10 05:51 Input_File36
-rw-r--r-- 1 a8hh8 abiuser 4966 Oct 10 05:51 Input_File33
-rw-r--r-- 1 a8hh8 abiuser 6873 Oct 10 05:51 Filter by Expression - (Transform)45
-rw-r--r-- 1 a8hh8 abiuser 6854 Oct 10 05:51 Filter_by_Expression40
-rw-r--r-- 1 a8hh8 abiuser 8572 Oct 10 05:51 Dedup_Sorted34
-rw-r--r-- 1 a8hh8 abiuser 2387 Oct 10 05:51 Replicate51
-rw-r--r-- 1 a8hh8 abiuser 8564 Oct 10 05:51 Reformat66
-rw-r--r-- 1 a8hh8 abiuser 8742 Oct 10 05:51 Reformat64
-rw-r--r-- 1 a8hh8 abiuser 7819 Oct 10 05:51 Partition_by_Key_and_Sort49
-rw-r--r-- 1 a8hh8 abiuser 1968 Oct 10 05:51 Partition_by_Key_and_Sort46
-rw-r--r-- 1 a8hh8 abiuser 7928 Oct 10 05:51 Partition_by_Key_and_Sort43
-rw-r--r-- 1 a8hh8 abiuser 7930 Oct 10 05:51 Partition_by_Key_and_Sort41
-rw-r--r-- 1 a8hh8 abiuser 311267 Oct 10 05:51 Join63
-rw-r--r-- 1 a8hh8 abiuser 31102 Oct 10 05:51 Join47
-rw-r--r-- 1 a8hh8 abiuser 31110 Oct 10 05:51 Join35
-rw-r--r-- 1 a8hh8 abiuser 117657 Oct 10 05:51 Input_File62
-rw-r--r-- 1 a8hh8 abiuser 1767 Oct 10 05:51 Input_File61
-rw-r--r-- 1 a8hh8 abiuser 1728 Oct 10 05:51 Input_File59
-rw-r--r-- 1 a8hh8 abiuser 1820 Oct 10 05:51 Input_File57
-rw-r--r-- 1 a8hh8 abiuser 1767 Oct 10 05:51 Input_File55
-rw-r--r-- 1 a8hh8 abiuser 1689 Oct 10 05:51 Input_File53
-rw-r--r-- 1 a8hh8 abiuser 2291 Oct 10 05:51 Input_File38
-rw-r--r-- 1 a8hh8 abiuser 7312 Oct 10 05:51 Partition_by_Key_and_Sort4
-rw-r--r-- 1 a8hh8 abiuser 8589 Oct 10 05:51 Partition_by_Key_and_Sort31
-rw-r--r-- 1 a8hh8 abiuser 7663 Oct 10 05:51 Partition_by_Key_and_Sort30
-rw-r--r-- 1 a8hh8 abiuser 7643 Oct 10 05:51 Partition_by_Key_and_Sort15
-rw-r--r-- 1 a8hh8 abiuser 31214 Oct 10 05:51 Join26
-rw-r--r-- 1 a8hh8 abiuser 31076 Oct 10 05:51 Join10
-rw-r--r-- 1 a8hh8 abiuser 2726 Oct 10 05:51 Input_File9
-rw-r--r-- 1 a8hh8 abiuser 2232 Oct 10 05:51 Input_File28
-rw-r--r-- 1 a8hh8 abiuser 3930 Oct 10 05:51 Input_File25
-rw-r--r-- 1 a8hh8 abiuser 1853 Oct 10 05:51 Input_File2
-rw-r--r-- 1 a8hh8 abiuser 2288 Oct 10 05:51 Input_File12
-rw-r--r-- 1 a8hh8 abiuser 6583 Oct 10 05:51 Filter_by_Expression13
-rw-r--r-- 1 a8hh8 abiuser 8435 Oct 10 05:51 Dedup_Sorted7
-rw-r--r-- 1 a8hh8 abiuser 10466 Oct 10 05:51 Rollup19
-rw-r--r-- 1 a8hh8 abiuser 8279 Oct 10 05:51 reformat32
-rw-r--r-- 1 a8hh8 abiuser 10307 Oct 10 05:51 Partition_by_Key_and_Sort6
-rw-r--r-- 1 a8hh8 abiuser 7658 Oct 10 05:51 Partition_by_Key_and_Sort29
-rw-r--r-- 1 a8hh8 abiuser 8198 Oct 10 05:51 Partition_by_Key_and_Sort17
-rw-r--r-- 1 a8hh8 abiuser 7639 Oct 10 05:51 Partition_by_Key_and_Sort16
-rw-r--r-- 1 a8hh8 abiuser 7640 Oct 10 05:51 Partition_by_Key_and_Sort14
-rw-r--r-- 1 a8hh8 abiuser 2046 Oct 10 05:51 Output_File5
-rw-r--r-- 1 a8hh8 abiuser 31066 Oct 10 05:51 Join8
-rw-r--r-- 1 a8hh8 abiuser 31085 Oct 10 05:51 Join1
-rw-r--r-- 1 a8hh8 abiuser 1969 Oct 10 05:51 Input_File3
-rw-r--r-- 1 a8hh8 abiuser 2291 Oct 10 05:51 Input_File27
-rw-r--r-- 1 a8hh8 abiuser 2514 Oct 10 05:51 Input_File20
-rw-r--r-- 1 a8hh8 abiuser 1903 Oct 10 05:51 Input_File11
-rw-r--r-- 1 a8hh8 abiuser 6601 Oct 10 05:51 Filter_by_Expression24
-rw-r--r-- 1 a8hh8 abiuser 6305 Oct 10 05:51 Filter_by_Expression23
-rw-r--r-- 1 a8hh8 abiuser 6512 Oct 10 05:51 Filter_by_Expression22
-rw-r--r-- 1 a8hh8 abiuser 7059 Oct 10 05:51 Dedup_Sorted21
-rw-r--r-- 1 a8hh8 abiuser 7321 Oct 10 05:51 Dedup_Sorted18

---------- Post updated at 03:33 PM ---------- Previous update was at 03:28 PM ----------

Code:
 
    1958lookup79    2185lookup77    1690Input_File60    2162Input_File58    2169Input_File56   11665reformat74  465601Reformat67    2042lookup80    2256Input_File78    1767Input_File61    1728Input_File59    1820Input_File57    1767Input_File55    4015Gather76    6647Filter_by_Expression99    6647Filter_by_Expression98   81422Filter_by_Expression91   24302Filter_by_Expression72    6647Filter_by_Expression100   44291Dedup_Sorted97   55484Rollup19   28224Replicate52   34417reformat32    9280Partition_by_Key_and_Sort4   78487Join47   35143Join26    1740Input_File54    2343Input_File39    2291Input_File27    1853Input_File2    2288Input_File12    6601Filter_by_Expression24   22387Filter_by_Expression23   51851Join8   34003Join37   44646Join35   33801Join10   31085Join1    1689Input_File53    2291Input_File38    1903Input_File11

File names with size for the command with loop expression
only 41 files
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

splitting files based on text in the file

I need to split a file based on certain context inside the file. Is there a unix command that can do this? I have looked into split and csplit but it does not seem like those would work because I need to split this file based on certain text. The file has multiple records and I need to split this... (1 Reply)
Discussion started by: matrix1067
1 Replies

2. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

3. Shell Programming and Scripting

Data Splitting into two files from one file

I have a file as: I/P File: Ground Car 2009 Lib 2008 Lib 2003 Ground Car 2009 Ground Car 2003 Car 2005 Car 2003 Car 2005 Sita 2900 2006 Car 2007 I have to split the file into two: - one for names and second for years. O/p1 (Names): Ground Car (3 Replies)
Discussion started by: karumudi7
3 Replies

4. Shell Programming and Scripting

Splitting single file into n files

Hi all, I am new to scripting and I have a requirement we have source file as HEADER 01.10.2010 14:32:37 NAYA TA0022 TA0000 20000001;20060612;99991231;K4;02;3 20000008;20080624;99991231;K4;02;3 20000026;19840724;99991231;KK;01;3 20000027;19840724;99991231;KK;01;3... (6 Replies)
Discussion started by: srk409
6 Replies

5. Shell Programming and Scripting

Urgent ...pls Sorting files based on timestamp and picking the latest file

Hi Friends, Newbie to shell scripting. Currently i have used the below to sort data based on filenames and datestamp $ printf '%s\n' *.dat* | sort -t. -k3,4 filename_1.dat.20120430.Z filename_2.dat.20120430.Z filename_3.dat.20120430.Z filename_1.dat.20120501.Z filename_2.dat.20120501.Z... (1 Reply)
Discussion started by: robertbrown624
1 Replies

6. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

7. UNIX for Dummies Questions & Answers

Extracting data from one file, based on another file (splitting)

Dear All, I have two files but want to extract data from one based on another... can you please help me file 1 David Tom Ellen and file 2 David|0010|testnamez|resultsz David|0004|testnamex|resultsx Tom|0010|testnamez|resultsz Tom|0004|testnamex|resultsx Ellen|0010|testnamez|resultsz... (12 Replies)
Discussion started by: A-V
12 Replies

8. Shell Programming and Scripting

Splitting a single file to multiple files

Hi Friends , Please guide me with the code to extract multiple files from one file . The File Looks like ( Suppose a file has 2 tables list ,column length may vary ) H..- > File Header.... H....- >Table 1 Header.... D....- > Table 1 Data.... T....- >Table 1 Trailer.... H..-> Table 2... (1 Reply)
Discussion started by: AspiringD
1 Replies

9. Shell Programming and Scripting

Split a single file into multiple files based on a value.

Hi All, I have the sales_data.csv file in the directory as below. SDDCCR; SOM ; MD6546474777 ;05-JAN-16 ABC ; KIRAN ; CB789 ;04-JAN-16 ABC ; RAMANA; KS566767477747 ;06-JAN-16 ABC ; KAMESH; A33535335 ;04-JAN-16 SDDCCR; DINESH; GD6674474747 ;08-JAN-16... (4 Replies)
Discussion started by: ROCK_PLSQL
4 Replies

10. Shell Programming and Scripting

In PErl script: need to read the data one file and generate multiple files based on the data

We have the data looks like below in a log file. I want to generat files based on the string between two hash(#) symbol like below Source: #ext1#test1.tale2 drop #ext1#test11.tale21 drop #ext1#test123.tale21 drop #ext2#test1.tale21 drop #ext2#test12.tale21 drop #ext3#test11.tale21 drop... (5 Replies)
Discussion started by: Sanjeev G
5 Replies
All times are GMT -4. The time now is 08:17 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy