Sponsored Content
Top Forums Shell Programming and Scripting Splitting large file into multiple files in unix based on pattern Post 302536625 by itkamaraj on Wednesday 6th of July 2011 02:49:41 AM
Old 07-06-2011
which will read input file i.e test from particular location

will the filename change everyday ?
how the filename looks like ?

Code:
 
filename="/tmp/test"
lineno=`wc -l < $filename`
nawk -v lineno="$lineno" '{ if (NR>1 && NR < lineno){f=substr($1,length($1)-2)".txt";print >> f;close(f)}}' $filename

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk - splitting 1 large file into multiple based on same key records

Hello gurus, I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files. e.g. my data is like: Row_Num,... (6 Replies)
Discussion started by: kam66
6 Replies

2. Shell Programming and Scripting

Problem with splitting large file based on pattern

Hi Experts, I have to split huge file based on the pattern to create smaller files. The pattern which is expected in the file is: Master..... First... second.... second... third.. third... Master... First.. second... third... Master... First... second.. second.. second..... (2 Replies)
Discussion started by: saisanthi
2 Replies

3. Shell Programming and Scripting

Splitting large file and renaming based on field

I am trying to update an older program on a small cluster. It uses individual files to send jobs to each node. However the newer database comes as one large file, containing over 10,000 records. I therefore need to split this file. It looks like this: HMMER3/b NAME 1-cysPrx_C ACC ... (2 Replies)
Discussion started by: fozrun
2 Replies

4. Shell Programming and Scripting

Help me pls : splitting single file in unix into different files based on data

I have a file in unix with sample data as follows : -------------------------------------------------------------- -------------------------------------------------------------- {30001002|XXparameter|Layout|$ I want this file to be splitted into different files and corresponding to the sample... (54 Replies)
Discussion started by: Ravindra Swan
54 Replies

5. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

6. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

7. Shell Programming and Scripting

Splitting textfile based on pattern and name new file after pattern

Hi there, I am pretty new to those things, so I couldn't figure out how to solve this, and if it is actually that easy. just found that awk could help:(. so i have a textfile with strings and numbers (originally copy pasted from word, therefore some empty cells) in the following structure: SC... (9 Replies)
Discussion started by: luja
9 Replies

8. Shell Programming and Scripting

Help with Splitting a Large XML file based on size AND tags

Hi All, This is my first post here. Hoping to share and gain knowledge from this great forum !!!! I've scanned this forum before posting my problem here, but I'm afraid I couldn't find any thread that addresses this exact problem. I'm trying to split a large XML file (with multiple tag... (7 Replies)
Discussion started by: Aviktheory11
7 Replies

9. UNIX for Advanced & Expert Users

Concatenation of multiple files based on file pattern

Hi, I have the following reports that get generated every 1 hour and this is my requirement: 1. 5 reports get generated every hour with the names "Report.Dddmmyy.Thhmiss.CTLR" "Report.Dddmmyy.Thhmiss.ACCD" "Report.Dddmmyy.Thhmiss.BCCD" "Report.Dddmmyy.Thhmiss.CCCD"... (1 Reply)
Discussion started by: Jesshelle David
1 Replies

10. UNIX for Beginners Questions & Answers

UNIX script to append multiple text files into one file based on pattern present in filaname

Hi All-I am new to Unix , I need to write a script. Can someone help me with a requirement where I have list of files in a directory, I want to Merge the files if a pattern of string matches in filenames? AAAL_555A_ORANGE1_F190404.TXT AAAL_555A_ORANGE2_F190404.TXT AAAL_555A_ORANGE3_F190404.TXT... (6 Replies)
Discussion started by: Shankar455
6 Replies
test(1F)							   FMLI Commands							  test(1F)

NAME
test - condition evaluation command SYNOPSIS
test expression expression DESCRIPTION
test evaluates the expression expression and if its value is true, sets a 0 (TRUE) exit status; otherwise, a non-zero (FALSE) exit status is set; test also sets a non-zero exit status if there are no arguments. When permissions are tested, the effective user ID of the process is used. All operators, flags, and brackets (brackets used as shown in the second SYNOPSIS line) must be separate arguments to test. Normally these items are separated by spaces. USAGE
Primitives The following primitives are used to construct expression: -r filename True if filename exists and is readable. -w filename True if filename exists and is writable. -x filename True if filename exists and is executable. -f filename True if filename exists and is a regular file. -d filename True if filename exists and is a directory. -c filename True if filename exists and is a character special file. -b filename True if filename exists and is a block special file. -p filename True if filename exists and is a named pipe (FIFO). -u filename True if filename exists and its set-user-ID bit is set. -g filename True if filename exists and its set-group-ID bit is set. -k filename True if filename exists and its sticky bit is set. -s filename True if filename exists and has a size greater than 0. -t[fildes] True if the open file whose file descriptor number is fildes (1 by default) is associated with a terminal device. -z s1 True if the length of string s1 is 0. -n s1 True if the length of the string s1 is non-zero. s1 = s2 True if strings s1 and s2 are identical. s1 != s2 True if strings s1 and s2 are not identical. s1 True if s1 is not the null string. n1 -eq n2 True if the integers n1 and n2 are algebraically equal. Any of the comparisons -ne, -gt, -ge, -lt, and -le may be used in place of -eq. Operators These primaries may be combined with the following operators: ! Unary negation operator. -a Binary and operator. -o Binary or operator (-a has higher precedence than -o). `(expression)` Parentheses for grouping. Notice also that parentheses are meaningful to the shell and, therefore, must be quoted. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
find(1), sh(1), attributes(5) NOTES
If you test a file you own (the -r , -w , or -x tests), but the permission tested does not have the owner bit set, a non-zero (false) exit status will be returned even though the file may have the group or other bit set for that permission. The correct exit status will be set if you are super-user. The = and != operators have a higher precedence than the -r through -n operators, and = and != always expect arguments; therefore, = and != cannot be used with the -r through -n operators. If more than one argument follows the -r through -n operators, only the first argument is examined; the others are ignored, unless a -a or a -o is the second argument. SunOS 5.10 5 Jul 1990 test(1F)
All times are GMT -4. The time now is 07:19 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy