Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Split large file into 24 small files on one hour basis Post 303036518 by Raghuram717 on Monday 1st of July 2019 02:25:44 AM
Old 07-01-2019
i Wrote some script in shell that was working for me
posting here for reference


timestamp=$1

Code:
z=1
 max=24
 for (( i=1; i <= $max; ++i ))
 do
 echo "starting date is $timestamp"
 get_starting_line_number=$(grep -nr "$timestamp" summaryReport_LoadTest.csv|head -1|cut -d: -f1)
 echo "starting_line_number is $get_starting_line_number"
 ntimestamp=$(date -d "$timestamp 1 hour" +"%Y-%m-%d %H:%M:%S")
 echo "ending date is $ntimestamp"
 get_ending_line_number=$(grep -nr "$ntimestamp" summaryReport_LoadTest.csv|tail -1|cut -d: -f1)
 echo "ending line number is $get_ending_line_number"
 awk 'NR >= '$get_starting_line_number' && NR <= '$get_ending_line_number'' summaryReport_LoadTest.csv > summaryReport_LoadTest_"$z".csv
 z=$(expr $z + 1)
 echo $z
 b=$(expr $get_ending_line_number - 1)
 sed -i ''$get_starting_line_number','$b'd'  summaryReport_LoadTest.csv
 timestamp=$ntimestamp
 echo "**********************************************************************************************"
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting large file into small files

Hi, I need to split a large file into small files based on a string. At different palces in the large I have the string ^Job. I need to split the file into different files starting from ^Job to the last character before the next ^Job. Also all the small files should be automatically named.... (4 Replies)
Discussion started by: dncs
4 Replies

2. Shell Programming and Scripting

Split a file into 16 small files

Hi I want to split a file that has 'n' number of records into 16 small files. Can some one suggest me how to do this using Unix script? Thanks rrkk (10 Replies)
Discussion started by: rrkks
10 Replies

3. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies

4. Shell Programming and Scripting

script to splite large file to number of small files

Dear All, Could you please help me to split a file contain around 240,000,000 line to 4 files all equally likely , note that we need to maintain that the end of each file should started by start flage (MSISDN) and ended by end flag (End), also the number of the line between the... (10 Replies)
Discussion started by: ahmed.gad
10 Replies

5. Shell Programming and Scripting

Split large zone file dump into multiple files

I have a large zone file dump that consists of ; DNS record for the adomain.com domain data1 data2 data3 data4 data5 CRLF CRLF CRLF ; DNS record for the anotherdomain.com domain data1 data2 data3 data4 data5 data6 CRLF (7 Replies)
Discussion started by: Bluemerlin
7 Replies

6. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies

7. Shell Programming and Scripting

How to split this txt file into small files?

Dear shell experts, I would like to spilt a txt file into small ones. However, I did not know how to program use shell. If someone could help, it is greatly appreciated! Specifically, I supposed there is file named A.txt. The content of the file likes this: Subject run condtion ACC time... (3 Replies)
Discussion started by: psychmyluo
3 Replies

8. Shell Programming and Scripting

Breaking large file into small files

Dear all, I have huge txt file with the input files for some setup_code. However for running my setup_code, I require txt files with maximum of 1000 input files Please help me in suggesting way to break down this big txt file to small txt file of 1000 entries only. thanks and Greetings, Emily (12 Replies)
Discussion started by: emily
12 Replies

9. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies

10. Shell Programming and Scripting

Split large xml into mutiple files and with header and footer in file

Split large xml into mutiple files and with header and footer in file tried below it splits unevenly and also i need help in adding header and footer command : csplit -s -k -f my_XML_split.xml extrfile.xml "/<Document>/" {1} sample xml <?xml version="1.0" encoding="UTF-8"?><Recipient>... (36 Replies)
Discussion started by: karthik
36 Replies
hexec(1)																  hexec(1)

NAME
hexec - a process execution hooking tool SYNOPSIS
hexec <OPTIONS> [expr] [cmd] [args...] DESCRIPTION
hexec is a tool to hook into process exececution calls (exec family of syscalls). You can define an expression that is executed against any hooked exec call. This expression may also contain a replacement exec call. OPTIONS SUMMARY
Here is a summary of the options to hexec. --help | -h Print a options/expr summary page --version | -v Print hexec version --log-out | -lo set output file for error and -print output OPTIONS
-h Print a options/expr summary page --help Print a options/expr summary page -lo Set the output file for error and -print output. This can be required if writing to stderr could cause malfunction because some processes read from stderr and expect a well defined output. EXPRESSIONS
The expression is executed against all process execution calls. If the expression returns true, the original call is skipped. <expr> -and <expr> <expr> -a <expr> <expr> <expr> Returns true if both expressions return true. If the left returns false, the right expression is never executed. <expr> -or <expr> <expr> -o <expr> Returns true if one of both expressions returns true. If the left returns true, the right expression is never executed. -path <pattern> Returns true if the path of the executable matches <pattern>. <pattern> is a bash compatible wild card pattern. -ipath <pattern> Same as -path, but case insensitive. -name <pattern> Returns true if the base name of the executable matches <pattern>. <pattern> is a bash compatible wild card pattern. -iname <pattern> Same as -name, but case insensitive. -contains <str> Returns true if the path of the executable containes the string <str>. -icontains <str> Same as -contains, but case insensitive. -print Print all arguments to the called process. Returns always true. -exec <cmd> [args...] ; Executes <cmd> with [args...] as arguments. This expression must be terminated with a semicolon. You can use argument placeholders in <cmd> and [args...] (see below). This expression always returns true. NOTE: Please be aware that you may need to escape or quote the terminating semicolon to not confuse your shell. -sh <script> Interprets <script> as a shell script by invoking /bin/sh with the arguments -c '<script>'. You can use argument placeholders inside <script>. Please note that -sh only expects a single argument and not a variable list of arguments (as -exec does). Using -sh is the same as using -exec sh -c <script>. PLACEHOLDERS FOR -exec EXPRESSION Every -exec expression can use placeholders in the argument list to obtain information from the original exec call. Each placeholder starts with { and ends with }. Use { if you want to use a { in your argument list. The placeholders are replaced when the -exec expression is evaluated. Placeholder types: {n} Will be replaced with the number of arguments in the original call. {<idx>} Will be replaced with the original argument at index <idx>. Example: {1} would give the first argument. You can prepend <idx> with placeholder flags. {} Will be replaced with all arguments from the original call. Each argument is seperated with a space. You can use placeholder flags. NOTE: The executable name is also considered as argument. This means that a call like "echo test" will result in two arguments, "echo" and "test". PLACEHOLDER FLAGS
s Every argument is inserted as single argument instead of concatenating all arguments. Consider the arguments 'a', 'b' and 'c'. Without s, -exec {} ; would result in -exec 'a b c' ;, which in many cases is not what you want. -exec {s} ; howewer would result in -exec 'a' 'b' 'c' ;. q Quote every single argument. e Escape all non alpha-numeric characters. This flag is very useful when using a "sh -c <...>" in the -exec expression. Examples -exec echo {} ; with the call 'a' 'b' 'c' results in: 'echo' 'a b c' -exec echo {q} ; with the call 'a' 'b' 'c' results in: 'echo' 'a' 'b' 'c' -exec sh -c 'echo {}; {}' ; with the call 'sh' '-c' 'gcc d.c >> log.txt' results in: 'sh' '-c' 'echo sh -c gcc d.c >> log.txt; sh -c gcc d.c >> log.txt' Please note that this will not do what you may expect, as the >> is handled wrong in this case. -exec sh -c 'echo {e}; {}' ; with the call 'sh' '-c' 'gcc d.c >> log.txt' results in: 'sh' '-c' 'echo sh -c gcc d.c >> log.txt; sh -c gcc d.c >> log.txt' CHANGES TO PROCESSES
hexec will add some environment variables to the hooked processes. These are (may not be complete): LD_PRELOAD hexec adds libhexec-hook.so to the list of preloaded libraries. HEXEC_EXPR_SHM Contains the name of the internal shared memory object. HEXEC_LOG_FD Contains the file descriptor for error and -print output. Please do never modify these environment variables. Also take care when you use these variables, because the name and content of the vari- ables may change in the future. EXAMPLES
hexec -name 'gcc' -exec ccache {s} ; make Calls make, which will then call gcc several times. The executable name of each hooked process execution is tested against the file pattern "gcc" and "ccache {s}" is called each time a match is found. '{s}' will be replaced with the original (the hooked) call. In this example, a call to "gcc -o test.o test.c" would be replaced with "ccache gcc -o test.o test.c" HOW IT WORKS
TODO BUGS
I'm sure there are alot...it's still beta :) AUTHOR
hexec was written by Alexander Block http://blocksoftware.net/ If you wish to report a problem or make a suggestion then please email ablock@blocksoftware.net hexec is released under the GNU General Public License version 2 or later. Please see the file COPYING for license details. November 2008 hexec(1)
All times are GMT -4. The time now is 05:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy