Hello gurus,
I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files.
e.g. my data is like:
Row_Num,... (6 Replies)
I need to write a shell script for below scenario
My input file has data in format:
qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26
qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28
qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43
qwerty0101CFG 12345... (19 Replies)
Hi Experts,
I have to split huge file based on the pattern to create smaller files. The pattern which is expected in the file is:
Master.....
First...
second....
second...
third..
third...
Master...
First..
second...
third...
Master...
First...
second..
second..
second..... (2 Replies)
I am trying to update an older program on a small cluster. It uses individual files to send jobs to each node. However the newer database comes as one large file, containing over 10,000 records. I therefore need to split this file. It looks like this:
HMMER3/b
NAME 1-cysPrx_C
ACC ... (2 Replies)
HI All,
I have to split a xml file into multiple xml files and append it in another .xml file. for example below is a sample xml and using shell script i have to split it into three xml files and append all the three xmls in a .xml file. Can some one help plz.
eg:
<?xml version="1.0"?>... (4 Replies)
I will simplify the explaination a bit, I need to parse through a 87m file -
I have a single text file in the form of :
<NAME>house........
SOMETEXT
SOMETEXT
SOMETEXT
.
.
.
.
</script>
MORETEXT
MORETEXT
.
.
. (6 Replies)
Hello All ,
Please help me with below requirement
I want to split a xml file based on tag.here is the file format
<data-set>
some-information
</data-set>
<data-set1>
some-information
</data-set1>
<data-set2>
some-information
</data-set2>
I want to split the above file into 3... (5 Replies)
Hi Everyone,
I'm new here and I was checking this old post:
/shell-programming-and-scripting/180669-splitting-file-into-several-smaller-files-using-perl.html
(cannot paste link because of lack of points)
I need to do something like this but understand very little of perl.
I also check... (4 Replies)
Hi,
I'm having a xml file with multiple xml header. so i want to split the file into multiple files.
Sample.xml consists multiple headers so how can we split these multiple headers into multiple files in unix.
eg :
<?xml version="1.0" encoding="UTF-8"?>
<ml:individual... (3 Replies)
extendedFILE(5) Standards, Environments, and Macros extendedFILE(5)NAME
extendedFILE - enable extended FILE facility usage
SYNOPSIS
$ ulimit -n N_file_descriptors
$ LD_PRELOAD_32=/usr/lib/extendedFILE.so.1 application [arg...]
DESCRIPTION
The extendedFILE.so.1 is not a library but an enabler of the extended FILE facility.
The extended FILE facility allows 32-bit processes to use any valid file descriptor with the standard I/O (see stdio(3C)) C library func-
tions. Historically, 32-bit applications have been limited to using the first 256 numerical file descriptors for use with standard I/O
streams. By using the extended FILE facility this limitation is lifted. Any valid file descriptor can be used with standard I/O. See the
NOTES section of enable_extended_FILE_stdio(3C).
The extended FILE facility is enabled from the shell level before an application is launched. The file descriptor limit must also be
raised. The syntax for raising the file descriptor limit is
$ ulimit -n max_file_descriptors
$ LD_PRELOAD_32=/usr/lib/extendedFILE.so.1 application [arg...]
where max_file_descriptors is the maximum number of file descriptors desired. See limit(1). The maximum value is the same as the maximum
value for open(2).
ENVIRONMENT VARIABLES
The following environment variables control the behavior of the extended FILE facility.
_STDIO_BADFD This variable takes an integer representing the lowest file descriptor, which will be made unallocatable. This
action provides a protection mechanism so that applications that abuse interfaces do not experience silent data cor-
ruption. The value must be between 3 and 255 inclusive.
_STDIO_BADFD_SIGNAL This variable takes an integer or string representing any valid signal. See signal.h(3HEAD) for valid values or
strings. This environment variable causes the specified signal to be sent to the application if certain exceptional
cases are detected during the use of this facility. The default signal is SIGABRT.
EXAMPLES
Example 1 Limit the number of file descriptors and FILE standard I/O structures.
The following example limits the number of file descriptors and FILE standard I/O structures to 1000.
$ ulimit -n 1000
$ LD_PRELOAD_32=/usr/lib/extendedFILE.so.1 application [arg...]
Example 2 Enable the extended FILE facility.
The following example enables the extended FILE facility. See enable_extended_FILE_stdio(3C) for more examples.
$ ulimit -n 1000
$ _STDIO_BADFD=100 _STDIO_BADFD_SIGNAL=SIGABRT
LD_PRELOAD_32=/usr/lib/extendedFILE.so.1
application [arg ...]
Example 3 Set up the extended FILE environment and start the application.
The following shell script first sets up the proper extended FILE environment and then starts the application:
#!/bin/sh
if [ $# = 0 ]; then
echo "usage: $0 application [arguments...]"
exit 1
fi
ulimit -n 1000
# _STDIO_BADFD=196; export _STDIO_BADFD
# _STDIO_BADFD_SIGNAL=SIGABRT; export _STDIO_BADFD_SIGNAL
LD_PRELOAD_32=/usr/lib/extendedFILE.so.1; export LD_PRELOAD_32
"$@"
FILES
/usr/lib/extendedFILE.so.1 enabling library
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWcsl (32-bit) |
+-----------------------------+-----------------------------+
|Interface Stability |Stable |
+-----------------------------+-----------------------------+
|MT-Level |Safe |
+-----------------------------+-----------------------------+
SEE ALSO limit(1), open(2), enable_extended_FILE_stdio(3C), fdopen(3C), fopen(3C), popen(3C), signal.h(3HEAD), stdio(3C), attributes(5)WARNINGS
The following displayed message
Application violated extended FILE safety mechanism.
Please read the man page for extendedFILE.
Aborting
is an indication that your application is modifying the internal file descriptor field of the FILE structure from standard I/O. Continued
use of this extended FILE facility could harm your data. Do not use the extended FILE facility with your application.
SunOS 5.11 18 Apr 2006 extendedFILE(5)