Script fails when generated output file reaches a particular size


 
Thread Tools Search this Thread
Operating Systems Solaris Script fails when generated output file reaches a particular size
# 1  
Old 07-01-2011
Script fails when generated output file reaches a particular size

Hi All,

New to unix. Here is the problem. Running a script that extracts data from hyperion essbase and generates a file in unix. This script fails most of the times with a very low success rate. The data has increased a lot in the last few months resulting in the file being more than 2 gb.

Some of the findings,

1. the last three runs, script failed when file size reaches 2147483647. Now when i a did a getconf to see if its a 32 bit os since my friend suggested that 32 bit systems have trouble handling files greater than 2gb(did an isainfo mentioned below for this).

2. In the result of getconf, found that SEM_VALUE_MAX and SEM_NSEMS_MAX are set to 2147483647 (file size mentioned above...Not sure how this is related).

3. result of isainfo -v is the below,

64-bit sparcv9 applications
ima fmaf vis2 vis popc

32-bit sparc applications
ima fmaf vis2 vis popc v8plus div32 mul32

Any help regarding this is much appreciated, because i might get fired soon Smilie

Thanks,
N shaw
# 2  
Old 07-02-2011
It's impossible to help without you telling us EXACTLY what's failing.
# 3  
Old 07-02-2011
SEM_VALUE_MAX probably has nothing to do with it since this constant relates to semaphores.

My gut feeling is that the programme that is creating the output file, being called by the script, wasn't compiled with the necessary large file support. If the programme writing the file is one you've created then rebuilding it with large file support enabled is all that should be necessary. For a programme written in C, you may need to add one or more of these options from the command line:
Code:
-D_USE_LARGEFILES
-D_FILE_OFFSET_BITS=64 
-D_LARGEFILE64_SOURCE=1

If the failing programme isn't yours, then you'll need to hunt for a version that is large file aware.

It's also possible that the filesystem being written to was mounted with a 'nolargefiles' option which limits a file to 2GiB. Can you create a 3GiB file manually on the filesystem? If so, then it's the programme and not the filesystem.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to call Oracle archive backup script when file system reaches threshold value

Hello All, I need immediate help in creating shell script to call archivebkup.ksh script when archive file system capacity reaches threshold value or 60% Need to identify the unique file system that reaches threshold value. ex: capacity ... (4 Replies)
Discussion started by: sasikanthdba
4 Replies

2. Shell Programming and Scripting

Generated csv files size are more than 3GB

Hi i have a shell script which will generate the CSV files from a group of XML files. but the XML files are in 50K plus. so the generated CSV files are more then 4 GB in size. is there any way i can take care in the scripting itself to make the shell script efficent to write data into CSV... (1 Reply)
Discussion started by: sunny13392
1 Replies

3. UNIX for Advanced & Expert Users

Script to rename file that was generated today and which starts with date

hello, can someone please suggest a script to rename a file that was generated today and filename that being generated daily starts with date, its a xml file. here is example. # find . -type f -mtime -1 ./20130529_4995733057260357019.xml # this finename should be renamed to this format.... (6 Replies)
Discussion started by: bobby320
6 Replies

4. Shell Programming and Scripting

Output after a perl script gives a file with size zero.

Hi, I have a unix shell script which generates a flat file after connecting to Teradata servers to fetch tables and views and also picks up modified unix scripts from the specified paths. Later on the script calls a perl script to assign a value based on the type of object in the flat file which... (2 Replies)
Discussion started by: yohasini
2 Replies

5. Shell Programming and Scripting

Script for deleting files and directories when the file system reaches the threshold

Hi Can someone assist in writing a script. I have a filesystem named /sybase in my aix lpar. When this filesystem becomes 94% full all the files and directories under /sybase/logs should be deleted immediately. :confused: (7 Replies)
Discussion started by: newtoaixos
7 Replies

6. Shell Programming and Scripting

Script to read file size and send email only if size > 0.

Hi Experts, I have a script like $ORACLE_HOME/bin/sqlplus username/password # << ENDSQL set pagesize 0 trim on feedback off verify off echo off newp none timing off set serveroutput on set heading off spool Schemaerrtmp.txt select ' TIMESTAMP COMPUTER NAME ... (5 Replies)
Discussion started by: welldone
5 Replies

7. Shell Programming and Scripting

Script to Grep column 3 from csv file generated yesterday

Hello, Can any one please assist how to scirpt it: Every day a new log file is create and I want to process only the one generated yesterday and get the data of column 3 and 6. For example today's date is 24 then I want to get the data of log file created on 23rd. Log Files in... (7 Replies)
Discussion started by: sureshcisco
7 Replies

8. Shell Programming and Scripting

shPID file generated from shell script

I have a ksh script that runs a sqlplus script. Every time I run the ksh script it generates a log file like it should and a file called shPID (sh47398.1) with the text of the sql script. Why is it doing this? It is the only ksh script that I have that does this. thanks. (0 Replies)
Discussion started by: djehresmann
0 Replies

9. Shell Programming and Scripting

compare file size from a output file from a script

Hi guys, firstly I'm working on SunOS 5.10 Generic_125100-10 sun4u sparc SUNW,Sun-Fire-V240 I've made a script to compress two directory and then send them to an other server via ftp. This is working very well. Inside theis script I decide to log usefull data for troubleshooting in case of... (7 Replies)
Discussion started by: moustik
7 Replies

10. Shell Programming and Scripting

Need a script to Append date to generated .txt file

Hi, Can anyone plz share their experience with - Building shell script to append the file with date in following format- Filename_MMDDYYYY.txt Thanks in advance (2 Replies)
Discussion started by: prince_of_focus
2 Replies
Login or Register to Ask a Question