Sponsored Content
Top Forums UNIX for Beginners Questions & Answers How to generate a large file in Linux ? Post 303003005 by rbatte1 on Wednesday 6th of September 2017 12:46:52 PM
Old 09-06-2017
You don't state the OS version, so I don't know what tools you may already have to do this for you, and it depends what you want the file for.

There is a way in C that you open a new file, skipping to a huge offset then writing a single character before closing it. That might do the trick but it doesn't initially consume disk space and won't worry you for IO load. That may not be good for what you are trying to test. Additionally, if you restore it or copy it, the target file will be the full size, so that might mean that the filesystem does not restore properly if you suddenly fill it during a restore.

A good way can be to take a tar of a good chunk of disk (any files will do) then append it to itself several times, repeating until you get the desired size. Something like:-
Code:
cd /path/to/empty/space                                           # An empty directory to build the file
tar -cvf - /path/to/source/data | compress > my_base_file         # Get a chunk of something to start with.  Anything really, but probably best not devices, database files etc. that might be open

cat my_base_file my_base_file my_base_file > my_large_file        # Triple the file size
mv my_large_file  my_base_file                                    # Move it back to save space

Repeat the last two lines as needed to get a big enough file. If you need to trim it down again, you could perhaps:-
Code:
split -d -a 1 -b 15G my_large_file part. 2>/dev/null              # Break it into 15Gb chunks with a numeric suffix of length 1
mv part.0 my_large_file                                           # Save the one you actually want
rm part*                                                          # Clean up the rest of the junk

I've made it compress the file (you may prefer gzip) so that it is a true performance test, i.e. it can't be further compressed during transmission etc.


I hope that this helps,
Robin
This User Gave Thanks to rbatte1 For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Script to generate text file from excel file

Hello, I have a excel file which has almost ten columns on the shared drive. I have to write a shell script to ftp that daily to unix server and then extract some columns from there and generate oracle standard text file. The columns should be in proper order and aligned properly, otherwise... (1 Reply)
Discussion started by: isingh786
1 Replies

2. Shell Programming and Scripting

need help in Parsing a CSV file and generate a new output file

Hi Scripting Gurus, I am trying to parse a csv file and generate a new output file. The input file will be a variable length in turns of rows and columns. output file will have 8 columns. we have three columns from the header for each set. just to give little bit more clarification each row... (15 Replies)
Discussion started by: vkr
15 Replies

3. Red Hat

not large enough to display the application in red hat linux x win desktop

I use red hat linux es 5 I use startx to start the x-win desktop. But when I use vritual manager . The display application is too large so the bottom part for the application cannot show out. I cannot scroll down to get the display of bottm part . So, I do not know what button display at the... (0 Replies)
Discussion started by: chuikingman
0 Replies

4. Shell Programming and Scripting

KSH - help needed for creating a script to generate xml file from text file

Dear Members, I have a table in Oracle DB and one of its column name is INFO which has data in text format which we need to fetch in a script and create an xml file of a new table from the input. The contents of a single cell of INFO column is like: Area:app - aam Clean Up Criteria:... (0 Replies)
Discussion started by: Yoodit
0 Replies

5. Programming

Make cannot generate .ko linux kernel module

cannot generate .ko file on my linux, although it can generate module.symvers. But when I copy .c file and Makefile to another linux computer, there's no problem. The strange thing is: make is successfuly executed, and returned 0; make output: make -C /lib/modules/2.6.18-92.el5xen/build ... (4 Replies)
Discussion started by: vistastar
4 Replies

6. Shell Programming and Scripting

Generate and copy files in UNICODE format from Linux to Windows

Hi,We have an interface which extracts data from database tables, spool the records into text files, and copy those files using SFTP to a windows server location. The target system loads these files into its database using some DTS packages in SQL Server. This interface of exporting text files... (13 Replies)
Discussion started by: urjagadeesh
13 Replies

7. Shell Programming and Scripting

Linux - Script to generate the output delimited by Comma/Pipe

Hi All, I have a requirement where I need to go to a directory, list all the files that start with person* (for eg) & read the most recent file from the list of files. While browsing through the forum, i found that the command ls -t will list the files. I am trying to generate the output... (1 Reply)
Discussion started by: dsfreddie
1 Replies

8. UNIX for Dummies Questions & Answers

LINUX how to generate multiple line output file

Hi All, I am trying to generate a file through a LINUX scripting as shown below export field1=`cat dir/filename1.txt |wc -l` echo "field1 : $field1 >>dir/WeeklySummaryReport.txt export field2=`cat dir/filename2.txt |wc -l` echo "field2 : $field2 >>dir/WeeklySummaryReport.txt While... (5 Replies)
Discussion started by: dsfreddie
5 Replies

9. UNIX for Dummies Questions & Answers

How to generate html reports through LINUX Scripting?

Hi All, I am trying to generate a weekly HTML report using LINUX Scripting. This will have record counts of some files. (like below) touch path/filename.html echo "Weekly Summary Report for Business Date : $P_BUS_DT">path/filename.html export A1=`cat path/filename1.txt |wc -l` echo "A1... (6 Replies)
Discussion started by: dsfreddie
6 Replies

10. Shell Programming and Scripting

Needed script to FTP a File and generate a quality checksum file

hi all i want a script to FTP a file and should generate a quality checksum file means when I FTP a file from one server to another server it should generate a QC file which should contain timestamp,no.of records in that file Thanks in advance saikumar (3 Replies)
Discussion started by: hemanthsaikumar
3 Replies
CREATE_MODULE(2)					     Linux Programmer's Manual						  CREATE_MODULE(2)

NAME
create_module - create a loadable module entry SYNOPSIS
#include <linux/module.h> caddr_t create_module(const char *name, size_t size); Note: No declaration of this system call is provided in glibc headers; see NOTES. DESCRIPTION
Note: This system call is present only in kernels before Linux 2.6. create_module() attempts to create a loadable module entry and reserve the kernel memory that will be needed to hold the module. This sys- tem call requires privilege. RETURN VALUE
On success, returns the kernel address at which the module will reside. On error, -1 is returned and errno is set appropriately. ERRORS
EEXIST A module by that name already exists. EFAULT name is outside the program's accessible address space. EINVAL The requested size is too small even for the module header information. ENOMEM The kernel could not allocate a contiguous block of memory large enough for the module. ENOSYS create_module() is not supported in this version of the kernel (e.g., the kernel is version 2.6 or later). EPERM The caller was not privileged (did not have the CAP_SYS_MODULE capability). VERSIONS
This system call is present on Linux only up until kernel 2.4; it was removed in Linux 2.6. CONFORMING TO
create_module() is Linux-specific. NOTES
This obsolete system call is not supported by glibc. No declaration is provided in glibc headers, but, through a quirk of history, glibc versions before 2.23 did export an ABI for this system call. Therefore, in order to employ this system call, it was sufficient to manually declare the interface in your code; alternatively, you could invoke the system call using syscall(2). SEE ALSO
delete_module(2), init_module(2), query_module(2) COLOPHON
This page is part of release 4.15 of the Linux man-pages project. A description of the project, information about reporting bugs, and the latest version of this page, can be found at https://www.kernel.org/doc/man-pages/. Linux 2017-09-15 CREATE_MODULE(2)
All times are GMT -4. The time now is 12:25 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy