Sponsored Content
Top Forums UNIX for Beginners Questions & Answers How to generate a large file in Linux ? Post 303003005 by rbatte1 on Wednesday 6th of September 2017 12:46:52 PM
Old 09-06-2017
You don't state the OS version, so I don't know what tools you may already have to do this for you, and it depends what you want the file for.

There is a way in C that you open a new file, skipping to a huge offset then writing a single character before closing it. That might do the trick but it doesn't initially consume disk space and won't worry you for IO load. That may not be good for what you are trying to test. Additionally, if you restore it or copy it, the target file will be the full size, so that might mean that the filesystem does not restore properly if you suddenly fill it during a restore.

A good way can be to take a tar of a good chunk of disk (any files will do) then append it to itself several times, repeating until you get the desired size. Something like:-
Code:
cd /path/to/empty/space                                           # An empty directory to build the file
tar -cvf - /path/to/source/data | compress > my_base_file         # Get a chunk of something to start with.  Anything really, but probably best not devices, database files etc. that might be open

cat my_base_file my_base_file my_base_file > my_large_file        # Triple the file size
mv my_large_file  my_base_file                                    # Move it back to save space

Repeat the last two lines as needed to get a big enough file. If you need to trim it down again, you could perhaps:-
Code:
split -d -a 1 -b 15G my_large_file part. 2>/dev/null              # Break it into 15Gb chunks with a numeric suffix of length 1
mv part.0 my_large_file                                           # Save the one you actually want
rm part*                                                          # Clean up the rest of the junk

I've made it compress the file (you may prefer gzip) so that it is a true performance test, i.e. it can't be further compressed during transmission etc.


I hope that this helps,
Robin
This User Gave Thanks to rbatte1 For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Script to generate text file from excel file

Hello, I have a excel file which has almost ten columns on the shared drive. I have to write a shell script to ftp that daily to unix server and then extract some columns from there and generate oracle standard text file. The columns should be in proper order and aligned properly, otherwise... (1 Reply)
Discussion started by: isingh786
1 Replies

2. Shell Programming and Scripting

need help in Parsing a CSV file and generate a new output file

Hi Scripting Gurus, I am trying to parse a csv file and generate a new output file. The input file will be a variable length in turns of rows and columns. output file will have 8 columns. we have three columns from the header for each set. just to give little bit more clarification each row... (15 Replies)
Discussion started by: vkr
15 Replies

3. Red Hat

not large enough to display the application in red hat linux x win desktop

I use red hat linux es 5 I use startx to start the x-win desktop. But when I use vritual manager . The display application is too large so the bottom part for the application cannot show out. I cannot scroll down to get the display of bottm part . So, I do not know what button display at the... (0 Replies)
Discussion started by: chuikingman
0 Replies

4. Shell Programming and Scripting

KSH - help needed for creating a script to generate xml file from text file

Dear Members, I have a table in Oracle DB and one of its column name is INFO which has data in text format which we need to fetch in a script and create an xml file of a new table from the input. The contents of a single cell of INFO column is like: Area:app - aam Clean Up Criteria:... (0 Replies)
Discussion started by: Yoodit
0 Replies

5. Programming

Make cannot generate .ko linux kernel module

cannot generate .ko file on my linux, although it can generate module.symvers. But when I copy .c file and Makefile to another linux computer, there's no problem. The strange thing is: make is successfuly executed, and returned 0; make output: make -C /lib/modules/2.6.18-92.el5xen/build ... (4 Replies)
Discussion started by: vistastar
4 Replies

6. Shell Programming and Scripting

Generate and copy files in UNICODE format from Linux to Windows

Hi,We have an interface which extracts data from database tables, spool the records into text files, and copy those files using SFTP to a windows server location. The target system loads these files into its database using some DTS packages in SQL Server. This interface of exporting text files... (13 Replies)
Discussion started by: urjagadeesh
13 Replies

7. Shell Programming and Scripting

Linux - Script to generate the output delimited by Comma/Pipe

Hi All, I have a requirement where I need to go to a directory, list all the files that start with person* (for eg) & read the most recent file from the list of files. While browsing through the forum, i found that the command ls -t will list the files. I am trying to generate the output... (1 Reply)
Discussion started by: dsfreddie
1 Replies

8. UNIX for Dummies Questions & Answers

LINUX how to generate multiple line output file

Hi All, I am trying to generate a file through a LINUX scripting as shown below export field1=`cat dir/filename1.txt |wc -l` echo "field1 : $field1 >>dir/WeeklySummaryReport.txt export field2=`cat dir/filename2.txt |wc -l` echo "field2 : $field2 >>dir/WeeklySummaryReport.txt While... (5 Replies)
Discussion started by: dsfreddie
5 Replies

9. UNIX for Dummies Questions & Answers

How to generate html reports through LINUX Scripting?

Hi All, I am trying to generate a weekly HTML report using LINUX Scripting. This will have record counts of some files. (like below) touch path/filename.html echo "Weekly Summary Report for Business Date : $P_BUS_DT">path/filename.html export A1=`cat path/filename1.txt |wc -l` echo "A1... (6 Replies)
Discussion started by: dsfreddie
6 Replies

10. Shell Programming and Scripting

Needed script to FTP a File and generate a quality checksum file

hi all i want a script to FTP a file and should generate a quality checksum file means when I FTP a file from one server to another server it should generate a QC file which should contain timestamp,no.of records in that file Thanks in advance saikumar (3 Replies)
Discussion started by: hemanthsaikumar
3 Replies
extendfs(8)						      System Manager's Manual						       extendfs(8)

NAME
extendfs - Extends UFS file systems SYNOPSIS
/sbin/extendfs [- s] [disk_blocks] device_name DESCRIPTION
Use the extendfs command to increase the storage space in a UFS file system. The file system must not be mounted when you perform this operation. To extend a mounted (in use) UFS file system, use the mount command with the -o extend option. The procedure for increasing the storage space of a UFS file system is as follows: Look at the contents the /etc/fstab file to identify the disk partition that maps to the file system. Ensure that there is available storage space on the target disk as follows: If LSM is in use on your system, use LSM commands to increase the size of the LSM volume as described in the Logical Storage Manager guide. If LSM is not in use on your system, use the disklabel command or the diskconfig graphical user interface to check the current size and use of partitions on the disk. If there is adequate space on an adjacent partition, use the disklabel command to write the current label to a file as fol- lows: # disklabel -r dsk4 > d4label Edit the disklabel file to change the size of the partition on which your UFS file system resides. Increase the number of disk blocks on the partition and decrease the disk block size of the adjacent partition by an equivalent number. Use the disklabel command with the -R option to write the revised label to the raw disk as follows: # disklabel -R /dev/rdisk/dsk4 d4label When the disk label is revised, extend the file system using the extendfs command. You can either use the full extent of the newly sized partition or extend the file system in stages. The following example commands show both methods. To extend the file system to use all the available space, you specify the disk partition on which the file system resides, as follows: # extendfs /dev/disk/dsk4g To extend the file system to use only part of the available space, you specify a number of disk blocks, as follows: # extendfs -s 300000 /dev/disk/dsk4g The remainder of the extended partion is reserved for future use. You can extend a file system as many times as necessary, up to the physical limit of the storage device. When no more space is available on the storage device, you must back up the file system using the dump command and restore the file system to a storage device that has more available space. Once you have extended a file system, the operation cannot be reversed except by a back up and restore operation. Use the dump command to back up the file system. You can then reset the partition sizes manually and restore the file system to the storage device. ERRORS
The disklabel command produces output similar to that of the newfs command. If a list of disk blocks is not displayed on the terminal, the command has failed. Verify the partition settings and the mount status of the target file system. The disklabel command does not permit you to overwrite a partition if it is in use. Refer to the disklabel(8) reference page for more information on label errors. FILES
Specifies the command path. RELATED INFORMATION
diskconfig(8), disklabel(8), mount(8), and fstab(4). extendfs(8)
All times are GMT -4. The time now is 03:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy