Sponsored Content
Top Forums Shell Programming and Scripting How to read and Split a file? Post 302885923 by Don Cragun on Wednesday 29th of January 2014 02:35:34 AM
Old 01-29-2014
Quote:
Originally Posted by azherkn3
First file having 100 records and last file having 33 and rest will have 1000 records each
I don't get it. Shouldn't the number of records in the last file be whatever is leftover after putting the first 100 lines from your input file in one file, and putting 1000 lines in subsequent files until less than 1000 lines are left? Are you saying that if your input file contains 1500 lines, you want 100 lines in one output file, 1000 lines in another file, 367 lines in the next file, and 33 lines in the last file? Why shouldn't it be 100 line in the first file, 1000 lines in the second file and 400 lines in the last file?

And, message #16 in this thread you said you wanted to call your script as:
Code:
./a.sh 2 1000 5

I see where the 1000 makes sense, but I don't understand what the 2 and the 5 are intended to tell your script. Why isn't the script to be invoked with something like:
Code:
./a.sh lines_to_go_in_file1 max_lines_to_go_in_remaining_files input_filename

This User Gave Thanks to Don Cragun For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Split a file with no pattern -- Split, Csplit, Awk

I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this. For example: split -l 3000000 filename.txt This is very slow and it splits the file with 3 million records in each... (10 Replies)
Discussion started by: madhunk
10 Replies

2. Programming

Cannot read a file with read(fd, buffer, buffersize) function

# include <stdio.h> # include <fcntl.h> # include <stdlib.h> # include <sys/stat.h> int main(int argc, char *argv) { int fRead, fPadded, padVal; int btRead; int BUFFSIZE = 512; char buff; if (argc != 4) { printf ("Please provide all of the... (3 Replies)
Discussion started by: naranja18she
3 Replies

3. Shell Programming and Scripting

PHP read large string & split in multidimensional arrays & assign fieldnames & write into MYSQL

Hi, I hope the title does not scare people to look into this thread but it describes roughly what I'm trying to do. I need a solution in PHP. I'm a programming beginner, so it might be that the approach to solve this, might be easier to solve with an other approach of someone else, so if you... (0 Replies)
Discussion started by: lowmaster
0 Replies

4. UNIX for Dummies Questions & Answers

Writing awk script to read csv files and split them

Hi Here is my script that calls my awk script #!/bin/bash set -x dir="/var/local/dsx/csv" testfile="$testfile" while getopts " f: " option do case $option in f ) testfile="$OPTARG";; esac; done ./scriptFile --testfile=$testfile >> $dir/$testfile.csv It calls my awk... (1 Reply)
Discussion started by: ladyAnne
1 Replies

5. UNIX for Dummies Questions & Answers

When reading a csv file, counter to read 20 lines and wait for minute then read next 20 till end

Hello All, i am a newbie and need some help when reading a csv file in a bourne shell script. I want to read 10 lines, then wait for a minute and then do a reading of another 10 lines and so on in the same way. I want to do this till the end of file. Any inputs are appreciated ... (3 Replies)
Discussion started by: victor.s
3 Replies

6. Shell Programming and Scripting

Split File by Pattern with File Names in Source File... Awk?

Hi all, I'm pretty new to Shell scripting and I need some help to split a source text file into multiple files. The source has a row with pattern where the file needs to be split, and the pattern row also contains the file name of the destination for that specific piece. Here is an example: ... (2 Replies)
Discussion started by: cul8er
2 Replies

7. Shell Programming and Scripting

Read from file and execute the read command

Hi, I am facing issues with the below: I have a lookup file say lookup.lkp.This lookup.lkp file contains strings delimited by comma(,). Now i want to read this command from file and execute it. So my code below is : Contents in the lookup.lkp file is : c_e,m,a,`cd $BOX | ls cef_*|tail... (7 Replies)
Discussion started by: vital_parsley
7 Replies

8. Shell Programming and Scripting

Read a File line by line and split into array word by word

Hi All, Hope you guys had a wonderful weekend I have a scenario where in which I have to read a file line by line and check for few words before redirecting to a file I have searched the forum but,either those answers dint work (perhaps because of my wrong under standing of how IFS... (6 Replies)
Discussion started by: Kingcobra
6 Replies

9. Shell Programming and Scripting

Read from file and split varialble

Hello Experts, Can anybody assist me in writing a code to do the following: I have a file present in the same directory from where this cod would run. The content of the file would be as below: Config Filename: config_details.txt Format: Server_prefix,IP_of_server,username, password,... (1 Reply)
Discussion started by: chetanojha
1 Replies

10. UNIX for Beginners Questions & Answers

Read Table in file and split

HELLO I need your help please , i need to read a file that contain a table like : Name | Status ------------------ DB 1 | UP DB 2 | UP DB 3 | DOWN DB 4 | UP DB 5 | UP the objective to read each line and check if DB is UP or Down and give me the name of Down database.... (10 Replies)
Discussion started by: Abdellah
10 Replies
rl(1)								   User Commands							     rl(1)

NAME
rl - Randomize Lines. SYNOPSIS
rl [OPTION]... [FILE]... DESCRIPTION
rl reads lines from a input file or stdin, randomizes the lines and outputs a specified number of lines. It does this with only a single pass over the input while trying to use as little memory as possible. -c, --count=N Select the number of lines to be returned in the output. If this argument is omitted all the lines in the file will be returned in random order. If the input contains less lines than specified and the --reselect option below is not specified a warning is printed and all lines are returned in random order. -r, --reselect When using this option a single line may be selected multiple times. The default behaviour is that any input line will only be selected once. This option makes it possible to specify a --count option with more lines than the file actually holds. -o, --output=FILE Send randomized lines to FILE instead of stdout. -d, --delimiter=DELIM Use specified character as a "line" delimiter instead of the newline character. -0, --null Input lines are terminated by a null character. This option is useful to process the output of the GNU find -print0 option. -n, --line-number Output lines are numbered with the line number from the input file. -q, --quiet, --silent Be quiet about any errors or warnings. -h, --help Show short summary of options. -v, --version Show version of program. EXAMPLES
Some simple demonstrations of how rl can help you do everyday tasks. Play a random sound after 4 minutes (perfect for toast): sleep 240 ; play `find /sounds -name '*.au' -print | rl --count=1` Play the 15 most recent .mp3 files in random order. ls -c *.mp3 | head -n 15 | rl | xargs --delimiter=' ' play Roll a dice: seq 6 | rl --count 2 Roll a dice 1000 times and see which number comes up more often: seq 6 | rl --reselect --count 1000 | sort | uniq -c | sort -n Shuffle the words of a sentence: echo -n "The rain in Spain stays mainly in the plain." | rl --delimiter=' ';echo Find all movies and play them in random order. find . -name '*.avi' -print0 | rl -0 | xargs -n 1 -0 mplayer Because -0 is used filenames with spaces (even newlines and other unusual characters) in them work. BUGS
The program currently does not have very smart memory management. If you feed it huge files and expect it to fully randomize all lines it will completely read the file in memory. If you specify the --count option it will only use the memory required for storing the specified number of lines. Improvements on this area are on the TODO list. The program uses the rand() system random function. This function returns a number between 0 and RAND_MAX, which may not be very large on some systems. This will result in non-random results for files containing more lines than RAND_MAX. Note that if you specify multiple input files they are randomized per file. This is a different result from when you cat all the files and pipe the result into rl. COPYRIGHT
Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 Arthur de Jong. This is free software; see the license for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Version 0.2.7 Jul 2008 rl(1)
All times are GMT -4. The time now is 12:51 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy