Sponsored Content
Top Forums Shell Programming and Scripting AWK Shell Program to Split Large Files Post 302329803 by mkastin on Monday 29th of June 2009 12:24:02 PM
Old 06-29-2009
Is it possible to substring the field instead of using the 'NF' option because in the files it is possible to there to be anywhere from 6 - 50 populated fields between the field to split on and the 'X' at the EOL. Sorry that I'm being such a pain, but I really appreciate the help. Also I'm looking for the output names to be like this <infile name>_<string>.dat

Thanks again and again!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies

2. Shell Programming and Scripting

Split line to multiple files Awk/Sed/Shell Script help

Hi, I need help to split lines from a file into multiple files. my input look like this: 13 23 45 45 6 7 33 44 55 66 7 13 34 5 6 7 87 45 7 8 8 9 13 44 55 66 77 8 44 66 88 99 6 I want to split every 3 lines from this file to be written to individual files. (3 Replies)
Discussion started by: saint2006
3 Replies

3. Shell Programming and Scripting

Split large zone file dump into multiple files

I have a large zone file dump that consists of ; DNS record for the adomain.com domain data1 data2 data3 data4 data5 CRLF CRLF CRLF ; DNS record for the anotherdomain.com domain data1 data2 data3 data4 data5 data6 CRLF (7 Replies)
Discussion started by: Bluemerlin
7 Replies

4. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

5. Shell Programming and Scripting

Split Large Files Based On Row Pattern..

Hi all. I've tried searching the web but could not find similar problem to mine. I have one large file to be splitted into several files based on the matching pattern found in each row. For example, let's say the file content: ... (13 Replies)
Discussion started by: aimy
13 Replies

6. Shell Programming and Scripting

Process multiple large files with awk

Hi there, I'm camor and I'm trying to process huge files with bash scripting and awk. I've got a dataset folder with 10 files (16 millions of row each one - 600MB), and I've got a sorted file with all keys inside. For example: a sample_1 200 a.b sample_2 10 a sample_3 10 a sample_1 10 a... (4 Replies)
Discussion started by: camor
4 Replies

7. UNIX for Beginners Questions & Answers

sed awk: split a large file to unique file names

Dear Users, Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file input file.txt scaffold1 928 929 C/T + scaffold1 942 943 G/C + scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies

8. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies

9. Shell Programming and Scripting

Split large xml into mutiple files and with header and footer in file

Split large xml into mutiple files and with header and footer in file tried below it splits unevenly and also i need help in adding header and footer command : csplit -s -k -f my_XML_split.xml extrfile.xml "/<Document>/" {1} sample xml <?xml version="1.0" encoding="UTF-8"?><Recipient>... (36 Replies)
Discussion started by: karthik
36 Replies

10. UNIX for Beginners Questions & Answers

Split large file into 24 small files on one hour basis

I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55 can any one help me on this.! ... (20 Replies)
Discussion started by: Raghuram717
20 Replies
USERDB(8)						      Double Precision, Inc.							 USERDB(8)

NAME
userdb - manipulate /etc/courier/userdb SYNOPSIS
userdb {addr} set {field=value...} userdb {addr} unset {field...} userdb {addr} del userdb {path/addr} [set | unset | del] ... userdb -f {file} {adr} [set | unset | del] ... userdb -show {path} userdb -show {path} {addr} userdb -show -f {file} userdb -show -f {file} {addr} DESCRIPTION
userdb is a convenient script to individually manipulate entries in /etc/courier/userdb. See makeuserdb(8)[1] for a description of its contents. /etc/courier/userdb can always be edited using any text editor, but userdb is a convenient way to modify this file from another script. /etc/courier/userdb can also be a subdirectory, instead of a file. Specify foo/bar/addr to manipulate addr in the file /etc/courier/userdb/foo/bar. You can also use the -f flag: -f /etc/courier/userdb/foo/bar is equivalent. Use whatever form makes the most sense to you. /etc/courier/userdb must not have any group or world permissions. That's because its contents may include system passwords (depending upon the application which uses this virtual user account database). Each line in /etc/courier/userdb takes following form: addr specifies a unique virtual address. It is followed by a single tab character, then a list of field=value pairs, separated by vertical slash characters. See makeuserdb(8)[1] for field definitions. A text editor can be used to add blank lines or comments in /etc/courier/userdb. Any blank lines or comments are ignored by the userdb script. The names of the actual fields, and their contents, are defined entirely by applications that use the /etc/courier/userdb database, the userdb command just adds or removes arbitrary fields. For example: userdb default/info set mail=/home/mail/info This command accesses the address "info" in /etc/courier/userdb/default. If the second argument to userdb is "set", the remaining arguments are taken as field=value pairs, which are added to the record for addr. If there is no record for addr, a new record will be appended to the file. If addr exists, any existing values of any specified fields are removed. If =value is missing, userdb stops and prompts for it. This is useful if you're setting a password field, where you do not want to specify the password on the command line, which can be seen by the ps(1) command. If userdb is being executed by a script, the value can be provided on standard input. Use "unset" to delete fields from an existing record. Use "del" to delete all fields in the existing record, plus the record itself. DISPLAYING /etc/courier/userdb If the first argument to userdb is -show, userdb displays the contents of /etc/courier/userdb. If /etc/courier/userdb is a subdirectory, path must refer to a specific file in /etc/courier/userdb. The -f option can be used instead of path in order to specify an arbitrary file. If addr is not specified, userdb produces a list, on standard output, containing all addresses found in the file, on per line. If addr is specified, userdb produces a list, on standard output, of all the fields in /etc/courier/userdb for this addr. REBUILDING /etc/courier/userdb.dat The actual virtual account/address database is /etc/courier/userdb.dat. This is a binary database file. /etc/courier/userdb is the plain text version. After running userdb, execute the makeuserdb(8)[1] command to rebuild /etc/courier/userdb.dat for the changes to take effect. BUGS
addr must be unique. If /etc/courier/userdb is a subdirectory, it's possible to create the same addr in different files in the subdirectory. This is an error that is not currently detected by userdb, however the subsequent makeuserdb(8)[1] command will fail with an error message. FILES
/etc/courier/userdb - plain text file, or directory of plain text files .lock.filename - lock file for filename .tmp.filename - temporary file used to create new contents of filename SEE ALSO
makeuserdb(8)[1], userdbpw(8)[2] NOTES
1. makeuserdb(8) makeuserdb.html 2. userdbpw(8) userdbpw.html Double Precision, Inc. 08/23/2008 USERDB(8)
All times are GMT -4. The time now is 11:54 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy