|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Splitting a delimited text file
Post 302899390 by lupin..the..3rd on Monday 28th of April 2014 06:00:10 PM
04-28-2014
Quote:
If your input files contain 12,000 messages, your script is invoking awk 12,000 times; not 2 times!
In your sample input in the first message in this thread you showed that the 1st line in your input file started with --dump. From those error messages, I have to assume that the 1st line of your input file does not start with that string.
In your sample input in the first message in this thread you showed that the 1st line in your input file started with --dump. From those error messages, I have to assume that the 1st line of your input file does not start with that string.
The first line of the input file does not begin with "--dump". There are a few lines of metadata at the head of the file. This metadata can be discarded. After the few lines of metadata, it's all "--dump" delimited emails.Thanks again for the suggestions, I'll try them when I'm back in the office tomorrow morning.
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Could one of you shad some light on this:
I need to split the file by determining the record count and than splitting it up into 4 files. Please note, this is not a fixed record length but rather a "|" delimited file.
I am not sure as how to handle reminder/offset for the 4th file.
For... (4 Replies)
Discussion started by: ddedic
4 Replies
2. Shell Programming and Scripting
Hi all, I need help to split a tab-delimited list into separate files by the filename-field. The list is already sorted ascendingly by filename, an example list would look like this;
filename001 word1 word2
filename001 word3 word4
filename002 word1 word2
filename002 word3 word4... (4 Replies)
Discussion started by: perkele
4 Replies
3. Shell Programming and Scripting
Hello
We have a text file with 400,000 lines and need to split into multiple files each with 5000 lines ( will result in 80 files)
Got an idea of using head and tail commands to do that with a loop but looked not efficient.
Please advise the simple and yet effective way to do it.
TIA... (3 Replies)
Discussion started by: prvnrk
3 Replies
4. UNIX for Dummies Questions & Answers
Hello Gurus,
I have a text file containing nearly 12,000 tab delimited characters with 4000 rows. If the file size is small, excel can convert the text into coloumns. However, the file that I have is very big. Can some body help me in solving this problem?
The input file example,
... (6 Replies)
Discussion started by: Unilearn
6 Replies
5. Linux
Hello,
Please help me. I have hundreds of text files composed of several rows of information and I need to separate each row into a new text file. I was trying to figure out how to split the text file into different text files, based on each row of text in the original text file. Here is an... (2 Replies)
Discussion started by: dvdrevilla
2 Replies
6. UNIX for Dummies Questions & Answers
I have a text file with irregular spacing between values which makes it really difficult to manipulate. Is there an easy way to convert it into a space delimited text file so that all the spaces, double spaces, triple spaces, tabs between numbers are converted into spaces. The file looks like this:... (5 Replies)
Discussion started by: evelibertine
5 Replies
7. Shell Programming and Scripting
hi
i have a requirement to input a string to a shell script and to split the string to multiple fields,
the string is copied from a row of three columns (name,age,address) in an excel sheet.
the three columns (from excel) are seperated with a tab when pasted in the command prompt, but when the ... (2 Replies)
Discussion started by: midhun19
2 Replies
8. UNIX for Dummies Questions & Answers
Hi, I have a rquirement in unix as below .
I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column.
ex:
Input Text file:
1|A|apple
2|B|bottle
excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies
9. Shell Programming and Scripting
Hi,
I have a requirement that has 50-60 million records that we need to split a delimited string (Delimeter is newline) into rows.
Source Date:
SerialID UnidID GENRE
100 A11 AAAchar(10)BBB
200 B11 CCCchar(10)DDD(10)ZZZZ
Field 'GENRE' is a string with new line as delimeter and not sure... (5 Replies)
Discussion started by: techmoris
5 Replies
10. Shell Programming and Scripting
Hello,
I have some large text files that look like,
putrescine
Mrv1583 01041713302D
6 5 0 0 0 0 999 V2000
2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0
5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0
3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies
LEARN ABOUT DEBIAN
lvmdump
LVMDUMP(8) System Manager's Manual LVMDUMP(8) NAME
lvmdump - create lvm2 information dumps for diagnostic purposes SYNOPSIS
lvmdump [options] [-d directory] DESCRIPTION
lvmdump is a tool to dump various information concerning LVM2. By default, it creates a tarball suitable for submission along with a prob- lem report. The content of the tarball is as follows: - dmsetup info - table of currently running processes - recent entries from /var/log/messages (containing system messages) - complete lvm configuration and cache (content of /etc/lvm) - list of device nodes present under /dev - list of files present /sys/block - list of files present /sys/devices/virtual/block - if enabled with -m, metadata dump will be also included - if enabled with -a, debug output of vgscan, pvscan and list of all available volume groups, physical volumes and logical volumes will be included - if enabled with -c, cluster status info OPTIONS
-h -- print help message -a -- advanced collection WARNING: if lvm is already hung, then this script may hang as well if -a is used -m -- gather LVM metadata from the PVs This option generates a 1:1 dump of the metadata area from all PVs visible to the system, which can cause the dump to increase in size considerably. However, the metadata dump may represent a valuable diagnostic resource. -d directory -- dump into a directory instead of tarball By default, lvmdump will produce a single compressed tarball containing all the information. Using this option, it can be instructed to only produce the raw dump tree, rooted in directory. -c -- if clvmd is running, gather cluster data as well ENVIRONMENT VARIABLES
LVM_BINARY The LVM2 binary to use. Defaults to "lvm". Sometimes you might need to set this to "/sbin/lvm.static", for example. DMSETUP_BINARY The dmsetup binary to use. Defaults to "dmsetup". Red Hat, Inc. LVM TOOLS 2.02.95(2) (2012-03-06) LVMDUMP(8)