06-14-2013
Well, rows are in databases and spreadsheets, text just has characters, lines, maybe pages. If you muck with the data, it is not always appreciated by tools that like the original. Adding data, lf characters for folding, will be different than replacing data, longer file for starters.
You have a line too long = full file and null characters. You can look at it using 'od' or similar tools, even 'cat -vt' can be useful. Maybe the nulls should be line feeds?
If you turn all the spaces into line feeds using tr, you have a lot of small lines you can concatenate as you please using the line oriented tools.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this.
For example:
split -l 3000000 filename.txt
This is very slow and it splits the file with 3 million records in each... (10 Replies)
Discussion started by: madhunk
10 Replies
2. Shell Programming and Scripting
I have one large file, after every 200 line i have to split the file and the add header and footer to each small file?
It is possible to add different header and footer to each file? (1 Reply)
Discussion started by: ashish4422
1 Replies
3. Shell Programming and Scripting
Hi, Please help on this. i want split the below file(11020111.CLT) to more files with some condition. :b:
1) %s stating of the report
2) %e ending of the report
example starting of the report:
%sAEGONCA| |MUMBAI | :EXPC|N|D
ending of the report
%eAEGONCA| |MUMBAI | :EXPC
3)so the... (10 Replies)
Discussion started by: krbala1985
10 Replies
4. Shell Programming and Scripting
Hi,
I have a data file xyz.dat similar to the one given below,
2345|98|809||x|969|0
2345|98|809||y|0|537
2345|97|809||x|544|0
2345|97|809||y|0|651
9685|98|809||x|321|0
9685|98|809||y|0|357
9685|98|709||x|687|0
9685|98|709||y|0|234
2315|98|809||x|564|0
2315|98|809||y|0|537... (2 Replies)
Discussion started by: nithins007
2 Replies
5. Shell Programming and Scripting
Hi all,
I'm pretty new to Shell scripting and I need some help to split a source text file into multiple files. The source has a row with pattern where the file needs to be split, and the pattern row also contains the file name of the destination for that specific piece. Here is an example:
... (2 Replies)
Discussion started by: cul8er
2 Replies
6. Shell Programming and Scripting
Hi ,
I do have a fixedwidth flatfile that has data for 10 different datasets each identified by the first two digits in the flatfile.
01 in the first two digit position refers to Set A
02 in the first two digit position refers to Set B and so on
I want to genrate 10 different files from my... (6 Replies)
Discussion started by: okkadu
6 Replies
7. Shell Programming and Scripting
Hello;
I have a file consists of 4 columns separated by tab. The problem is the third fields. Some of the them are very long but can be split by the vertical bar "|". Also some of them do not contain the string "UniProt", but I could ignore it at this moment, and sort the file afterwards. Here is... (5 Replies)
Discussion started by: yifangt
5 Replies
8. Shell Programming and Scripting
I need to split a file if it is over 2GB in size (or any size), preferably split on the lines. I have figured out how to get the file size using awk, and I can split the file based on the number of lines (which I got with wc -l) but I can't figure out how to connect them together in the script.
... (6 Replies)
Discussion started by: ssemple2000
6 Replies
9. Shell Programming and Scripting
Good day all
I need some helps,
say that I have data like below, each field separated by a tab
DATE NAME ADDRESS
15/7/2012 LX a.b.c
15/7/2012 LX1 a.b.c
16/7/2012 AB a.b.c
16/7/2012 AB2 a.b.c
15/7/2012 LX2 a.b.c... (2 Replies)
Discussion started by: alexyyw
2 Replies
10. UNIX for Beginners Questions & Answers
Dear Users,
Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file
input file.txt
scaffold1 928 929 C/T +
scaffold1 942 943 G/C +
scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies
COL(1) BSD General Commands Manual COL(1)
NAME
col -- filter reverse line feeds from input
SYNOPSIS
col [-bfpx] [-l num]
DESCRIPTION
Col filters out reverse (and half reverse) line feeds so the output is in the correct order with only forward and half forward line feeds,
and replaces white-space characters with tabs where possible. This can be useful in processing the output of nroff(1) and tbl(1).
Col reads from standard input and writes to standard output.
The options are as follows:
-b Do not output any backspaces, printing only the last character written to each column position.
-f Forward half line feeds are permitted (``fine'' mode). Normally characters printed on a half line boundary are printed on the follow-
ing line.
-p Force unknown control sequences to be passed through unchanged. Normally, col will filter out any control sequences from the input
other than those recognized and interpreted by itself, which are listed below.
-x Output multiple spaces instead of tabs.
-lnum Buffer at least num lines in memory. By default, 128 lines are buffered.
The control sequences for carriage motion that col understands and their decimal values are listed in the following table:
ESC-7 reverse line feed (escape then 7)
ESC-8 half reverse line feed (escape then 8)
ESC-9 half forward line feed (escape then 9)
backspace moves back one column (8); ignored in the first column
carriage return (13)
newline forward line feed (10); also does carriage return
shift in shift to normal character set (15)
shift out shift to alternate character set (14)
space moves forward one column (32)
tab moves forward to next tab stop (9)
vertical tab reverse line feed (11)
All unrecognized control characters and escape sequences are discarded.
Col keeps track of the character set as characters are read and makes sure the character set is correct when they are output.
If the input attempts to back up to the last flushed line, col will display a warning message.
SEE ALSO
expand(1), nroff(1), tbl(1)
STANDARDS
The col utility conforms to the Single UNIX Specification, Version 2. The -l option is an extension to the standard.
HISTORY
A col command appeared in Version 6 AT&T UNIX.
BSD
June 17, 1991 BSD