KSH script -text file processing NULL issues


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting KSH script -text file processing NULL issues
# 1  
Old 10-01-2008
KSH script -text file processing NULL issues

I'm trying to strip any garbage that may be at the end of my text file and that part is working. The problem only seems to be with the really long lines in the file. When the head command is executed I am directing the output to a new file. The new file always get a null in the 4096 position but only on the long lines. The long lines are 4405 long and the rest of the line is in the new file but has a null in position 4096. Not sure what the issue might be. Any thoughts?

Here's my script:

#! /bin/ksh

# Strip off extraneous characters from text file
# To envoke, type ./s.fixfile <filename> <tempfilename>

if [ $# -ne 2 ]
then
echo "Usage: $0 filename tempfilename"
exit 1
elif [ ! -a "$1" ]
then
echo "$1 does not exist"
exit 1
elif [ ! -f "$1" ]
then
echo "$1 is not an ordinary file"
exit 1
elif [ ! -r "$1" -o ! -w "$1" ]
then
echo "$1 is either not a readable or a writable file"
exit 1
fi
integer WCSIZE

WCSIZE=$(cat $1 | wc -l) # Get line count of file

head -$WCSIZE $1 > $2 # Create temp file based on line count

cp $2 $1 # Copy temp file to original

rm $2
# 2  
Old 10-01-2008
Try
Code:
getconf LINE_MAX

Most UNIX tool utilties will not read a line longer than LINE_MAX. C, perl, python, dd, and some others do not have that problem. I'm guessing your LINE_MAX is 4096.
# 3  
Old 10-01-2008
Bingo!

2048 words = 4096 bytes

Thanks, that's just the ticket I was hunting for!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Verify the null filed of the text file

Here is my sample data Test.txt column 1|columne 2|columne 3|columne 4 test|test||test test|test|test| test||test|test test|test|test|test |test|test|test In that example having NULL value of the row 2-column 3,row 3-column 4,row 4 - column 2,row 6- column 1 How i can validate... (5 Replies)
Discussion started by: krish2014
5 Replies

2. UNIX for Dummies Questions & Answers

How to find the null member or blank in the text file?

Hello All, I am new to unix scripting and wanted to know, is it possible if we find any null value or blank record in the text file. For example we have a text file with only one column and there are 90 records. But some times we will have a null value or blank row record in the text file. I... (4 Replies)
Discussion started by: Ram11111
4 Replies

3. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

4. UNIX for Dummies Questions & Answers

Take output of processing in text file

Hi ALL, I am presently using perl script mukesh.pl I just want to catch its output into another text file . So I am using > File.txt . I am getting output but i want the whole processing of the script into that file please let me know . Thanks in advance Cheers Mukesh (1 Reply)
Discussion started by: mumakhij
1 Replies

5. Shell Programming and Scripting

Text processing of file

I have a text file which is a dataset. and I need to convert it into a CSV format The file is as follows : First line : -1 3:1 11:1 14:1 19:1 39:1 42:1 55:1 64:1 67:1 73:1 75:1 76:1 80:1 83:1 Second line " +1 5:1 11:1 15:1 32:1 39:1 40:1 52:1 63:1 67:1 73:1 74:1 76:1 78:1 83:1 There are a... (6 Replies)
Discussion started by: ajayram
6 Replies

6. Shell Programming and Scripting

KSH - help needed for creating a script to generate xml file from text file

Dear Members, I have a table in Oracle DB and one of its column name is INFO which has data in text format which we need to fetch in a script and create an xml file of a new table from the input. The contents of a single cell of INFO column is like: Area:app - aam Clean Up Criteria:... (0 Replies)
Discussion started by: Yoodit
0 Replies

7. Shell Programming and Scripting

Help to improve speed of text processing script

Hey together, You should know, that I'am relatively new to shell scripting, so my solution is probably a little awkward. Here is the script: #!/bin/bash live_dir=/var/lib/pokerhands/live for limit in `find $live_dir/ -type d | sed -e s#$live_dir/##`; do cat $live_dir/$limit/*... (19 Replies)
Discussion started by: lorus
19 Replies

8. Shell Programming and Scripting

awk, perl Script for processing a single line text file

I need a script to process a huge single line text file: The sample of the text is: "forward_inline_item": "Inline", "options_region_Australia": "Australia", "server_event_err_msg": "There was an error attempting to save", "Token": "Yes", "family": "Family","pwd_login_tab": "Enter Your... (1 Reply)
Discussion started by: hmsadiq
1 Replies

9. UNIX for Dummies Questions & Answers

text file processing

Hello! There is a text file, that contains hierarchy of menues, like: Aaaaa->Bbbbb Aaaaa->Cccc Aaaaa-> {spaces} Ddddd (it means that the full path is Aaaaa->Cccc->Ddddd ) Aaaaa-> {more spaces} Eeeee (it means that the full path is Aaaaa->Cccc->Ddddd->Eeeee ) Fffffff->Ggggg... (1 Reply)
Discussion started by: alias47
1 Replies

10. UNIX for Dummies Questions & Answers

Processing a text file

A file contains one name per line, such as: john doe jack bruce nancy smith sam riley When I 'cat' the file, the white space is treated as a new line. For example list=`(cat /path/to/file.txt)` for items in $list do echo $items done I get: john doe (1 Reply)
Discussion started by: TheCrunge
1 Replies
Login or Register to Ask a Question