Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk to extract incorrect fixed length records

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to extract incorrect fixed length records
# 1  
Old 08-20-2009
awk to extract incorrect fixed length records
I have a number of unix text files containing fixed-length records (normal unix linefeed terminator) where I need to find odd records which are an incorrect length.
The data is not validated and records can contain odd backslash characters and control characters which makes them awkward to process in shell. Files are unlikely to exceed 1000 records, so efficiency is not important. Each file has a different fixed length record size.

I am looking for an awk program which can read the length of the first record of a file, then output only those records which are of a different length to the first record. The program does not need to cater for the first record being wrong.

This post works beautifully, but you need to know the length of the record in advance.

https://www.unix.com/shell-programmin...th-record.html
# 2  
Old 08-20-2009
Something like this:

Code:
awk 'NR==1{len=length}length != len' file

Regards
# 3  
Old 08-20-2009
That worked perfectly. Even with the records containg garbage characters.

Thank you very much.

methyl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting fixed length file using awk

Hi, I need to split a fixed length file of 160 characters based on value of a column. Example: ABC 456780001 DGDG SDFSF BCD 444440002 SSSS TTTTT ABC 777750003 HHHH UUUUU THH 888880001 FFFF LLLLLL HHH 999990002 GGGG OOOOO I need to split this file on basis of column from... (7 Replies)
Discussion started by: Neelkanth
7 Replies

2. Shell Programming and Scripting

awk to print fixed length columns to right side

Hi, I am in a situation to print the message on a column, where the each line starting position should be same. For example code: HOSTNAME1="1.2.3.4.5.6.7" TARGET_DIR="/tmp" echo "HOSTNAME1:" "$HOSTNAME1" | awk -v var="Everyone" '{len=55-length;printf("%s%*s\n",$0,len,var)}' echo... (4 Replies)
Discussion started by: tprabhaker
4 Replies

3. Shell Programming and Scripting

Unix sort for fixed length columns and records

I was trying to use the AIX 6.1 sort command to sort fixed-length data records, sorting by specific columns only. It took some time to figure out how to get it to work, so I wanted to share the solution. The sort man page wasn't much help, because it talks about field delimeters (default space... (1 Reply)
Discussion started by: CheeseHead1
1 Replies

4. Shell Programming and Scripting

Need awk script to compare 2 fields in fixed length file.

Need a script that manipulates a fixed length file that will compare 2 fields in that file and if they are equal write that line to a new file. i.e. If fields 87-93 = fields 119-125, then write the entire line to a new file. Do this for every line in the file. After we get only the fields... (1 Reply)
Discussion started by: Muga801
1 Replies

5. Shell Programming and Scripting

Using awk to search variable length records

New to awk and need some help. I have a script that I would like to make more compact. I want to read a file and grab every field, from every record, except the last field. The records are variable length and have varying number of fields. A record will have at least two fields, but can have... (9 Replies)
Discussion started by: synergy_texas
9 Replies

6. Shell Programming and Scripting

Awk - Working with fixed length files

OK I am somewhat new to UNIX programming please see what you can do to help. I have a flat file that is a fixed length file containing different records based on the 1st character of each line. The 1st number at the beginning of the line is the record number, in this case it's record #1. I... (3 Replies)
Discussion started by: ambroze
3 Replies

7. Shell Programming and Scripting

Fixed Length records- Korne Shell Program.

Hi, I need some help regarding in writing a Korne shell script, in determining the fixed length records in a data file. We have already utility in place, which does this work. The Code for this is as below. In the below $1 is the parameter passed to the script, which is the data file name. ... (4 Replies)
Discussion started by: nrajesh_2009
4 Replies

8. Shell Programming and Scripting

Awk with fixed length files

Hi Unix Champs, I want to awk on a fixed length file. Instead if the file was a delimited file, then I could have used -F and then could have easily done manipulation on the fields. How do i do the same in case of fixed length file? Thanks in Advance. Regards. (7 Replies)
Discussion started by: c2b2
7 Replies

9. Shell Programming and Scripting

fixed length fields in awk

I am trying to display df -h command out in proper format, how can I display each field of each record in a fixed length. (2 Replies)
Discussion started by: roopla
2 Replies

10. Shell Programming and Scripting

creating a fixed length output from a variable length input

Is there a command that sets a variable length? I have a input of a variable length field but my output for that field needs to be set to 32 char. Is there such a command? I am on a sun box running ksh Thanks (2 Replies)
Discussion started by: r1500
2 Replies
Login or Register to Ask a Question