Using awk to search variable length records


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using awk to search variable length records
# 1  
Old 08-21-2008
Using awk to search variable length records

New to awk and need some help. I have a script that I would like to make more compact. I want to read a file and grab every field, from every record, except the last field. The records are variable length and have varying number of fields. A record will have at least two fields, but can have many more. Here is what I am currently using. Can someone provide a compact solution? Thanks.

awk '{ print $1 }' sftplist.txt >> users.txt
awk '{ print $2 }' sftplist.txt >> users.txt
awk '{ print $3 }' sftplist.txt >> users.txt
awk '{ print $4 }' sftplist.txt >> users.txt
awk '{ print $5 }' sftplist.txt >> users.txt
awk '{ print $6 }' sftplist.txt >> users.txt
awk '{ print $7 }' sftplist.txt >> users.txt
awk '{ print $8 }' sftplist.txt >> users.txt
awk '{ print $9 }' sftplist.txt >> users.txt
awk '{ print $10 }' sftplist.txt >> users.txt

This gives me what I want, but then I have a lot of cleanup after it, like removing blank lines and lines that contained the last field when I didn't want the last field.
# 2  
Old 08-21-2008
rev sftplist.txt | cut -f2- | rev > users.txt
# 3  
Old 08-21-2008
Thanks for the suggestion, but your solution still provides me the last field of the record of which I have to cleanup from the file. It's not that big of a deal to perform the cleanup, I just like to keep my scripts as compact as possible and not perform unnecessary processing. Smilie
# 4  
Old 08-21-2008
Any sample input and output?
# 5  
Old 08-21-2008
Sure,
Anything to help.

Here is sftplist.txt

user3 user1 user2 /mydir/next/dir1
user2 /mydir/next/dir2
user1 user4 /staging/next/dir1


Output users.txt

user3
user1
user2
user2
user1
user4

Hope this helps. I can manage the sort -u afterwards. Smilie
# 6  
Old 08-21-2008
rev sftplist.txt | cut -d' ' -f2- | rev | tr ' ' '\n' > users.txt
# 7  
Old 08-21-2008
Here is what I have,

rev sftplist.txt | cut -d ' ' -f2- | rev | tr ' ' '\n' > users.txt

This is what I get.

tr: 0653-712 The combination of options and String parameters is not legal.
Usage: tr [ -[c|C] | -[c|C]ds | -[c|C]s | -ds | -s ] [-A] String1 String2
tr { -[c|C]d | -[c|C]s | -d | -s } [-A] String1

Last edited by synergy_texas; 08-21-2008 at 06:51 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert variable length record to fixed length

Hi Team, I have an issue to split the file which is having special chracter(German Char) using awk command. I have a different length records in a file. I am separating the files based on the length using awk command. The command is working fine if the record is not having any... (7 Replies)
Discussion started by: Anthuvan
7 Replies

2. Shell Programming and Scripting

awk variable search and line count between variable-search pattern

Input: |Running the Rsync|Sun Oct 16 22:48:01 BST 2016 |End of the Rsync|Sun Oct 16 22:49:54 BST 2016 |Running the Rsync|Sun Oct 16 22:54:01 BST 2016 |End of the Rsync|Sun Oct 16 22:55:45 BST 2016 |Running the Rsync|Sun Oct 16 23:00:02 BST 2016 |End of the Rsync|Sun Oct 16 23:01:44 BST 2016... (4 Replies)
Discussion started by: busyboy
4 Replies

3. UNIX for Dummies Questions & Answers

Help with awk, where line length and field position are variable

I have several questions about using awk. I'm hoping someone could lend me a hand. (I'm also hoping that my questions make sense.) I have a file that contains pipe separated data. Each line has similar data but the number of fields and the field position on each line is variable. ... (3 Replies)
Discussion started by: Cheese64
3 Replies

4. Shell Programming and Scripting

changing a variable length text to a fixed length

Hi, Can anyone help with a effective solution ? I need to change a variable length text field (between 1 - 18 characters) to a fixed length text of 18 characters with the unused portion, at the end, filled with spaces. The text field is actually field 10 of a .csv file however I could cut... (7 Replies)
Discussion started by: dc18
7 Replies

5. Shell Programming and Scripting

Make variable length record a fixed length

Very, very new to unix scripting and have a unique situation. I have a file of records that contain 3 records types: (H)eader Records (D)etail Records (T)railer Records The Detail records are 82 bytes in length which is perfect. The Header and Trailer records sometimes are 82 bytes in... (3 Replies)
Discussion started by: jclanc8
3 Replies

6. Shell Programming and Scripting

awk to extract incorrect fixed length records

I have a number of unix text files containing fixed-length records (normal unix linefeed terminator) where I need to find odd records which are an incorrect length. The data is not validated and records can contain odd backslash characters and control characters which makes them awkward to process... (2 Replies)
Discussion started by: methyl
2 Replies

7. Shell Programming and Scripting

awk - replace number of string length from search and replace for a serialized array

Hello, I really would appreciate some help with a bash script for some string manipulation on an SQL dump: I'd like to be able to rename "sites/WHATEVER/files" to "sites/SOMETHINGELSE/files" within the sql dump. This is quite easy with sed: sed -e... (1 Reply)
Discussion started by: otrotipo
1 Replies

8. Shell Programming and Scripting

Using a variable to select records with awk

As part of a bigger task, I had to read thru a file and separate records into various batches based on a field. Specifically, separate records based on the value in the batch field as defined below. The batch field left-justified numbers. The datafile is here > cat infile 12345 1 John Smith ... (5 Replies)
Discussion started by: joeyg
5 Replies

9. Shell Programming and Scripting

use awk to read variable length csv

Any help to read the contents of a variable length csv ....??(using awk) The csv mite look like this : anjali,ram,rahul,mohini,sam,.... and so on ... I need to pick up each name.. Thanks in advance SD (3 Replies)
Discussion started by: shweta_d
3 Replies

10. Shell Programming and Scripting

creating a fixed length output from a variable length input

Is there a command that sets a variable length? I have a input of a variable length field but my output for that field needs to be set to 32 char. Is there such a command? I am on a sun box running ksh Thanks (2 Replies)
Discussion started by: r1500
2 Replies
Login or Register to Ask a Question