parsing file based on characters/bytes


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting parsing file based on characters/bytes
# 1  
Old 12-22-2010
Question parsing file based on characters/bytes

I have a datafile that is formatted as fixed.
I know that each line should contain 880 characters.
I want to separate the file into 2 files, one that has lines with 880 characters and the other file with everything else.
Is this possible ?
# 2  
Old 12-22-2010
Yes,
please post a small representative sample of the input data and an example of the expected output.
# 3  
Old 12-22-2010
Tools Simplifying your request?

Rather than 880 characters, perhaps simpler to see/explain with 8. So, if the following input file:
Code:
AAAAAAAA
BBBBBBBBBB
BBBBBBB
AAAAAAAA
AAAAAAAA
BBBBB

Thus, you want to create two files, selecting on if '8' characters. Therefore you will get file1 with the 3 A records all at 8 characters, and file2 with the 3 B records with lengths not 8 characters.

Is that correct?
# 4  
Old 12-22-2010
example...any line that has 50 chars goes in one file, everything else goes in the other

Code:
HEADER
12345678901234567890123456789012345678901234567890
123456   John      Doe        1 Main St       asdw
789012   John      Doe      22 Main St      xcdf 
345678   John      Doe        1 Main St       fgty
901234   John      Doe        1 Main St       gg
567890   John      Doe        1 Main St       dety
1122337   John      Doe        1 Main St       lgky
445566   John      Doe        1 Main St       oprd
778899   John      Doe        1 Main St         kgit
EOF


12345678901234567890123456789012345678901234567890
123456   John      Doe        1 Main St       asdw
345678   John      Doe        1 Main St       fgty
567890   John      Doe        1 Main St       dety
445566   John      Doe        1 Main St       oprd


HEADER
789012   John      Doe      22 Main St      xcdf 
901234   John      Doe        1 Main St       gg
1122337   John      Doe        1 Main St       lgky
778899   John      Doe        1 Main St         kgit
EOF

---------- Post updated at 10:07 AM ---------- Previous update was at 10:06 AM ----------

joeyg...........you are correct

Last edited by radoulov; 12-22-2010 at 11:07 AM.. Reason: Code tags, please!
# 5  
Old 12-22-2010
Question Does this previous post help?

https://www.unix.com/shell-programmin...alidation.html

The basic premise is:
Code:
echo abcdef | awk 'length==6{print}'
abcdef


Last edited by joeyg; 12-22-2010 at 11:27 AM.. Reason: added text example
# 6  
Old 12-22-2010
how do I pass in the contents of the datafile ?
# 7  
Old 12-22-2010
Code:
awk '{if(length==50) print > "file1"; else print > "file2";}' data.txt


Last edited by anurag.singh; 12-22-2010 at 06:35 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing a file containing special characters

I want to parse a file containing special characters, below is a sample content of file content of file : Serial_no:1$$@#first_name:Rahane$$@last_name:Ajiyenke@@#profession:cricketer!@#*&^ Serial_no:1$$@#first_name:Rahane$$@last_name:Ajiyenke@@#profession:cricketer!@#*&^... (3 Replies)
Discussion started by: rajMjar
3 Replies

2. Shell Programming and Scripting

File Parsing based on a character in a specific field

Hi All, I'm having a hard time finding a starting point for my issue. I have a 30k line file (fspsec.txt) that I would like to parse into smaller files based on any character existing in field 1. ACCOUNTANT LEVEL 1 (ACCT.ACCOUNTANT) OPERATORS: DOEJO (418) TOOLS: Branch Maintenance ... (2 Replies)
Discussion started by: aahlrich
2 Replies

3. Shell Programming and Scripting

Parsing a file based on positional constraints

I have a list file1 like dog cow fox cat fish duck crowI want to classify the elements of file1 based on constrains applied on file2. Additionally the number of elements (words) in the each line of file2 is not fixed. This is my file2 cow cat fox dog cow fox dog fish crow fox dog cat ... (5 Replies)
Discussion started by: sammy777
5 Replies

4. Shell Programming and Scripting

Parsing a file based on next line

I have a file1 like ID E2AK1_HUMAN Reviewed; 630 AA. CC -!- SUBCELLULAR LOCATION: Host nucleus {ECO:0000305}. ID E1A_ADEM1 Reviewed; 200 AA. ID E1A_ADES7 Reviewed; 266 AA. CC -!- SUBCELLULAR LOCATION: Host nucleus... (8 Replies)
Discussion started by: sammy777
8 Replies

5. Shell Programming and Scripting

Find unique lines based off of bytes

Hello All, I have two VERY large .csv files that I want to compare values based on substrings. If the lines are unique, then print the line. For example, if I run a diff file1.csv and file2.csv I get results similar to +_id34,brown,car,2006 +_id1,blue,train,1985... (5 Replies)
Discussion started by: jl487
5 Replies

6. UNIX for Dummies Questions & Answers

X bytes of 0, Y bytes of random data, Z bytes of 5, T bytes of 1. ??

Hello guys. I really hope someone will help me with this one.. So, I have to write this script who: - creates a file home/student/vmdisk of 10 mb - formats that file to ext3 - mounts that partition to /mnt/partition - creates a file /mnt/partition/data. In this file, there will... (1 Reply)
Discussion started by: razolo13
1 Replies

7. Shell Programming and Scripting

parsing characters and number from a big file with brackets

I have a big file with many brackets () in it from which I need to parse number characters and numbers. Below is an example of my file 14 (((A__0:0.02,B__1:0.3)0:0.04,C__0:0.025)2:0.01),(D__0:0.00978,E__2:0.01031)1:0.00362; 15... (1 Reply)
Discussion started by: Lucky Ali
1 Replies

8. Shell Programming and Scripting

Parsing Log File Based on Date & Error

I'm still up trying to figure this out and it is driving me nuts. I have a log file which has a basic format of this... 2010-10-10 22:25:42 Init block 'UA Deployment Date': Dynamic refresh of repository scope variables has failed. The ODBC function has returned an error. The database... (4 Replies)
Discussion started by: k1ko
4 Replies

9. UNIX for Dummies Questions & Answers

Rename file based on first 3 characters of data in file

I'm looking to determine if I can use a grep command to read file and rename the file based on the first 3 characters of the data in the file. An example is: Read FileA If the first 3 positions of the data in the file are "ITP", then rename the file as FileA_ITP, else if the first 3... (3 Replies)
Discussion started by: jchappel
3 Replies

10. Shell Programming and Scripting

Remove first N bytes and last N bytes from a binary file on AIX.

Hi all, Does anybody know or guide me on how to remove the first N bytes and the last N bytes from a binary file? Is there any AWK or SED or any command that I can use to achieve this? Your help is greatly appreciated!! Best Regards, Naveen. (1 Reply)
Discussion started by: naveendronavall
1 Replies
Login or Register to Ask a Question