Gathering data from complex/large dataspreads .txt format


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Gathering data from complex/large dataspreads .txt format
# 1  
Old 05-18-2010
Gathering data from complex/large dataspreads .txt format

Hi, I'm working on gathering information stored in .txt files. The format of the data within the .txt files is shown in the picture uploaded with this post. Sections like the one pictured are repeated (with different data, same format) many times within each .txt file but each section is of data from different instances.

I have very little experience with coding/unix etc.
I am using terminal with Mac OS X.
I need to use a program because the amount of data I have to comb through makes brute force copy and pasting impractical.

Within each instance i need to find and extract the time, location, the data listed in the lines marked MOT, HNR, IBM, ABR, MWM, OFK, ALO, SOW, AND SBM. Then i need to transfer that information into a new .txt file.

I'm not sure where to start basically.
Hopefully i provided enough information to get a start but if you need more please ask.

Thank you.
Gathering data from complex/large dataspreads .txt format-datasetjpg
# 2  
Old 05-18-2010
Please post the file in text format (instead of a picture) and the desired output between code tags.
# 3  
Old 05-18-2010
The .txt file is too large and I havn't been able to open the file without freezing. Weirdly, it shows the full text in the preview window in explorer.

I also don't know how to use code tags properly for this situation.
# 4  
Old 05-18-2010
you might try to head the file, such as:
Code:
head -25 file.txt

# 5  
Old 05-18-2010
Quote:
Originally Posted by p43hd
The .txt file is too large and I havn't been able to open the file without freezing. Weirdly, it shows the full text in the preview window in explorer.

I also don't know how to use code tags properly for this situation.

from Message tab below this page click "#" to use code tags after selecting a code part or a sentence.

you can split your file into many parts. Lets say your file is 800MB, you simply split it to 40 MBytes of parts:

Code:
split -b 40m FILE

then attach a part here Smilie
 
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Gathering data in columns from multiple files

Hello world! I need to gather all the data from different folders and copy it in an one unique text file in columns format. Let me explain, let´s say "a, b, c" are 3 data files of thousands and thousands lines (in the reality, I have nearly one hundred). "a, b, c" are located in the folders... (5 Replies)
Discussion started by: user10600
5 Replies

2. Shell Programming and Scripting

Need help to format one txt file to required format

Hello Everyone, I have one source file which is genarated by SAP in different format(Which I've never seen). I need to convert that file to required format and I need to read this target file from Datastage to use this in my Jobs. So I do not have any other options except to use Unix script to... (4 Replies)
Discussion started by: Prathyu
4 Replies

3. UNIX and Linux Applications

Ssmtp -t < /path/to/the/message.txt (How to format message.txt for html email)

ssmtp has been running well under Kubuntu 12.04.1 for plain text messages. I would like to send html messages with ssmtp -t < /path/to/the/message.txt, but I cannot seem to get the message.txt file properly formatted. I have tried various charsets, Content-Transfer-Encoding, rearranging the... (0 Replies)
Discussion started by: Ronald B
0 Replies

4. UNIX for Dummies Questions & Answers

Use Regex to identify / format a complex string

First of all, please have mercy on me. I am not a noob to programming, but I am about as noob as you can get with regex. That being said, I have a problem. I've got a string that looks something like this: Publication - Bob M. Jones, Tony X. Stark, and Fred D. Man, \"Really Awesome Article... (1 Reply)
Discussion started by: egill
1 Replies

5. Red Hat

What is the best tools for performance data gathering and analysis?

Dear Guru, IHAC who complaint that his CentOS is getting performance issue. I have to help him out of there. Could you please tell me which tools is better to gathering the whole system performance data? -- CPU/Memory/IO(disk & Network)/swap I would like the tools could be... (6 Replies)
Discussion started by: devyfong
6 Replies

6. UNIX for Advanced & Expert Users

Gathering data using SAR

Hi everyone, I would like to ask if it is possible to gather SAR data on a specified time. let say yesterdays report, i want to get data at around 12PM and 5PM. thank you. (2 Replies)
Discussion started by: cwiggler
2 Replies

7. Shell Programming and Scripting

AWK CSV to TXT format, TXT file not in a correct column format

HI guys, I have created a script to read 1 column in a csv file and then place it in text file. However, when i checked out the text file, it is not in a column format... Example: CSV file contains name,age aa,11 bb,22 cc,33 After using awk to get first column TXT file... (1 Reply)
Discussion started by: mdap
1 Replies

8. Shell Programming and Scripting

Data gathering script

I am pretty new at shell scripting, but I managed to make a primative shell script to connect from my webserver (bluehost) to an external data server via ftp or sftp or whatever I need. This script is : #!/bin/sh HOST='ftp.host.com' USER='username' PASSWD='password' FILE='file' ftp -n... (7 Replies)
Discussion started by: mesoeast
7 Replies
Login or Register to Ask a Question