Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Extracting information from text fields. Post 302120067 by spindoctor on Sunday 3rd of June 2007 03:03:05 PM
Old 06-03-2007
Extracting information from text fields.

Dear friends,
I'm a novice Unix user and I'm trying to learn the ropes. I have a big task I have to accomplish and I'm convinced Unix can get the job done, I just haven't figured out how. I recently posted on the topic of cutting text between unique text patterns and somebody helped me a great deal. It worked great.
There are other tasks, however, that I want to accomplish.
I'm doing a content analysis of newspaper articles that I've exported in .txt format from a ProQuest database. The .rtf files look like this when I cat them in Unix.

Quote:
\
Adoptive parents often face tactless questions: Curiosity lies behind rude\
inquiries:[Final Edition]\
Karen Miles. Edmonton Journal. Edmonton, Alta.:Mar 15, 2000. p. F7 \
\
Author(s): Karen Miles\
\
Document types: News\
\
Section: Living\
\
Publication title: Edmonton Journal. Edmonton, Alta.: Mar 15, 2000. pg. F.7\
\
Source type: Newspaper\
\
ProQuest document 220675241\
ID:\
\
Text Word Count 314\
\
Document URL: {\field{\*\fldinst{HYPERLINK "http://proquest.umi.com"}}{\fldrslt http://proquest.umi.com}}/\
pqdweb?did=220675241&Fmt=3&clientId=14119&RQT=309&VName=PQD\
\
Abstract (Document Summary)\
\
"Is she adopted?" the school registrar whispered to Kathryn Creedy, of\
Alexandria, Va., when Creedy signed Alexis, her then five- year-old Romanian\
daughter, up for school. Since Alexis could clearly hear the question, Creedy\
curtly replied: "Yes, my daughter is adopted -- and she knows it!"\
\
"The media loves to sensationalize stories." Then point out that those stories\
are rare, and that there are many successful people who were adopted -\
- including playwright Edward Albee, Olympic skater Scott Hamilton, actor\
Melissa Gilbert and Apple Computer co- founder Steven Jobs.\
\
\
\
Full Text (314 words)\
\
Copyright Southam Publications Inc. Mar 15, 2000\
\
"Is she adopted?" the school registrar whispered to Kathryn Creedy, of\
Alexandria, Va., when Creedy signed Alexis, her then five- year-old Romanian\
daughter, up for school. Since Alexis could clearly hear the question, Creedy\
curtly replied: "Yes, my daughter is adopted -- and she knows it!"\
\
But Creedy resented the probe, especially when she later learned that the\
registrar hadn't needed the information. "The woman violated our privacy just\
to satisfy her own curiosity," she says.\
The information contained in the 'fields' such as author, date, document type, and text word count would be immensely valuable to me. I would like to be able to extract the information (preferably only the text after the field titles (author, date, text word count) although this is not necessary. I imagine find and replace functions in Excel could be used to delete that easily. I would like to be able to get this information to some kind of excel database, probably via a .csv file.
Ultimately, I will have hundreds of these news stories to extract the information from.

Does anybody have any suggestions?

Simon
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting information from a template

I have a template that I usually use to generate stats on an hourly basis for a number of cell sites altogether. I would like to be able to write a script that would go to the template and extract the information for any single site at any time during the day. For example, let's say that my... (4 Replies)
Discussion started by: Ernst
4 Replies

2. Shell Programming and Scripting

extracting fields

Hi, i have a line with several fields (indefinite number of - count varies) separated by colon. Now, i need to pick each field (except the first one) and have it assigned to variable within a loop. In other words, in the first iteration of the loop, the variable must be assigned with 2nd... (2 Replies)
Discussion started by: prvnrk
2 Replies

3. Shell Programming and Scripting

Problems with extracting information

Hi all, <select name="comp" id="comp" style="width:130px;"> <?php $sqlcomp = mysql_query("SELECT * FROM comp"); while ($redcomp = mysql_fetch_array($sqlcomp)) { extract($redcomp); echo "<option value=\"$comp_id\">comp_name</option>"; } ?> ... (0 Replies)
Discussion started by: c0mrade
0 Replies

4. Shell Programming and Scripting

Extracting information from Config files /text processing

Hello All, This is my first post on this forums, which I consider one of the best of its kind. The reason for my post is that I want to export some information form Nagios configuration files to a DB. I know that there are other tools available to do this, like NDO, monarch, etc... But I want to... (3 Replies)
Discussion started by: oconmx
3 Replies

5. Shell Programming and Scripting

extracting information from multiple files

Hello there, I am trying to extract (string) information ( a list words) from 4 files and then put the results into 1 file. Currently I am doing this using grep -f list.txt file1 . and repeat the process for the other 3 files. The reasons i am doing that (a) I do know how to code (b) each file... (4 Replies)
Discussion started by: houkto
4 Replies

6. Shell Programming and Scripting

Extracting relevant information from syslogs.

I need to analyse some syslogs and I want to print out all the lines containing SSH connections to the inside interface of the firewall and ignore lines where the originating port is 22. So basically I want to print all matches after "to inside:" that contains /22 and ignore lines where /22 occur... (2 Replies)
Discussion started by: lewk
2 Replies

7. Shell Programming and Scripting

Problems extracting some information

Hi there! Well, I'm writing a script to obtain certain information about files. Specifically, I want to get the information about those files which last access were in the last 24 hours, so I'm doing something like this: find <directory_name> -atime -1 -printf '%f %a\n' I would also... (4 Replies)
Discussion started by: Skirmish
4 Replies

8. Shell Programming and Scripting

Extracting information using awk

I want to write a script that extracts a value from a line of text. I know it can be done using awk but I've never used awk before so I don't know how to do it. The text is: Mem: 100M Active, 2150K Cache, 500M Buf, 10G Free I want to extract the free memory value to use as a variable. In... (5 Replies)
Discussion started by: millsy5
5 Replies

9. Shell Programming and Scripting

[Solved] Extracting information from DDL's

Dear Experts, I need your help here. I have lot of teradata DDL's as follows, i want to extract field names , field attributes and NOT NULL information from DDL.Could you please help here. Sample DDL: CREATE MULTISET TABLE APS_CALL_IN_PICKUP_CANCELED ,NO FALLBACK , NO BEFORE... (2 Replies)
Discussion started by: srikanth38
2 Replies

10. Shell Programming and Scripting

Extracting information from XML to excel

Hi, I am trying to extract information from a XML file and write it to a excel sheet. I am not sure where to start from. Here is the content from my input XML file. <com.cloudbees.hudson.plugins.folder.properties.FolderProxyGroupContainer plugin="nectar-rbac@4.5"> <groups> ... (4 Replies)
Discussion started by: Sajjadmehdi
4 Replies
htload(1)						      General Commands Manual							 htload(1)

NAME
htload - reads in an ASCII-text version of the document database SYNOPSIS
htload [options] DESCRIPTION
Htload reads in an ASCII-text version of the document database in the same form as the -t option of htdig and htdump. Note that this will overwrite data in your databases, so this should be used with great care. OPTIONS
-a Use alternate work files. Tells htload to append .work to database files, allowing it to operate on a second set of databases. -c configfile Use the specified configfile instead of the default. -i Initial. Do not use any old databases. This is accomplished by first erasing the databases. -v Verbose mode. This doesn't have much effect. File Formats Document Database Each line in the file starts with the document id followed by a list of fieldname : value separated by tabs. The fields always appear in the order listed below: u URL t Title a State (0 = normal, 1 = not found, 2 = not indexed, 3 = obsolete) m Last modification time as reported by the server s Size in bytes H Excerpt h Meta description l Time of last retrieval L Count of the links in the document (outgoing links) b Count of the links to the document (incoming links or backlinks) c HopCount of this document g Signature of the document used for duplicate-detection e E-mail address to use for a notification message from htnotify n Date to send out a notification e-mail message S Subject for a notification e-mail message d The text of links pointing to this document. (e.g. <a href="docURL">description</a>) A Anchors in the document (i.e. <A NAME=...) Word Database While htdump and htload don't deal with the word database directly, it's worth mentioning it here because you need to deal with it when copying the ASCII databases from one system to another. The initial word database produced by htdig is already in ASCII format, and a binary version of it is produced by htmerge, for use by htsearch. So, when you copy over the ASCII version of the document database produced by htdump, you need to copy over the wordlist as well, then run htload to make the binary document database on the target system, followed by running htmerge to make the word index. Each line in the word list file starts with the word followed by a list of fieldname : value separated by tabs. The fields always appear in the order listed below, with the last two being optional: i Document ID l Location of word in document (1 to 1000) w Weight of word based on scoring factors c Count of word's appearances in document, if more than 1 a Anchor number if word occurred after a named anchor FILES
/etc/htdig/htdig.conf The default configuration file. /var/lib/htdig/db.docs The default ASCII document database file. /var/lib/htdig/db.wordlist The default ASCII word database file. SEE ALSO
Please refer to the HTML pages (in the htdig-doc package) /usr/share/doc/htdig-doc/html/index.html and the manual pages htdig(1) , htmerge(1) and htdump(1) for a detailed description of ht://Dig and its commands. AUTHOR
This manual page was written by Stijn de Bekker, based on the HTML documentation of ht://Dig. 15 October 2001 htload(1)
All times are GMT -4. The time now is 05:13 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy