Data to import the database as snippets Post: 302983282

Sponsored Content

Top Forums Shell Programming and Scripting Data to import the database as snippets Post 302983282 by blastit.fr on Sunday 9th of October 2016 07:11:23 PM

10-09-2016

Registered User

Hi lxdorney,
The script should fit your request .

Code:

#
#data2imp.sh
#we need first to format the input file
awk 'NF{
     gsub(/  */," ")
     printf "%s" ,$0 ;istext++;next}
     istext{print "";istext=0}' 1181.txt  > 1181.res
# Creation of a sample .csv file , ready for database input
# Take care there is no semicolumn inside the original text . In such case the field separator might be changed
awk -v OFS=";" '
NR==1 {DATE=$1 " " $2}
NR==2 { if (length($0) <= 100) TITLE = $0
        else {
           PART2=substr($0,51)
           dotposition=index(PART2,".")
           if (dotposition == 0) {
             TITLE = substr($0,1,100) "..."}
           else {
             TITLE = substr($0,1,50 + dotposition)
           }
        }
       }
NR==4{ if (length($0) <= 500) SNIPPET = $0
        else {
           PART2=substr($0,401)
           dotposition=index(PART2,".")
           if (dotposition == 0) {
             SNIPPET = substr($0,1,500) "..."}
           else {
             SNIPPET = substr($0,1,400 + dotposition)
           }
        }
print DATE,TITLE,SNIPPET
}' 1181.res >1181.csv

Last edited by blastit.fr; 10-09-2016 at 08:15 PM.. Reason: comments added

blastit.fr

View Public Profile for blastit.fr

Find all posts by blastit.fr

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Import data from compressed file

HI I need to import data from a file which is in comressed format but system doesn't have enough space to uncompress file Is there any way so that i can do import from compressed file.

2. Programming

Code Snippets

Can Anyone give me an implimentation of virtual memory (simulation using paging only) .it should have the following algos for page replacement 1. LRU 2.FIFO 3.Clock references to web sites would be gr8 too it should have the code/algo no executables(in C only)

3. Windows & DOS: Issues & Discussions

import data files from Unix to Windows?

Hi, Is there any way to import data files from Unix system to Windows system? I have many data files on Unix machine generated every night. I need to pick certain data from each file and plug them into this windows file on the network share drive. Anyone has any idea? Thanks in advance!

4. UNIX for Dummies Questions & Answers

How can import data files to XL sheet.

Hi, I have the file(F1.XL) in Unix Box. it's updating every 1hr. I would like to import f1.xl to Windows excel sheet, when i need see the reports. can any one clarify, is there any VB script for importing data from UNIX, like sql connection.... thanks

5. Shell Programming and Scripting

Data Import perl script

Hi, I have a requirement for creating a Perl Script which will perform Data Import process in an automated way and I am elaborating herewith : Section 1 ) - use the following command line format : "./import.pl -h hostname -p port -f datafile.txt" Section 2) datafile.txt will...

6. Shell Programming and Scripting

Shell snip to import CSV data into BASH array

I have been trying to write a simple snip of bash shell code to import from 1 to 100 records into a BASH array. I have a CSV file that is structured like: record1,item1,item2,item3,item4,etc.,etc. .... (<= 100 items) record2,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)...

7. UNIX for Dummies Questions & Answers

Import dump to database

Hi... I have dump in unix machine...How can I this import dump to Oracle database? Many thanks in advance.

8. Solaris

import lun data to mount point - Solaris 10

Hi Guys, I have EMC Storage and from this storage I have maped lun5 to Sun Solaris server and I have created on this lun mount point with name /application I have anothere Sun Solaris server and I'll colne lun5 to lun10 from storage level so the data of lun5 will be in lun10 how to...

9. Shell Programming and Scripting

shellscript to read data from txt file and import to oracle db

Hi all, Help needed urgently. I am currently writing a shellscript to read data/record from a flat file (.txt) file, and import/upload the data to oracle database. The script is working fine, but it takes too long time (for 18000 records, it takes around 90 mins). I guess it takes so long...

10. UNIX for Dummies Questions & Answers

SSH import mysql database

Hi all, I am trying to import a database in putty with the syntax: mysql –u database_username –p database_name < filename.mysql As you can see in the screenshot it asks me for the database password - which suggests that the syntax is correct - but then after I enter the password it gives...

LEARN ABOUT LINUX

igawk

IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME

       igawk - gawk with include files

SYNOPSIS

       igawk [ all gawk options ] -f program-file [ -- ] file ...
       igawk [ all gawk options ] [ -- ] program-text file ...

DESCRIPTION

       Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).

       AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like

	      @include getopt.awk

       in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.

OPTIONS

       See gawk(1) for a full description of the AWK language and the options that gawk supports.

EXAMPLES

       cat << EOF > test.awk
       @include getopt.awk

       BEGIN {
	    while (getopt(ARGC, ARGV, "am:q") != -1)
		 ...
       }
       EOF

       igawk -f test.awk

SEE ALSO

       gawk(1)

       Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.

AUTHOR

       Arnold Robbins (arnold@skeeve.com).

Free Software Foundation					    Nov 3 1999								  IGAWK(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Import data from compressed file

Discussion started by: ap_gore79

2. Programming

Code Snippets

Discussion started by: wojtyla

3. Windows & DOS: Issues & Discussions

import data files from Unix to Windows?

Discussion started by: whatisthis

4. UNIX for Dummies Questions & Answers

How can import data files to XL sheet.

Discussion started by: koti_rama