04-25-2009
Minimum whitespace separated CSV file generation
Hi,
I have a flat text file consisting of rows (each field separated by '|') from a table. e.g;
NSW|Gulliver Travels|236||5000|BW
This has to be converted to the following format
NSW "Gulliver Travels" 236 5000 BW
No data field has to be left as a blank character so that we have three blank characters in effect between the two consecutive populated fields in the above scenario.
I have actually other flat files also with the same requirement. The no. of fields are different in the other files. So I am in need of a unix script to do this which can be applied directly to all the flat files irrespective of the no. of columns or fields. I have been trying with awk and I accomplished only the following which is not desirable.
NSW Gulliver Travels 236 5000 BW
Also I used hard-coding for this.
Any pointers to solve this are greatly appreciated and would help me a lot.
9 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Say a directory contains files
1) "file name 1.xxx"
2) "file name2.yyy"
3) etc
Using a cshell script, is there a way to (1)search for all files that contain " " in the filename and then (2)rename the files so that you replace the whiitespace " " with a "_".
First problem I run into is... (6 Replies)
Discussion started by: orlando47
6 Replies
2. Shell Programming and Scripting
I have a file in which I clean out a bunch of nonsense text as well as path information.
What I end up with is something like the following:
johnson.........................................................933
Where the periods represent the whitespace
The file comes out originally with... (2 Replies)
Discussion started by: roninuta
2 Replies
3. Shell Programming and Scripting
Hi
Following is an example line.
echo "192.22.22.22 \"33dffwef\" 200 300 dsdsd" | sed "s:\(\ *\ \):\1:"
I want it's output to be
200
However this is not the case. Can you tell me how to do it? I don't want to use AWK for this. Secondly, how can i fetch just 300? Should I use "\2"... (3 Replies)
Discussion started by: shahanali
3 Replies
4. UNIX for Dummies Questions & Answers
Hi,
I have a file like:
0.000000 124.085533 124.085533 124.085533 124.085533 124.085533 124.085533 124.085533 124.085533 124.085533 33.097845 33.363764
0.000000 266.483441 262.519130 266.380993 274.989622 289.594799 309.523518 336.124848 372.386124 413.522043 429.984825 421.621810... (6 Replies)
Discussion started by: cosmologist
6 Replies
5. Shell Programming and Scripting
Hello,
I have some data in a text file where fields are separated by blank lines. There are only 6 fields however some fields have several lines of data as I will explain. Also data in a particular field is not consistently the same size but does end on a blank line. The first field start with... (6 Replies)
Discussion started by: vestport
6 Replies
6. Shell Programming and Scripting
Hi,
I am newbie in unix.
Could someone tell me how do I redirect my grep output to a csv/excel ?
I have used below command but the outputs are appearing in one column Not in different column.
grep -e cmd -e cmd1 test.txt | cut -f 5 | sort | uniq -c> op.csv
I do not understand how do I... (14 Replies)
Discussion started by: kmajumder
14 Replies
7. Shell Programming and Scripting
Hello there,
I have a comma separated csv , and all the text field is wrapped by double quote. Issue is some text field contain comma as well inside double quote. so it is difficult to process.
Input in the csv file is ,
1,234,"abc,12,gh","GH234TY",34
I need output like below,... (8 Replies)
Discussion started by: Uttam Maji
8 Replies
8. UNIX for Dummies Questions & Answers
Hi,
I have a large number of files which are written as csv (comma-separated values).
Does anyone know of simple sed/awk command do achieve this?
Thanks!
---------- Post updated at 10:59 AM ---------- Previous update was at 10:54 AM ----------
Guess I asked this too soon. Found the... (0 Replies)
Discussion started by: lost.identity
0 Replies
9. Shell Programming and Scripting
Hi, I am newbie in shell script.
I need your help to solve my problem.
Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file.
File1:
SourceFile,DateTimeOriginal
/home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies
LEARN ABOUT REDHAT
regexp_table
REGEXP_TABLE(5) File Formats Manual REGEXP_TABLE(5)
NAME
regexp_table - format of Postfix regular expression tables
SYNOPSIS
regexp:/etc/postfix/filename
DESCRIPTION
The Postfix mail system uses optional tables for address rewriting or mail routing. These tables are usually in dbm or db format. Alterna-
tively, lookup tables can be specified in POSIX regular expression form.
To find out what types of lookup tables your Postfix system supports use the postconf -m command.
The general form of a Postfix regular expression table is:
pattern result
When pattern matches a search string, use the corresponding result.
blank lines and comments
Empty lines and whitespace-only lines are ignored, as are lines whose first non-whitespace character is a `#'.
multi-line text
A logical line starts with non-whitespace text. A line that starts with whitespace continues a logical line.
pattern1!pattern2 result
Matches pattern1 but not pattern2.
Each pattern is a regular expression enclosed by a pair of delimiters. The regular expression syntax is described in re_format(7). The
expression delimiter can be any character, except whitespace or characters that have special meaning (traditionally the forward slash is
used). The regular expression can contain whitespace.
By default, matching is case-insensitive, although following the second slash with an `i' flag will reverse this. Other flags are `x' (dis-
able extended expression syntax), and `m' (enable multi-line mode).
Each pattern is applied to the entire lookup key string. Depending on the application, that string is an entire client hostname, an entire
client IP address, or an entire mail address. Thus, no parent domain or parent network search is done, and user@domain mail addresses are
not broken up into their user and domain constituent parts, nor is user+foo broken up into user and foo.
Patterns are applied in the order as specified in the table, until a pattern is found that matches the search string.
Substitution of substrings from the matched expression into the result string is possible using $1, $2, etc.. The macros in the result
string may need to be written as ${n} or $(n) if they aren't followed by whitespace.
EXAMPLE SMTPD ACCESS MAP
# Disallow sender-specified routing. This is a must if you relay mail
# for other domains.
/[%!@].*[%!@]/ 550 Sender-specified routing rejected
# Postmaster is OK, that way they can talk to us about how to fix
# their problem.
/^postmaster@/ OK
# Protect your outgoing majordomo exploders
/^(.*)-outgoing@(.*)$/!/^owner-/ 550 Use ${1}@${2} instead
EXAMPLE HEADER FILTER MAP
# These were once common in junk mail.
/^Subject: make money fast/ REJECT
/^To: friend@public.com/ REJECT
SEE ALSO
pcre_table(5) format of PCRE tables
AUTHOR(S)
The regexp table lookup code was originally written by:
LaMont Jones
lamont@hp.com
That code was based on the PCRE dictionary contributed by:
Andrew McNamara
andrewm@connect.com.au
connect.com.au Pty. Ltd.
Level 3, 213 Miller St
North Sydney, NSW, Australia
Adopted and adapted by:
Wietse Venema
IBM T.J. Watson Research
P.O. Box 704
Yorktown Heights, NY 10598, USA
REGEXP_TABLE(5)