01-08-2015
ASCII comma-delimited
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
1. The problem statement, all variables and given/known data:
Hi Guys,
I am new on the scripting world and would like ask for help if you can.
Here are my questions below.
1) What Unix commmand(s) would you put in a shell script to read an ASCII comma-delimited input file and validate that field 5 only contains numeric characters?
2) What Unix command(s) would you use from the command line to find all files underneath a given path that have duplicate lines of data?
2. Relevant commands, code, scripts, algorithms:
Perl, awk, cut, uniq
3. The attempts at a solution (include all code and scripts):
Tried this command to tell you honestly I am not sure if this is the right command.
1. sed -e 's/^/| /' -e 's/,/,| /g' -e 's/$/,|/' inputfile | column -t -s,
2 . find . -type f -name '*f*' | sed -r 's|/[^/]+$||' |sort |uniq
4. Complete Name of School (University), City (State), Country, Name of Professor, and Course Number (Link to Course):
This is a work related question some how like a personal research. I was inform by Don Cragun to go here
Note: Without school/professor/course information, you will be banned if you post here! You must complete the entire template (not just parts of it).
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a comma delimited file that sometimes has addresses details in. The problem is that the address detail can be seen as:
"Sample House, Sample Road". When I run a script specifying the file is comma delimited I would like it to ignore comma's that are in between speech marks.
Is this... (2 Replies)
Discussion started by: dbrundrett
2 Replies
2. Shell Programming and Scripting
Hi all,
I have an input file that I am pulling out certain phases using the following commands:
cat /nodes.txt | egrep -e 'OSVersion|PrimaryNodeName'
Currently the output looks like this:
OSVersion - 5.0
PrimaryNodeName - serverA
OSVersion - 5.0
PrimaryNodeName - serverB
OSVersion... (2 Replies)
Discussion started by: indianadoug
2 Replies
3. Shell Programming and Scripting
Hi,
Can anyone let me know on how to convert a Tab delimited file to Comma delimited file in Unix
Thanks!! (22 Replies)
Discussion started by: charan81
22 Replies
4. UNIX for Dummies Questions & Answers
Hi All,
I have output of sql saved in comma separated file.
Now i need to read line by line this file
and assign word to a unix variable for further processing
Eg:
Test file
world, 1, 3, 4
earth,2,3,4,5
moon,1,2,3,4
Output should be
word1= world
word2=1
echo " first word... (7 Replies)
Discussion started by: gwrm
7 Replies
5. UNIX for Dummies Questions & Answers
I have a flat file whose contents are comma delimited and there are 84 columns in total, so everytime I try to view the contents, things get over lapped it becomes diffcult to read through the result set.
Is there a command / what would be the best way...if I want to view the results alligned... (4 Replies)
Discussion started by: priya33184
4 Replies
6. Shell Programming and Scripting
Question about how to change the first & last name in column one & two so that the names have a capital letter for just the first letter. Example:
asdf@asdf.com,asdf,asdfasdf,176.23.22.345,4/12/2012
changed to:
asdf@asdf.com,Asdf,Asdfasdf,176.23.22.345,4/12/2012
Thank you kindly,
Nick (2 Replies)
Discussion started by: nickytcom
2 Replies
7. Shell Programming and Scripting
Hi All,
I need a unix script to convert .csv files to .skv files (changing a comma delimited file to a semi colon delimited file). I am a unix newbie and so don't know where to start. The script will be scheduled using cron and needs to convert each .csv file in a particular folder to a .skv... (4 Replies)
Discussion started by: CarpKing
4 Replies
8. Shell Programming and Scripting
I have an ASCII 28 delimited text file(non-printable delimiter) with 4 columns along with the header.I need to open this text file in Excel or any other tool to view each column content.
Please help..
Thanks (6 Replies)
Discussion started by: aneeta13
6 Replies
9. Shell Programming and Scripting
Hi,
I am wanting to create a script that will construct a SQL statement based on a a space delimited string that it read from a config file.
Example of the SQL will be
For example, it will read a string like "AAA BBB CCC" and assign to a variable named IN_STRING.
I then concatenate... (2 Replies)
Discussion started by: newbie_01
2 Replies
10. UNIX for Beginners Questions & Answers
<Any suggestion how to create a file where the values are separated by ASCII code 1,with data extracted from a table using shell script The format is :/>
<columnname1(binary1)columnvalue(binary1)columnname2(binary1)columnvalue(binary1)columnname3(binary1)columnvalue... 1st row/>... (6 Replies)
Discussion started by: dasun
6 Replies
UNIQ(1) BSD General Commands Manual UNIQ(1)
NAME
uniq -- report or filter out repeated lines in a file
SYNOPSIS
uniq [-c | -d | -u] [-i] [-f num] [-s chars] [input_file [output_file]]
DESCRIPTION
The uniq utility reads the specified input_file comparing adjacent lines, and writes a copy of each unique input line to the output_file. If
input_file is a single dash ('-') or absent, the standard input is read. If output_file is absent, standard output is used for output. The
second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are
not adjacent, so it may be necessary to sort the files first.
The following options are available:
-c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space.
-d Only output lines that are repeated in the input.
-f num Ignore the first num fields in each input line when doing comparisons. A field is a string of non-blank characters separated from
adjacent fields by blanks. Field numbers are one based, i.e., the first field is field one.
-s chars
Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the
first chars characters after the first num fields will be ignored. Character numbers are one based, i.e., the first character is
character one.
-u Only output lines that are not repeated in the input.
-i Case insensitive comparison of lines.
ENVIRONMENT
The LANG, LC_ALL, LC_COLLATE and LC_CTYPE environment variables affect the execution of uniq as described in environ(7).
EXIT STATUS
The uniq utility exits 0 on success, and >0 if an error occurs.
COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation.
SEE ALSO
sort(1)
STANDARDS
The uniq utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'') as amended by Cor. 1-2002.
HISTORY
A uniq command appeared in Version 3 AT&T UNIX.
BSD
December 17, 2009 BSD