Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Problem with cut and wc

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Problem with cut and wc
# 8  
Old 05-25-2012
Quote:
Originally Posted by methyl
What Operating System and version are your running and what Shell is this?

Are any Microsoft Operating Systems involved anywhere in the process?

What character set do you use? There is scope for UTF to give misleading character counts if the database was not set to expect UTF.

Is this all within the same computer? i.e. no file transfers, conversions or whatever?

Try the od command mentioned in earlier posts on a bad record.
I am using Solaris 10.
No Microsoft OS involved in this except I use putty to login to UNIX machine from my Windows os on client machine.
Its complete ASCII character set what we use.
Yes, its on same computer but db is on other machine, shouldn't be a matter.
# 9  
Old 05-25-2012
... and the results from od on a bad record?
# 10  
Old 05-25-2012
Quote:
Originally Posted by methyl
... and the results from od on a bad record?
The records have special characters. something like \224 and \230 which is coming as single character on shell prompt after using vi.
# 11  
Old 05-25-2012
Hmm. Extended ASCII (greater than Octal 127). Probably come from a Microsoft system or possibly a foreign language ASCII character set? There is potential for these characters to convert to two-character UTF in your SQL loader or possibly some other multi-character sequence (like the octal sequence cited).
We know so little about your software that this is pure speculation.
# 12  
Old 05-25-2012
the files are coming from mainframe to UNIX via connect:direct software , so that can be one reason.
# 13  
Old 05-25-2012
This is a common problem in multi-platform system design.
I recommend that you instigate a systematic software re-test on a test system, paying particular attention to Extended ASCII character sets.

The big decision is what to do with each Extended ASCII character such that the data loads correctly into your database. This is not trivial.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Problem in using cut command

Hi Friend , i have one file say xyz.lst and it has content like dn: cn=m.hariharan,cn=employee,cn=delhi circle,cn=users,dc=industowers,dc=c dn: cn=ajay.jain,cn=employee,cn=gujarat circle,cn=users,dc=industowers,dc=com dn: cn=ajitkumar.thakor,cn=employee,cn=gujarat... (4 Replies)
Discussion started by: harish anand
4 Replies

2. UNIX for Dummies Questions & Answers

Problem with echo and cut

Hi, My source file happens to be src.src with following contents: 1,liv 52,syrup,twice a day,children When I do a : for ii in `cat src.src` do echo $ii medicine_name=`echo $ii | cut -f -d2","` echo $medicine_name done I get output of echo $medicine_name as : an the output of... (2 Replies)
Discussion started by: dipanchandra
2 Replies

3. Shell Programming and Scripting

Problem in Cut and paste

Hi, I have a file like this --> Consider z as space #cat filename ABC <!--Nzzzzz--> <!--RESUMO--> EFG XYZ <!--Nzzzzz--> <!--RESUMO--> I need to cut the <!--RESUMO--> part and paste it to the previous line so that the file will look like this--> ABC <!--Nzzzzz--><!--RESUMO-->... (4 Replies)
Discussion started by: samsonata
4 Replies

4. Shell Programming and Scripting

Problem with cut command

Hi! I get a md5 hash of a file with this command: openssl md5 /Users/me/MyLogo.png | cut -f 2 -d ' ' "cut" because I just want the hash. But there is a problem -> that doen't work with a path with spaces: openssl md5 /Users/me/MyLogo 2.png | cut -f 2 -d ' ' The result is "2.png)=" ...... (1 Reply)
Discussion started by: Dirk Einecke
1 Replies

5. Shell Programming and Scripting

CUT command problem

Hi I have a strange problem when using cut command when i am using the below command, it is working fine,I am getting the data in new file xyz.dat cut -c 1-75 abc.dat > xyz.dat when i am using the below command, I am getting the data in new file abc.dat , but empty file cut -c 1-75... (4 Replies)
Discussion started by: vaas
4 Replies

6. Shell Programming and Scripting

Problem with cut

I need to read in a file and output it without the comments or newlines. The problem is that is not outputting it properly. STUDENTSDETAILFILE="../data/studentDetails.txt" for getlines in `sed '/#/d' $STUDENTSDETAILFILE` do STUDENTID=`echo $getlines | cut -d: -f1` ... (1 Reply)
Discussion started by: nerdbot
1 Replies

7. Shell Programming and Scripting

Cut and WhiteSpace Problem

Hi, I have a shell script that reads a parameter file to set variables. I have an issue when the parameter I try to read contains whitespace. e.g File Contents Code The result is SUBJECT is set to and I want subject set to I've tried different variations but nothing seems to... (19 Replies)
Discussion started by: Greygor
19 Replies

8. Shell Programming and Scripting

cut problem

Hi, sample datas are : drwxr-xr-x 2 beewin abidev 96 Jun 13 2006 bwin drwxrwxr-x 2 blsmg01 smg 96 Jun 13 2006 blsmg01 drwxr-xr-x 2 ccmdummy ccm_root 8192 Jun 13 2006 ccmdum drwxr-xr-x 5 dipayan users 8192 Oct 29 09:05 dip I want to cut the last field. I use ls -ll | grep... (7 Replies)
Discussion started by: samir_standing
7 Replies

9. Shell Programming and Scripting

Problem with cut command

I am trying to take one part of my text from file and save it to variable $x I tryed this... x=`cut -c 6-9 $fajl` my file looks like this fajl: 21890001277 89386911 23638FBCDC 28EE01A1 0000 26855 124 244326 21890001277 89766911 23638FBCDC 28E021A1 0000 26557 134 684326 21890001277... (7 Replies)
Discussion started by: amon
7 Replies

10. UNIX for Dummies Questions & Answers

Cut command problem !

Hi all! Here is my problem : $ more file yougli:passwd:123456:3265:Yepa Yepo:/home/yougli:/bin/ksh As you can see, in the field "information", there are two spaces between "Yepa" and "yepo". My problem is : $ PARAM='more file | cut -d":" -f5' $ echo $PARAM Yepa Yepo Now i only... (2 Replies)
Discussion started by: tomapam
2 Replies
Login or Register to Ask a Question