Visit The New, Modern Unix Linux Community


Ascii Mode appending extra records to csv file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Ascii Mode appending extra records to csv file
# 1  
Ascii Mode appending extra records to csv file

I am relatively new to this forum and Unix scripting.

ksh script:
part 1 :will call a PL\SQL program will create 3 CSV file at the unix directory.
part 2 : will sftp the files to the EFT server.

Once the EFT server receives these file , it will transfer them to a shared windows folders.

EFT team uses tumbleweed tools to configure there settings,
Initially with auto transfer mode, each records in the csv file at the windows shared folder has a weird character "[]" at the end of each line to avoid this we changed the tranfer mode setting to "ascii" . The process runs just fine for small files ( say below 2200 records) for a csv with 2700 records , the EFT received the file right ( size was correct say 10mb) but the file when leaving EFT(with ascii setting on ) increases its size (to say 15 mb) and the final CSV when looked at windows shared folders has 3500+ records (instead of 2700+) records.

when the transfer mode is set back to "auto" , its transfers the file right but with a"[]" at the end of each line.
# 2  
Look at some of the lines on the unix box before using sftp to transfer it to the EFT host, then look at the same lines on the EFT host.

Is the file ok after it arrives on the EFT host, If so, the EFT host is most likely a unix or linux box.
Look into the unix command: unix2dos

sftp doesn't have the separate ASCII and binary transfer modes of standard FTP, only binary. All files are transferred literally. Therefore, if you copy ASCII text files between Windows and Unix with sftp, end-of-line characters aren't translated properly.

If the file is ok on the EFT box, maybe they can zip up the file or run the unix2dos command before transferring it to the windows host.
# 3  
some time back i had the similar issue. my csv file was generated from cognos on unix box but it was having some special characters. I just did find and replace from ksh script before processing it.
# 4  
This might be relating to the file format. You can explore doing a unix2dos command on your files before sending to the windows destination.

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #206
Difficulty: Easy
Exterior Gateway Protocol (EGP) is a modern, widely used, routing protocol.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to Insert three extra columns in csv file

Hello Experts, I got a requirement i have a input file which am getting from different source,Now i want to add extra 3 columns to this file like BASE,ACTUAL and DATE. Input File Looks like QUAL CHGE TYP LAW COM1 COM2 A 1 X SED HO ASE B 3 Z CDE SE ... (5 Replies)
Discussion started by: ahmed.vaghar
5 Replies

2. Shell Programming and Scripting

Randomly inserting extra columns into csv file

Hi Tech Guru, I have a test file as below , which needs some more fields to be populated randomly : dks3243;12;20130823;1420;25m;0;syt dks3243;rocy;10 dks3243;kiop;18 sde21p4;77;20151210;8479;7py;9;vfr sde21p4;temp;67 sfq6i01;12;20120123;3412;4rd;7;jui sfq6i01;uymk;90 sfq6i01;kiop;51 ... (8 Replies)
Discussion started by: Lokesha
8 Replies

3. Shell Programming and Scripting

Compare two files with different number of records and output only the Extra records from file1

Hi Freinds , I have 2 files . File 1 |nag|HYd|1|Che |esw|Gun|2|hyd |pra|bhe|3|hyd |omu|hei|4|bnsj |uer|oeri|5|uery File 2 |nag|HYd|1|Che |esw|Gun|2|hyd |uer|oi|3|uery output : (9 Replies)
Discussion started by: i150371485
9 Replies

4. UNIX for Dummies Questions & Answers

To Add extra commas to a CSV file.

Hi All, I got this requirement to process a complex CSV file. Eg File. Line 1: Name:,XYz Line 2: Age:,15 Line 3: Grade:,7 Line 4: Line 5: English, Maths, Science,Spanish Line 6:10,11,13,14 As you can see the maximum column is 4 . The file i need to make is Line 1: Name:,XYz,,... (12 Replies)
Discussion started by: chillblue
12 Replies

5. UNIX for Dummies Questions & Answers

To Add extra commas to a CSV file using 2 files...

Hi , Based on my previous requirement the code works fine for comma as delimiter. Now my Req is widened up a bit .. There will be two set of files .. one with comma as delimiter and other with semi-colon ; as delimiter. Second Sample file. With Double Quotes (Semi-Colon... (1 Reply)
Discussion started by: chillblue
1 Replies

6. Shell Programming and Scripting

Adding Extra Commas to a CSV file

Trying in this forum. Not sure if it is permitted.... but in need of help. Please find the requirements in the below link. https://www.unix.com/unix-dummies-questions-answers/191503-add-extra-commas-csv-file-2.html#post302665179 Thanks in Advance. (1 Reply)
Discussion started by: chillblue
1 Replies

7. Shell Programming and Scripting

Error while appending records to a file

Hi, I have a sample file which contains records. Input File : 1 user1 username1\password@database-name 2 user2 username2\password@database-name 3 user3 username1\password@database-name I should search for a 'username1\' in those records. If 'username1\' is found in those records, that record... (7 Replies)
Discussion started by: siri_886
7 Replies

8. Shell Programming and Scripting

How to find whether a file is transferred thro bin or ascii mode?

Hi Gurus, I need to find out, if i have transferred a file from ftp thro bin mode or ascii mode. Say if i have a file called "dec.sh"( u shuld normally transfer thro ascii mode). How can i find out that dec.sh is tranferred from ftp server thro ascii mode or bin mode? Any help would be... (9 Replies)
Discussion started by: Ashok_oct22
9 Replies

9. Shell Programming and Scripting

Appending file in the read mode.

I have situation, wherein while reading the file, I intend to write the same file & loop ahead until the end of file. Problem I am facing is I can only read the file. Like ... cat file | while read row; do .... done So if I write anything to this file within while block, is not read... (4 Replies)
Discussion started by: videsh77
4 Replies

10. SCO

compressed file was ftpied in ascii mode needs recovery

I am unbale to uncompress a file which was compress then moved to another pc in ascii mode instead of binary mode. Is there any way to recover it. Please help us. While uncompress it is giving corrupt input. (3 Replies)
Discussion started by: raj2610
3 Replies

Featured Tech Videos