Sponsored Content
Full Discussion: Modify CSV file
Top Forums Shell Programming and Scripting Modify CSV file Post 302986079 by durden_tyler on Saturday 19th of November 2016 04:19:26 PM
Old 11-19-2016
Quote:
Originally Posted by omuhans123
...

[COLOR="#738fbf"][SIZE=1]...
I have it now like this in the script:
Code:
#!/bin/bash
file="/archive/ftp/TNM_SUBSDATA_001_20161118235001.txt"
...
...

However the last " behind the text is not present in the output file?
I notice that you have "ftp" in your path and my wild guess is that you are doing ftp from one OS to another and that is messing up your end-of-line characters, due to which you don't see the last character.

Do you observe the missing character after an ftp of "output.txt" from Unix/Linux to Windows?

First thing to check is that the "output.txt" is formed correctly in your Unix/Linux box. Run your script on a small data file (maybe 3 or 4 lines) and once "output.txt" is created, run the following command:

Code:
od -bc output.txt

You should see "\n" as the end-of-line character in Unix/Linux.
If you see "\r\n" as the end-of-line character, then that file is in DOS (or Windows) format.
If you are doing an ftp of the file, ensure that you do a binary ftp and not ascii. The ftp program usually takes care of end-of-line character unless you override it.

If the assumptions above are not true, then your problem might lie somewhere else.
You could start with showing us the output of "od -bc output.txt".
Also, what exactly do you do after the "output.txt" is created to find that the last character is missing?
Do you open it in vim/emacs/nano etc? Do you transfer it somewhere and then open it in some editor?
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to modify csv-file with bash script

Hi Guys, I need to write a script, that exports the "moz_places" table of the "places.sqlite"-file (firefox browser history) into a csv-file. That part works. After the export, my csv looks like this: ... 4429;http://www.sqlite.org/sqlite.html;"Command Line Shell For... (11 Replies)
Discussion started by: Sebi0815
11 Replies

2. Shell Programming and Scripting

Read data from .csv file through shell script & modify

I need to read data from a file called "test.csv" through shell script where the file contains values like name,price,descriptor etc. There are rows where descriptor (& in some rows name) are written as string & other characters like "car_+" OR "bike*" etc where it should contains strings like... (3 Replies)
Discussion started by: raj100
3 Replies

3. Shell Programming and Scripting

Calling Pl/sql function in shell script to modify csv

I need to 1.Open a csv 2.Process the csv i.e. Modify 2 column in the csv. To modify the column the value needs to be passed to a pl/sql function and the return value should be updated For eg: If column 2 E,then E will be passed in database function which will return Employee. 3. Write a... (5 Replies)
Discussion started by: Chinky23
5 Replies

4. Shell Programming and Scripting

Perl script to modify csv file

Hi Friends, I want to convert a csv file into a ordinary .txt file. I am able to convert but I want the output to look as shown below in the .txt file table findhost= { {"xyz","abc"}, {"rxz","mmz"}, {"vrr","nnz"}, } default={"NONE"} My current perl script #!/usr/bin/env perl... (12 Replies)
Discussion started by: dbashyam
12 Replies

5. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

6. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

7. Shell Programming and Scripting

Save output of updated csv file as csv file itself

Hi, all I want to sort a csv file based on timestamp from oldest to newest and save the output as csv file itself. Here is an example of my csv file. test.csv SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0739.JPG,2015:02:17 11:32:21 /home/intannf/foto/IMG_0749.JPG,2015:02:17 11:37:28... (10 Replies)
Discussion started by: refrain
10 Replies

8. Shell Programming and Scripting

Save output of updated csv file as csv file itself, part 2

Hi, I have another problem. I want to sort another csv file by the first field. result.csv SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw /home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 ... (2 Replies)
Discussion started by: refrain
2 Replies

9. Shell Programming and Scripting

Modify csv-files with awk

Hello everyone! I have thousands of csv files I have to import into a Database table. As usually the files aren't perfect. For example they have a different number of columns and some weird columns. The second problem is, that I have to add 3 parts of the filename into 3 rows in the... (6 Replies)
Discussion started by: elRonaldo
6 Replies
Handle Management(3)						 globus ftp client					      Handle Management(3)

NAME
Handle Management - Typedefs typedef struct globus_i_ftp_client_handle_t * globus_ftp_client_handle_t" Initialize globus_result_t globus_ftp_client_handle_init (globus_ftp_client_handle_t *handle, globus_ftp_client_handleattr_t *attr) Destroy globus_result_t globus_ftp_client_handle_destroy (globus_ftp_client_handle_t *handle) URL Caching globus_result_t globus_ftp_client_handle_cache_url_state (globus_ftp_client_handle_t *handle, const char *url) globus_result_t globus_ftp_client_handle_flush_url_state (globus_ftp_client_handle_t *handle, const char *url) User Pointer globus_result_t globus_ftp_client_handle_set_user_pointer (globus_ftp_client_handle_t *handle, void *user_pointer) globus_result_t globus_ftp_client_handle_get_user_pointer (const globus_ftp_client_handle_t *handle, void **user_pointer) Plugins globus_result_t globus_ftp_client_handle_add_plugin (globus_ftp_client_handle_t *handle, globus_ftp_client_plugin_t *plugin) globus_result_t globus_ftp_client_handle_remove_plugin (globus_ftp_client_handle_t *handle, globus_ftp_client_plugin_t *plugin) Detailed Description Create/Destroy/Modify an FTP Client Handle. Within the Globus FTP Client Library, all FTP operations require a handle parameter. Currently, only one FTP operation may be in progress at once per FTP handle. FTP connections may be cached between FTP operations, for improved performance. This section defines operations to create and destroy FTP Client handles, as well as to modify handles' connection caches. Typedef Documentation typedef struct globus_i_ftp_client_handle_t* globus_ftp_client_handle_t FTP Client Handle. An FTP client handle is used to associate state with a group of operations. Handles can have attributes associated with them. All FTP operations take a handle pointer as a parameter. See also: globus_ftp_client_handle_init(), globus_ftp_client_handle_destroy(), globus_ftp_client_handleattr_t Function Documentation globus_result_t globus_ftp_client_handle_init (globus_ftp_client_handle_t *handle, globus_ftp_client_handleattr_t *attr) Initialize a client FTP handle. Initialize an FTP handle which can be used in subsequent get, put, or transfer requests. A handle may have at most one get, put, or third-party transfer in progress. Parameters: handle The handle to be initialized. attr Initial attributes to be used to create this handle. See also: globus_ftp_client_handle_destroy() globus_result_t globus_ftp_client_handle_destroy (globus_ftp_client_handle_t *handle) Destroy a client FTP handle. A FTP client handle may not be destroyed if a get, put, or third-party transfer is in progress. Parameters: handle The handle to be destroyed. See also: globus_ftp_client_handle_init() globus_result_t globus_ftp_client_handle_cache_url_state (globus_ftp_client_handle_t *handle, const char *url) Cache connections to an FTP server. Explicitly cache connections to URL server in an FTP handle. When an URL is cached, the client library will not close the connection to the URL server after a file transfer completes. Parameters: handle Handle which will contain a cached connection to the URL server. url The URL of the FTP or GSIFTP server to cache. See also: globus_ftp_client_flush_url_state() globus_result_t globus_ftp_client_handle_flush_url_state (globus_ftp_client_handle_t *handle, const char *url) Remove a cached connection from the FTP client handle. Explicitly remove a cached connection to an FTP server from the FTP handle. If an idle connection to an FTP server exists, it will be closed. Parameters: handle Handle which will contain a cached connection to the URL server. url The URL of the FTP or GSIFTP server to cache. globus_result_t globus_ftp_client_handle_set_user_pointer (globus_ftp_client_handle_t *handle, void *user_pointer) Set/Get the user pointer field from an ftp client handle. The user pointer is provided to all the user of the FTP client library to assocate a pointer to any application-specific data to an FTP client handle. This pointer is never internally used by the client library. Parameters: handle The FTP client handle to set or query. user_pointer The value of the user pointer field. Note: Access to the user_pointer are not synchronized, the user must take care to make sure that multiple threads are not modifying it's value. globus_result_t globus_ftp_client_handle_add_plugin (globus_ftp_client_handle_t *handle, globus_ftp_client_plugin_t *plugin) Add a plugin to an FTP client handle. This function adds a plugin to an FTP client handle after it has been created. Plugins may be added to an ftp client handle whenever an operation is not in progress. The plugin will be appended to the list of plugins present in the handle, and will be invoked during any subsequent operations processed with this handle. Only one instance of a particular plugin may be added to a particular handle. Plugins may be removed from a handle by calling globus_ftp_client_remove_plugin(). Parameters: handle The FTP client handle to set or query. plugin A pointer to the plugin structure to add to this handle. See also: globus_ftp_client_remove_plugin(), globus_ftp_clent_handleattr_add_plugin(), globus_ftp_clent_handleattr_remove_plugin() globus_result_t globus_ftp_client_handle_remove_plugin (globus_ftp_client_handle_t *handle, globus_ftp_client_plugin_t *plugin) Remove a plugin to an FTP client handle. This function removes a plugin from an FTP client handle after it has been created. Plugins may be removed from an ftp client handle whenever an operation is not in progress. The plugin will be removed from the list of plugins, and will not be used during any subsequent operations processed with this handle. This function can remove plugins which were added at handle initialization time or by calling globus_ftp_client_handle_add_plugin(). Parameters: handle The FTP client handle to set or query. plugin A pointer to the plugin structure to remove from this handle. See also: globus_ftp_client_add_plugin(), globus_ftp_clent_handleattr_add_plugin(), globus_ftp_clent_handleattr_remove_plugin() globus_result_t globus_ftp_client_handle_get_user_pointer (const globus_ftp_client_handle_t *handle, void **user_pointer) Set/Get the user pointer field from an ftp client handle. The user pointer is provided to all the user of the FTP client library to assocate a pointer to any application-specific data to an FTP client handle. This pointer is never internally used by the client library. Parameters: handle The FTP client handle to set or query. user_pointer The value of the user pointer field. Note: Access to the user_pointer are not synchronized, the user must take care to make sure that multiple threads are not modifying it's value. Author Generated automatically by Doxygen for globus ftp client from the source code. Version 7.3 Mon Apr 30 2012 Handle Management(3)
All times are GMT -4. The time now is 11:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy