Sponsored Content
Top Forums Shell Programming and Scripting Help Manipulating Large Csv File Post 302450111 by xxwohxx on Wednesday 1st of September 2010 04:52:00 PM
Old 09-01-2010
Sure, Here is a sample of the dataset.

Image
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

manipulating csv to add extra row

hi how do i manipulate .csv file to add an extra row after each row using shell script? I need a blank line added for each 1000 records in my file? I will then need to copy and paste some data in the blank row created. thanks 4 ur support neil (3 Replies)
Discussion started by: neil546
3 Replies

2. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies

3. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

4. Shell Programming and Scripting

Manipulating csv file

We need to convert a field in a csv file which is in cents to dollars.(divide by 100) in our shell script. Can some body help me? (3 Replies)
Discussion started by: Deepthz
3 Replies

5. SCO

Need advice: Copying large CSV report files off SCO system

I have a SCO Unix server from 1999 running SCO 5.0.5 and some ancient accounting software called Real World A report writer program on the system is used to generate CSV files from accounting that we write with DOSCOPY commands to 3.5" floppies In the next 60 days we will be decommissioning... (11 Replies)
Discussion started by: magnetman
11 Replies

6. Shell Programming and Scripting

Merging Very large CSV files in Unix

Hi, I have two very large CSV files, which I want to merge (equi-join) based on a key (column). One of the file (say F1) would have ~30 MM records and 700 columns. The other file (~f2) would have same # of records and lesser columns (say 50). I want to create an output file joining on a... (3 Replies)
Discussion started by: student_007
3 Replies

7. Shell Programming and Scripting

Comparing two large unsorted csv files

Hi All, My requirement is to write a shell script to compare two large csv files. I've created sample files for explaining my problem i.e., a.csv and b.csv contents of files: ----------------- a.csv ------ Type,Memory (Kb),Location HD,Size (Mb),Serial # XT,640,D402,0,MG0010... (2 Replies)
Discussion started by: vasavi
2 Replies

8. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

9. Shell Programming and Scripting

Output large volume of data to CSV file

I have a program that output the ownership and permission on each directory and file on the server to a csv file. I am getting error message when I run the program. The program is not outputting to the csv file. Error: the file access permissions do not allow the specified action cannot... (2 Replies)
Discussion started by: dellanicholson
2 Replies
lfcompile64(5)                                          Standards, Environments, and Macros                                         lfcompile64(5)

NAME
lfcompile64 - transitional compilation environment DESCRIPTION
All 64-bit applications can manipulate large files by default. The transitional interfaces described on this page can be used by 32-bit and 64-bit applications to manipulate large files. In the transitional compilation environment, explicit 64-bit functions, structures, and types are added to the API. Compiling in this environment allows both 32-bit and 64-bit applications to access files whose size is greater than or equal to 2 Gbyte ( 2**31 bytes). The transitional compilation environment exports all the explicit 64-bit functions (xxx64()) and types in addition to all the regular func- tions (xxx()) and types. Both xxx() and xxx64() functions are available to the program source. A 32-bit application must use the xxx64() functions in order to access large files. See the lf64(5) manual page for a complete listing of the 64-bit transitional interfaces. The transitional compilation environment differs from the large file compilation environment, wherein the underlying interfaces are bound to 64-bit functions, structures, and types. An application compiled in the large file compilation environment is able to use the xxx() source interfaces to access both large and small files, rather than having to explicitly utilize the transitional xxx64() interface calls to access large files. See the lfcompile(5) manual page for more information regarding the large file compilation environment. Applications may combine objects produced in the large file compilation environment with objects produced in the transitional compilation environment, but must be careful with respect to interoperability between those objects. Applications should not declare global variables of types whose sizes change between compilation environments. For applications that do not wish to conform to the POSIX or X/Open specifications, the 64-bit transitional interfaces are available by default. No compile-time flags need to be set. Access to Additional Large File Interfaces Applications that wish to access the transitional interfaces as well as the POSIX or X/Open specification-conforming interfaces should use the following compilation methods and set whichever feature test macros are appropriate to obtain the desired environment (see stan- dards(5)). o Set the compile-time flag _LARGEFILE64_SOURCE to 1 before including any headers. o Use the getconf(1) command with one or more of the following arguments: +------------------+----------------------------------------+ | argument | purpose | |LFS64_CFLAGS | obtain compilation flags necessary to | | | enable the transitional compilation | | | environment | |LFS64_LDFLAGS | obtain link editor options | |LFS64_LIBS | obtain link library names | |LFS64_LINTFLAGS | obtain lint options | +------------------+----------------------------------------+ EXAMPLES
In the following examples, the transitional compilation environment is accessed by invoking the getconf utility with one of the arguments listed in the table above. The additional large file interfaces are accessed either by specifying -D_LARGEFILE64_SOURCE or by invoking the getconf utility with the arguments listed above. The example that uses the form of command substitution specifying the command within parentheses preceded by a dollar sign can be executed only in a POSIX-conforming shell such as the Korn Shell (see ksh(1)). In a shell that is not POSIX-conforming, such as the Bourne Shell (see sh(1)) and the C Shell (see csh(1)), the command must be enclosed within grave accent marks. Example 1: An example of compiling a program using transitional interfaces such as lseek64() and fopen64(): $ c89 -D_LARGEFILE64_SOURCE $(getconf LFS64_CFLAGS) a.c $(getconf LFS64_LDFLAGS) $(getconf LFS64_LIBS) Example 2: An example of running lint on a program using transitional interfaces: % lint -D_LARGEFILE64_SOURCE `getconf LFS64_LINTFLAGS` ... `getconf LFS64_LIBS` SEE ALSO
getconf(1), lseek(2), fopen(3C), lf64(5), standards(5) SunOS 5.10 26 Jan 1998 lfcompile64(5)
All times are GMT -4. The time now is 02:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy