Sponsored Content
Top Forums Shell Programming and Scripting Output large volume of data to CSV file Post 302956675 by dellanicholson on Thursday 1st of October 2015 11:45:42 PM
Old 10-02-2015
Output large volume of data to CSV file

I have a program that output the ownership and permission on each directory and file on the server to a csv file. I am getting error message
when I run the program. The program is not outputting to the csv file.

Code:
Error:
the file access permissions do not allow the specified action
cannot change directory
xrealloc: cannot allocate 9900416 bytes

Here is my code
Code:
#!/bin/bash
monitorf=/sc/sb/monitor.csv
ls -l $(find /sasem/* -type f) > "$monitorf"
mail -a /sc/sb/monitor.csv -s "monitor" jgk@yahoo.com; billygm@yahoo.com


Last edited by Don Cragun; 10-02-2015 at 05:51 AM.. Reason: Add missing CODE tags.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Large volume file formatting

Hi, I have a file which is around 193 gb in size. This file has tonnes of spaces and I need to sanitize it. I tried to use awk script to split this file but it gave me an error like line to long... As of now I am using a sed command to search replace the spaces; however its too slow for such a... (2 Replies)
Discussion started by: darshanw
2 Replies

2. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

3. Shell Programming and Scripting

select data from oracle table and save the output as csv file

Hi I need to execute a select statement in a solaris environment with oracle database. The select statement returns number of rows of data. I need the data to be inserted into a CSV file with proper format. For that we normally use "You have to select all your columns as one big string,... (2 Replies)
Discussion started by: rdhanek
2 Replies

4. Shell Programming and Scripting

Help Manipulating Large Csv File

Hello everyone, I am trying to manipulate a large .csv file where I have output similar to the following - http://imgur.com/TEXD8.png The result that I am looking for would be to consolidate the first column, but combine the second and third column so it still relates to the first. I... (8 Replies)
Discussion started by: xxwohxx
8 Replies

5. Shell Programming and Scripting

Severe performance issue while 'grep'ing on large volume of data

Background ------------- The Unix flavor can be any amongst Solaris, AIX, HP-UX and Linux. I have below 2 flat files. File-1 ------ Contains 50,000 rows with 2 fields in each row, separated by pipe. Row structure is like Object_Id|Object_Name, as following: 111|XXX 222|YYY 333|ZZZ ... (6 Replies)
Discussion started by: Souvik
6 Replies

6. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

7. Shell Programming and Scripting

Save output of updated csv file as csv file itself, part 2

Hi, I have another problem. I want to sort another csv file by the first field. result.csv SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw /home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 ... (2 Replies)
Discussion started by: refrain
2 Replies

8. Shell Programming and Scripting

Copy data to CSV file from txt output

Hi Everyone , Below is output from a TL1 , I want just the NE Name: and beside that the Temperature and the voltages in a csv file , Is this possible? > act-user:AB1S2a:ArshadFO:493::**********; AB1S2a 2016-02-07 10:13:24 M 493 COMPLD "ArshadFO:2016-02-07 10-04-55,0" ;... (11 Replies)
Discussion started by: adgjmpt
11 Replies

9. Shell Programming and Scripting

Need a script to parse data and output to csv

I am not too savvy with arrays and am assuming that what I am looking for needs arrays. This is my requirement. So I have the raw data that gets updated to a log as shown below StudentInfo: FullInfo = { Address = Newark Age = 20 Name= John } StudentInfo:... (2 Replies)
Discussion started by: sidnow
2 Replies

10. Programming

Python script to run multiple command and append data in output csv file

Experts, I am writing a script and able to write only small piece of code and not able to collect logic to complete this task. In input file have to look for name like like this (BGL_HSR_901_1AG_A_CR9KTR10) before sh iss neors. Record this (BGL_HSR_901_1AG_A_CR9KTR10) in csv file Now have to... (0 Replies)
Discussion started by: as7951
0 Replies
ATF-REPORT(1)						    BSD General Commands Manual 					     ATF-REPORT(1)

NAME
atf-report -- transforms the output of atf-run to different formats SYNOPSIS
atf-report [-o fmt1:path1 [.. -o fmtN:pathN]] atf-report -h DESCRIPTION
atf-report reads the output of atf-run and transforms it to different formats. Some of these are user-friendly and others are machine- parseable, which opens a wide range of possibilities to analyze the results of a test suite's execution. See Output formats below for more details on which these formats are. In the first synopsis form, atf-report reads the output of atf-run through its standard input and, if no -o options are given, prints a user- friendly report on its standard output using the 'ticker' format. If -o options are provided (more than one are allowed), they specify the complete list of reports to generate. They are all generated simultaneously, and for obvious reasons, two reports cannot be written to the same file. Note that the default output is suppressed when -o is provided. In the second synopsis form, atf-report will print information about all supported options and their purpose. The following options are available: -h Shows a short summary of all available options and their purpose. -o fmt:path Adds a new output format. fmt is one of the formats described later on in Output formats. path specifies where the report will be written to. Depending on the chosen format, this may refer to a single file or to a directory. For those formats that write to a single file, specifying a '-' as the path will redirect the report to the standard output. Output formats The following output formats are allowed: csv A machine-parseable Comma-Separated Values (CSV) file. This file contains the results for all test cases and test programs. Test cases are logged using the following syntax: tc, duration, test-program, test-case, result[, reason] The 'result' field for test cases is always one of 'passed', 'skipped' or 'failed'. The last two are always followed by a reason. Test programs are logged with the following syntax: tp, duration, test-program, result[, reason] In this case, the 'result' can be one of: 'passed', which denotes test programs that ran without any failure; 'failed', which refers to test programs in which one or more test cases failed; or 'bogus', which mentions those test programs that failed to exe- cute by some reason. The reason field is only available in the last case. The time required to execute each test case and test program is also provided. You should not rely on the order of the entries in the resulting output. ticker A user-friendly report that shows the progress of the test suite's execution as it operates. This type of report should always be redirected to a virtual terminal, not a file, as it may use control sequences that will make the output unreadable in regular files. xml A report contained in a single XML file. Ideal for later processing with xsltproc(1) to generate nice HTML reports. EXAMPLES
The most simple way of running a test suite is to pipe the output of atf-run through atf-report without any additional flags. This will use the default output format, which is suitable to most users: atf-run | atf-report In some situations, it may be interesting to get a machine-parseable file aside from the standard report. This can be done as follows: atf-run | atf-report -o csv:testsuite.csv -o ticker:- Or if the standard report is not desired, thus achieving completely silent operation: atf-run | atf-report -o csv:testsuite.csv SEE ALSO
atf-run(1), atf(7) BSD
December 16, 2011 BSD
All times are GMT -4. The time now is 03:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy