Sponsored Content
Top Forums Shell Programming and Scripting How to parse a huge 600MB zipped file? Post 302668717 by DeltaComp on Tuesday 10th of July 2012 12:50:39 AM
Old 07-10-2012
pikk45, thanks for the clarification... I know that I could do bzcat twice...
but this way, I'm unzipping the source file twice which's consuming alot of server's resources (hight CPU utilization) on a prod server....
in my original request I mentioned that "I need to bzcat this file once and do some calculations" as it's taking over 20min for each operation
any better solution would be much appreciated.
I believe there must be a way to do it
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

zipped or unzipped file

Is there a way you can tell if a file is still zipped or it's unzipped I have a file called ssss.zip and I would like to know if this file is still zipped or if it's unzipped? I'm on IBM AIX/RS6000 (3 Replies)
Discussion started by: ted
3 Replies

2. UNIX for Dummies Questions & Answers

sendind a zipped file via email

Hi, I was not sure if I can do this. Suppose I have a file under /tmp Suppose the file is called any_11_52.txt Fisrt QUESTION??? If I zip this file using gzip will the user be able to unzip it , if I send it as an attachment in an email. Secondly is there a command by which we can... (2 Replies)
Discussion started by: rooh
2 Replies

3. Shell Programming and Scripting

How to search a pattern inside a zipped file ie (.gz file) with out unzipping it

How to search a pattern inside a zipped file ie (.gz file) with out unzipping it? using grep command.. Bit urgent.. pls..help me (2 Replies)
Discussion started by: senraj01
2 Replies

4. UNIX for Dummies Questions & Answers

how to check if file is zipped

I have a script that grabs files from directory , zips and moves them somewhere else. It works fine except the case when files it grabs are already zipped. Then it trys to zip it again which does not make sence. How can I check before zipping if file is already zipped? thanks in advance (3 Replies)
Discussion started by: arushunter
3 Replies

5. UNIX for Dummies Questions & Answers

reading a zipped file without unzipping it?

Dear all, I would like to ask how i can read a zipped file (file.gz) without actually unzipping it? i think there is a way to do so but i can't remember it.. can anyone help? thanks in advance.. (1 Reply)
Discussion started by: marwan
1 Replies

6. UNIX for Dummies Questions & Answers

Zipped tar file is corrupt

Hello, I am currently dumping 30-40 reports on a Unix folder located here /home/apps/reports/prode/excel I use K-shell to do this task. In that, I use the gzip command to compress these files. I want to be able to use a tar command to first load the entire directory into one file then gzip that... (2 Replies)
Discussion started by: Pramodini Rode
2 Replies

7. Solaris

How can I tell if a file is zipped or not?

SunOS xxxxxx 5.10 Generic_142900-15 sun4v sparc SUNW,T5240 We receive files that are sometimes zipped, but the file may not have the .gz or other extention that would indicated that the file is zipped. Is there a unix "test" command that I could use or something similar? Thanks in advance (7 Replies)
Discussion started by: Harleyrci
7 Replies

8. Shell Programming and Scripting

FTP'ing the zipped file

Hi, I need to have a shell script that FTP's a zipped file from a particular location. I have some path and inside that path i will have folders like x_timestamp and inside x_timestamp there may many folders based upon events like y_111,y_222,y_333.Inside each event there will be another... (3 Replies)
Discussion started by: weknowd
3 Replies

9. Shell Programming and Scripting

awk to parse huge files

Hello All, I have a situation as below: (1) Read a source file (a single file of 1.2 million rows in it ) (2) Read Destination files one by one and replace the content ( few fields in it ) with the corresponding matching field from source file. I tried as below: ( please note I am not... (4 Replies)
Discussion started by: panyam
4 Replies

10. Shell Programming and Scripting

Work with huge Zipped files

Hello dear members, I have one general and one specific question which I will be very grateful if you could help me with them. Let's start with my general question: 1. I am working on cluster computer shared with other people and I need to manipulate a big zipped text file of 13 GB. There is... (1 Reply)
Discussion started by: Homa
1 Replies
MPSTAT(1)							Linux User's Manual							 MPSTAT(1)

NAME
mpstat - Report processors related statistics. SYNOPSIS
mpstat [ -A ] [ -I { SUM | CPU | SCPU | ALL } ] [ -u ] [ -P { cpu [,...] | ON | ALL } ] [ -V ] [ interval [ count ] ] DESCRIPTION
The mpstat command writes to standard output activities for each available processor, processor 0 being the first one. Global average activities among all processors are also reported. The mpstat command can be used both on SMP and UP machines, but in the latter, only global average activities will be printed. If no activity has been selected, then the default report is the CPU utilization report. The interval parameter specifies the amount of time in seconds between each report. A value of 0 (or no parameters at all) indicates that processors statistics are to be reported for the time since system startup (boot). The count parameter can be specified in conjunction with the interval parameter if this one is not set to zero. The value of count determines the number of reports generated at interval sec- onds apart. If the interval parameter is specified without the count parameter, the mpstat command generates reports continuously. OPTIONS
-A This option is equivalent to specifying -I ALL -u -P ALL -I { SUM | CPU | SCPU | ALL } Report interrupts statistics. With the SUM keyword, the mpstat command reports the total number of interrupts per processor. The following values are displayed: CPU Processor number. The keyword all indicates that statistics are calculated as averages among all processors. intr/s Show the total number of interrupts received per second by the CPU or CPUs. With the CPU keyword, the number of each individual interrupt received per second by the CPU or CPUs is displayed. With the SCPU keyword, the number of each individual software interrupt received per second by the CPU or CPUs is displayed. This option works only with kernels 2.6.31 and later. The ALL keyword is equivalent to specifying all the keywords above and therefore all the interrupts statistics are displayed. -P { cpu [,...] | ON | ALL } Indicate the processor number for which statistics are to be reported. cpu is the processor number. Note that processor 0 is the first processor. The ON keyword indicates that statistics are to be reported for every online processor, whereas the ALL keyword indicates that statistics are to be reported for all processors. -u Report CPU utilization. The following values are displayed: CPU Processor number. The keyword all indicates that statistics are calculated as averages among all processors. %usr Show the percentage of CPU utilization that occurred while executing at the user level (application). %nice Show the percentage of CPU utilization that occurred while executing at the user level with nice priority. %sys Show the percentage of CPU utilization that occurred while executing at the system level (kernel). Note that this does not include time spent servicing hardware and software interrupts. %iowait Show the percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request. %irq Show the percentage of time spent by the CPU or CPUs to service hardware interrupts. %soft Show the percentage of time spent by the CPU or CPUs to service software interrupts. %steal Show the percentage of time spent in involuntary wait by the virtual CPU or CPUs while the hypervisor was servicing another virtual processor. %guest Show the percentage of time spent by the CPU or CPUs to run a virtual processor. %idle Show the percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request. Note: On SMP machines a processor that does not have any activity at all is a disabled (offline) processor. -V Print version number then exit. ENVIRONMENT
The mpstat command takes into account the following environment variable: S_TIME_FORMAT If this variable exists and its value is ISO then the current locale will be ignored when printing the date in the report header. The mpstat command will use the ISO 8601 format (YYYY-MM-DD) instead. EXAMPLES
mpstat 2 5 Display five reports of global statistics among all processors at two second intervals. mpstat -P ALL 2 5 Display five reports of statistics for all processors at two second intervals. BUGS
/proc filesystem must be mounted for the mpstat command to work. Only a few activities are given by the Linux kernel for each processor. FILES
/proc contains various files with system statistics. AUTHOR
Sebastien Godard (sysstat <at> orange.fr) SEE ALSO
sar(1), pidstat(1), iostat(1), vmstat(8) http://pagesperso-orange.fr/sebastien.godard/ Linux AUGUST 2011 MPSTAT(1)
All times are GMT -4. The time now is 03:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy