Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Work with huge Zipped files
Top Forums Shell Programming and Scripting Work with huge Zipped files Post 302872273 by Homa on Friday 8th of November 2013 04:57:52 AM
Old 11-08-2013
Ubuntu Work with huge Zipped files
Hello dear members,

I have one general and one specific question which I will be very grateful if you could help me with them. Let's start with my general question:

1. I am working on cluster computer shared with other people and I need to manipulate a big zipped text file of 13 GB. There is no possibility of unzipping it, if I could do that, I would have any problem but my problem is that working with zipped files using Awk or Linux commands become very difficult. I would be very grateful if you could give me some directions in this regard.

2. The second question is that being in that 13 GB zipped, I want to select one part of the file to make an example file to write my code to be able to later apply them to the whole file. When I display the file on shell by the command:

Code:
zcat zippedfile | less

and I copy a section just by clicking and I paste it in a text editor, the spacing, number of fields, everything gets destroyed. How could I select a part of the file by command line without unzipping it?

Thank you
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

uploading Zipped files get 553 error

I am trying to upload .zip files to Unix server and get the error 553 qmerev2002.zip: Permission denied, what is my problem?? I am able to load other files and folders fine. (3 Replies)
Discussion started by: CoastGuard1970
3 Replies

2. Shell Programming and Scripting

Greping from zipped files without unzipping

I have more than 500 gzipped files in a directory. I have one lookup file in the same directory with 200 key values. I need to get the name of the gzipped file which have any of these 200 key values. Here my criteria is do not unzip the files due to space constraint. Any suggestion? (3 Replies)
Discussion started by: kanu_kanu
3 Replies

3. Linux

grep thru zipped files

Hi All, I need to check the logs and grep it. The problem is that the previous days are zipped with *.gz. How do grep on the *.gz? Thank you in advance. (1 Reply)
Discussion started by: itik
1 Replies

4. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

5. Shell Programming and Scripting

Difference between 2 zipped text files.

Hi, I have below two zipped files - file1.gz 023384148,1,,,02077301961,R,02077301961,N,0,02077301961,N,0,0,8010,02077300518,U,N,,02077300518,U,20100501011732,20100501011732,0,20100501011815,00000430,16,16,10,N;... (8 Replies)
Discussion started by: ravigupta2u
8 Replies

6. Shell Programming and Scripting

Copy all zipped files from one folder to another

Hi everyone, when I try to copy *.gz files run cp within the correct source folder it works as follow: Source folder = C:/Documents and Settings/user/Recent papers/2771/ Destination folder = C:/Documents and Settings/user/My documents/1532/temp cp *.gz "C:/Documents and Settings/user/My... (2 Replies)
Discussion started by: cgkmal
2 Replies

7. Shell Programming and Scripting

How to parse a huge 600MB zipped file?

I'm new to Unix, trying to parse a huge 600MB zipped file... I need to bzcat this file once and do some calculations (word count) on the lines based on certain criteria (see script) the correct result/output should be: column1=6 column2=4 the problem is that I'm getting column2=0 (see... (16 Replies)
Discussion started by: DeltaComp
16 Replies

8. Shell Programming and Scripting

Renaming all files inside a zipped file

Hi, To all the Unix gurus this should be a simple task, but as a newbie I'm finding it hard to crack this. Any help is highly appreciated... Scenario: Step 1 : Move zip file from FTP folder to WORK folder Step 2: Unzip the file "Sample_YYYYMMDDHHMMSS.tar.gz" which contains many file... (10 Replies)
Discussion started by: asandy1234
10 Replies

9. Shell Programming and Scripting

Grabing the same timestamp from files that are ZIPPED

Hi, I am zipping more than 20 files that has same timestamp in all of them. I need to create the zip file with the same timestamp as in the files that are zipped. So I have files: Dummytest_20140601W110515_file1.txt Dummytest_20140601W110515_file2.txt ....... .......... (5 Replies)
Discussion started by: Saanvi1
5 Replies

10. Shell Programming and Scripting

awk does not work well with huge data?

Dear all , I found that if we work with thousands line of data, awk does not work perfectly. It will cut hundreds line (others are deleted) and works only on the remain data. I used this command : awk '$1==1{$1="Si"}{print>FILENAME}' coba.xyz to change value of first column whose value is 1... (4 Replies)
Discussion started by: ariesto
4 Replies

LEARN ABOUT DEBIAN

gpe-question

GPE-QUESTION(1) 						   gpe-question 						   GPE-QUESTION(1)

NAME

       gpe-question - GPE interface for asking questions from shell scripts.

SYNOPSIS

       gpe-question [---icon] name {--question "Question text"} {--buttons icon1:text1} [icon2:text2 ...]

DESCRIPTION

       This manual page was written for the Debian(TM) distribution because the original program does not have a manual page.

       gpe-question provides dialogues to raise a question within the GPE Palmtop Environment with cancel and ok buttons.

OPTIONS

       --question STRING
	  The question to ask the user via the dialogue.

       --buttons type:label
	  The type and label of the button(s) to be shown. Repeat for additional buttons.

EXAMPLES

       gpe-question "test" --question "Do you want to check first?"  --buttons YES:yes NO:no

AUTHOR

       This manual page was written by Neil Williams <codehelp@debian.org> for the Debian(TM) system (but may be used by others). Permission is
       granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 any later version
       published by the Free Software Foundation.

       On Debian systems, the complete text of the GNU General Public License can be found in /usr/share/common-licenses/GPL.

AUTHOR

       Moray Allen
	  Author.

COPYRIGHT

       Copyright (C) 2007 Neil Williams

Debian 1							 January 16, 2007						   GPE-QUESTION(1)
All times are GMT -4. The time now is 06:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy