Sponsored Content
Top Forums Shell Programming and Scripting Using -text command and creating multiple files Post 302974785 by stomp on Friday 3rd of June 2016 08:24:49 AM
Old 06-03-2016
...and put the split at the end of your pipe, so you do not need to write an intermediate 350 GB file(Will be a lot of faster because it saves you to write+read 350 GB).

Code:
hadoop bla ... |  hadoop fs -put - - | split ... outputfile

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

grep multiple text files in folder into 1 text file?

How do I use the grep command to take mutiple text files in a folder and make one huge text file out of them. I'm using Mac OS X and can not find a text tool that does it so I figured I'd resort to the BSD Unix CLI for a solution... there are 5,300 files that I want to write to one huge file so... (7 Replies)
Discussion started by: coppertone
7 Replies

2. Shell Programming and Scripting

Dynamically creating text files using shell script

Hi All, I want to create a shell script which dynamically create text files. i am using the following script $i=1 while do cat > test_$i.txt done but while running the script it was stopping(the cursor not going to next step, i have to enter ctrl+c to make it stop). it is creating only... (2 Replies)
Discussion started by: KiranKumarKarre
2 Replies

3. Shell Programming and Scripting

Replacing text from multiple files at multiple location

Hi, I have many files scattered in all different folders. I want to replace the text within all the files using a single command ( awk, sed...) Is it possible? example find all the files in which there is text "memory" and replace it with "branded_memories". the files can be at the... (2 Replies)
Discussion started by: rudoraj
2 Replies

4. Shell Programming and Scripting

creating a multiple zip files

Hi, Please help me, i need a single line command for te below steps 1: listing a directory 2: searching a particular pattern in the names and creating the zip files of each eg : ls -lrt | grep jil | awk '{print $9}' output is kap.jil pranabjil pranabjil1 pranabjil2... (1 Reply)
Discussion started by: pranabrana
1 Replies

5. UNIX for Dummies Questions & Answers

Grep multiple strings in multiple files using single command

Hi, I will use below command for grep single string ("osuser" is search string) ex: find . -type f | xarg grep -il osuser but i have one more string "v$session" here i want to grep in which file these two strings are present. any help is appreciated, Thanks in advance. Gagan (2 Replies)
Discussion started by: gagan4599
2 Replies

6. UNIX for Dummies Questions & Answers

Changing text in multiple files, but with different text for each file

Hello, I have a situation where I want to change a line of text in multiple files, but the problem is that I want to change the text to something unique for each file. For example, let's say I have five files named bob.txt, joe.txt, john.txt, tom.txt, and zach.txt. Each of these files has a... (5 Replies)
Discussion started by: Scatterbrain26
5 Replies

7. Shell Programming and Scripting

Creating single pattern for matching multiple files.

Hi friends, I have a some files in a directory. for example 856-abc 856-def 851-abc 945-def 956-abc 852-abc i want to display only those files whose name starts with 856* 945* and 851* using a single pattern. i.e 856-abc 856-def 851-abc 945-def the rest of the two files... (2 Replies)
Discussion started by: Little
2 Replies

8. Shell Programming and Scripting

Split a text file into multiple text files?

I have a text file with entries like 1186 5556 90844 7873 7722 12 7890.6 78.52 6679 3455 9867 1127 5642 ..N so many records like this. I want to split this file into multiple files like cluster1.txt, cluster2.txt, cluster3.txt, ..... clusterN.txt. (4 Replies)
Discussion started by: sammy777
4 Replies

9. Shell Programming and Scripting

Creating Multiple data files with spaces in the filename

Hi, I have a list of filenames in the format with Spaces in the filename. As an example : Sample File 1.txt Sample File 2.txt Sample File 3.txt.....I have about 100 files like this. I am trying to create a block of code or use an available command to a) Create a file b) Put in some... (2 Replies)
Discussion started by: ban3rj33
2 Replies

10. UNIX for Beginners Questions & Answers

Creating multiple empty files with touch

Hello, sorry to bother anyone reading this I have an assignment with a question that reads: Your current directory is stenton. Create empty files called f1, f2, and f12 (in that order), within stenton So my first thought was to enter: touch f1 f2 f12 but that does not work, does anyone... (1 Reply)
Discussion started by: eleuin
1 Replies
DH_COMPRESS(1)							     Debhelper							    DH_COMPRESS(1)

NAME
dh_compress - compress files and fix symlinks in package build directories SYNOPSIS
dh_compress [debhelperoptions] [-Xitem] [-A] [file...] DESCRIPTION
dh_compress is a debhelper program that is responsible for compressing the files in package build directories, and makes sure that any symlinks that pointed to the files before they were compressed are updated to point to the new files. By default, dh_compress compresses files that Debian policy mandates should be compressed, namely all files in usr/share/info, usr/share/man, files in usr/share/doc that are larger than 4k in size, (except the copyright file, .html and other web files, image files, and files that appear to be already compressed based on their extensions), and all changelog files. Plus PCF fonts underneath usr/share/fonts/X11/ FILES
debian/package.compress These files are deprecated. If this file exists, the default files are not compressed. Instead, the file is ran as a shell script, and all filenames that the shell script outputs will be compressed. The shell script will be run from inside the package build directory. Note though that using -X is a much better idea in general; you should only use a debian/package.compress file if you really need to. OPTIONS
-Xitem, --exclude=item Exclude files that contain item anywhere in their filename from being compressed. For example, -X.tiff will exclude TIFF files from compression. You may use this option multiple times to build up a list of things to exclude. -A, --all Compress all files specified by command line parameters in ALL packages acted on. file ... Add these files to the list of files to compress. CONFORMS TO
Debian policy, version 3.0 SEE ALSO
debhelper(7) This program is a part of debhelper. AUTHOR
Joey Hess <joeyh@debian.org> 11.1.6ubuntu2 2018-05-10 DH_COMPRESS(1)
All times are GMT -4. The time now is 01:24 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy