Script to sort large file with frequency Post: 302663315

10 More Discussions You Might Find Interesting

1. HP-UX

Need to split a large data file using a Unix script

Greetings all: I am still new to Unix environment and I need help with the following requirement. I have a large sequential file sorted on a field (say store#) that is being split into several smaller files, one for each store. That means if there are 500 stores, there will be 500 files. This...

2. Shell Programming and Scripting

script to splite large file to number of small files

Dear All, Could you please help me to split a file contain around 240,000,000 line to 4 files all equally likely , note that we need to maintain that the end of each file should started by start flage (MSISDN) and ended by end flag (End), also the number of the line between the...

3. UNIX for Dummies Questions & Answers

Sort large file

I was wondering how sort works. Does file size and time to sort increase geometrically? I have a 5.3 billion line file I'd like to use with sort -u I'm wondering if that'll take forever because of a geometric expansion? If it takes 100 hours that's fine but not 100 days. Thanks so much.

4. Shell Programming and Scripting

Script to search a large file with a list of terms in another file

Hi- I am trying to search a large file with a number of different search terms that are listed one per line in 3 different files. Most importantly I need to be able to do a case insensitive search. I have tried just using egrep -f but it doesn't seam to be able to handle the -i option when...

5. Shell Programming and Scripting

Word Frequency Sort

hello, Here is a program for creating a word-frequency # wf.gk --- program to generate word frequencies from a file { # remove punctuation: This will remove all punctuations from the file gsub(/_]/, "", $0) #Start frequency analysis for (i = 1; i <= NF; i++) freq++ } END #Print output...

6. UNIX for Advanced & Expert Users

Script to sort the files and append the extension .sort to the sorted version of the file

Hello all - I am to this forum and fairly new in learning unix and finding some difficulty in preparing a small shell script. I am trying to make script to sort all the files given by user as input (either the exact full name of the file or say the files matching the criteria like all files...

7. Shell Programming and Scripting

Script to pull hashes out of large text file

I am attempting to write a script that will pull out NTLM hashes from a text file that contains about 500,000 lines of data. Not all accounts contain hashes and I only need the ones that do contain hashes. Here is a sample of what the data looks like: There are thousands of other lines in...

8. UNIX for Advanced & Expert Users

Help optimizing sort of large files

I'm doing a hobby project that has me sorting huge files with sort of monotonous keys. It's very slow -- the current file is about 300 GB and has been sorting for a day. I know that sort has this --batch-size and --buffer-size parameters, but I'd like a jump start if possible to limit the...

9. Shell Programming and Scripting

Frequency of Words in a File, sed script from 1980

tr -cs A-Za-z\' '\n' | tr A-Z a-z | sort | uniq -c | sort -k1,1nr -k2 | sed ${1:-25} < book7.txt This is not my script, it can be found way back from 1980 but once it worked fine to give me the most used words in a text file. Now the shell is complaining about an error in sed sed: -e...

10. Shell Programming and Scripting

Script to compare files in 2 folders and delete the large file

Hello, my first thread here. I've been searching and fiddling around for about a week and I cannot find a solution.:confused: I have been converting all of my home videos to HEVC and sometimes the files end up smaller and sometimes they don't. I am currently comparing all the video files...

LEARN ABOUT V7

ppmhist

ppmhist(1)						      General Commands Manual							ppmhist(1)

NAME

       ppmhist - print a histogram of a portable pixmap

SYNOPSIS

       ppmhist [-hexcolor] [-noheader] [-map] [-nomap] [-sort={frequency,rgb}] [ppmfile]

DESCRIPTION

       Reads  a  PPM  image as input.  Generates a histogram of the colors in the image, i.e. a list of all the colors and how many pixels of each
       color are in the image.

OPTIONS

       -sort={frequency,rgb}
	      The -sort option determines the order in which the colors are listed in the output.  frequency means to list them in  order  of  how
	      pixels  in the input image have the color, with the most represented colors first.  rgb means to sort them first by the intensity of
	      the red component of the color, the of the green, then of the blue, with the least intense first.

	      The default is frequency.

       -hexcolor
	      Print the color components in hexadecimal.  Default is decimal.

       -noheader
	      Do not print the column headings.

       -map   Generates a PPM file of the colormap for the image, with the color histogram as comments.

       -nomap Generates the histogram for human reading.  This is the default.

SEE ALSO

       ppm(5), pgmhist(1), ppmtomap(1), pnmhistmap(1), ppmchange(1)

AUTHOR

       Copyright (C) 1989 by Jef Poskanzer.

								 17 September 2000							ppmhist(1)

10 More Discussions You Might Find Interesting

1. HP-UX

Need to split a large data file using a Unix script

Discussion started by: SAIK

2. Shell Programming and Scripting

script to splite large file to number of small files

Discussion started by: ahmed.gad

3. UNIX for Dummies Questions & Answers

Sort large file

Discussion started by: dcfargo

4. Shell Programming and Scripting

Script to search a large file with a list of terms in another file

Discussion started by: dougzilla