I've solved the problem. More or less. This is what I went with for my shell script. It is probably far more complicated than it needs to be, but for my first shell script that isn't a basic option list, I'm quite pleased with it.
I have a file that is 20 - 80+ MB in size that is a certain type of log file.
It logs one of our processes and this process is multi-threaded. Therefore the log file is kind of a mess. Here's an example:
The logfile looks like: "DATE TIME - THREAD ID - Details", and a new file is created... (4 Replies)
I'm trying to make a simple search script but cannot get it right. The script should search for keywords inside files. Then return the file paths in a variable. (Each file path separated with \n).
#!/bin/bash
SEARCHQUERY="searchword1 searchword2 searchword3";
for WORD in $SEARCHQUERY
do
... (6 Replies)
Hi,
I have a text file with data in that I wish to extract, assign to a variable and process through a loop.
Kind of the process that I am after:
1: Grep the text file for the values.
Currently using:
cat /root/test.txt | grep TESTING= | awk -F"=" '{ a = $2 } {print a}' | sort -u
... (0 Replies)
Hi all,
I have problem with searching hundreds of CSV files, the problem is that search is lasting too long (over 5min).
Csv files are "," delimited, and have 30 fields each line, but I always grep same 4 fields - so is there a way to grep just those 4 fields to speed-up search.
Example:... (11 Replies)
Hello,
I am using sed in a for loop to replace text in a 100MB file. I have about 55,000 entries to convert in a csv file with two entries per line. The following script works to search file.txt for the first field from conversion.csv and then replace it with the second field. While it works fine,... (15 Replies)
This is my first experience writing unix script. I've created the following script. It does what I want it to do, but I need it to be a lot faster. Is there any way to speed it up?
cat 'Tax_Provision_Sample.dat' | sort | while read p; do fn=`echo $p|cut -d~ -f2,4,3,8,9`; echo $p >> "$fn.txt";... (20 Replies)
Dear all,
Please help with the following.
I have a file, let's call it data.txt, that has 3 columns and approx 700,000 lines, and looks like this:
rs1234 A C
rs1236 T G
rs2345 G T
Please use code tags as required by forum rules!
I have a second file, called reference.txt,... (1 Reply)
HI Guys hoping some one can help
I have two files on both containing uk phone numbers
master is a file which has been collated over a few years ad currently contains around 4 million numbers
new is a file which also contains 4 million number i need to split new nto two separate files... (4 Replies)
Hi,
I've written a ksh script that read a file and parse/filter/format each line. The script runs as expected but it runs for 24+ hours for a file that has 2million lines. And sometimes, the input file has 10million lines which means it can be running for more than 2 days and still not finish.... (9 Replies)
Hello experts,
we have input files with 700K lines each (one generated for every hour). and we need to convert them as below and move them to another directory once.
Sample INPUT:-
# cat test1
1559205600000,8474,NormalizedPortInfo,PctDiscards,0.0,Interface,BG-CTA-AX1.test.com,Vl111... (7 Replies)
Discussion started by: prvnrk
7 Replies
LEARN ABOUT PLAN9
dh_movefiles
DH_MOVEFILES(1) Debhelper DH_MOVEFILES(1)NAME
dh_movefiles - move files out of debian/tmp into subpackages
SYNOPSIS
dh_movefiles [debhelperoptions] [--sourcedir=dir] [-Xitem] [file...]
DESCRIPTION
dh_movefiles is a debhelper program that is responsible for moving files out of debian/tmp or some other directory and into other package
build directories. This may be useful if your package has a Makefile that installs everything into debian/tmp, and you need to break that
up into subpackages.
Note: dh_install is a much better program, and you are recommended to use it instead of dh_movefiles.
FILES
debian/package.files
Lists the files to be moved into a package, separated by whitespace. The filenames listed should be relative to debian/tmp/. You can
also list directory names, and the whole directory will be moved.
OPTIONS --sourcedir=dir
Instead of moving files out of debian/tmp (the default), this option makes it move files out of some other directory. Since the entire
contents of the sourcedir is moved, specifying something like --sourcedir=/ is very unsafe, so to prevent mistakes, the sourcedir must
be a relative filename; it cannot begin with a `/'.
-Xitem, --exclude=item
Exclude files that contain item anywhere in their filename from being installed.
file ...
Lists files to move. The filenames listed should be relative to debian/tmp/. You can also list directory names, and the whole directory
will be moved. It is an error to list files here unless you use -p, -i, or -a to tell dh_movefiles which subpackage to put them in.
NOTES
Note that files are always moved out of debian/tmp by default (even if you have instructed debhelper to use a compatibility level higher
than one, which does not otherwise use debian/tmp for anything at all). The idea behind this is that the package that is being built can be
told to install into debian/tmp, and then files can be moved by dh_movefiles from that directory. Any files or directories that remain are
ignored, and get deleted by dh_clean later.
SEE ALSO debhelper(7)
This program is a part of debhelper.
AUTHOR
Joey Hess <joeyh@debian.org>
11.1.6ubuntu2 2018-05-10 DH_MOVEFILES(1)