05-15-2013
Bash script with python slicing on multiple data files
I have 2 files generated in linux that has common output and were produced across multiple hosts with the same setup/configs. These files do some simple reporting on resource allocation and user sessions. So, essentially, say, 10 hosts, with the same (2) system reporting in the files, so a combination of 20 files total.
Neither of the files contain any reference to the host name, but do contain resource info and correlative user info per machine.
I need to meet some basic reporting objectives that include:
- taking cumulative reporting data on the resource and ranking the machines by availability
- taking cumulative reporting data on the users and ranking on usage
I created a simple python script that runs through all the files and produces output that summarizes per machine, but I've not got the parsed data converted into a real data value, so its not sortable
I also think that the following needs to happen:
- merge data files in linux w/ piping
- append the machine name/# to the 1st column
- read that w/ python slicing (since the machine name will always be the first "X" # of characters in the line
- use a bash script, wrapped, then execute the python script after bash has merged all the data files
so something like
/bin/bash
merge
push to python
I might need to change my python script so that the function input isnt the machine name, but more like "big_data_file" etc
that make sense?
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
its urgent!!!!!!111
i need a script which can delete data from multiple files.
plz if anybody knows the script plz write a mail to me :
(Email addresses are not allowed) (5 Replies)
Discussion started by: uni_ajay_r
5 Replies
2. Shell Programming and Scripting
Hi, I'm new here an dlearning a lot from this forum. i didnt find any solution for this in the forum.
I have already checked in folders in subversion named
HTT01,... HTT21.. and have files in each folder like below:
HTT01/HTT01_00000.hex
HTT01/HTT01_00000_fb_result.hex... (2 Replies)
Discussion started by: ravishan21
2 Replies
3. Shell Programming and Scripting
I have a directory full of zip files.
How would I write a bash script to enumerate all the zip files, remove the ".zip" from the file name, create a directory by that name and unzip each zip file into its corresponding directory?
Thanks!
Siegfried (3 Replies)
Discussion started by: siegfried
3 Replies
4. Shell Programming and Scripting
I'm trying to write a bash script and call it "compile" such that running it allows me to compile multiple files with the options "-help," "-backup," and "-clean". I've got the code for the options written, i just can't figure out how to read the input string and then translate that into option... (5 Replies)
Discussion started by: travis.batzer
5 Replies
5. Shell Programming and Scripting
Hi, I have a bunch of media files in a directory that have been converted (from MTS to MOV format), so my directory contains something like this:
clip1.mts
clip1.mov
clip2.mts
clip2.mov
The problem is that the .mov files that have been created have the timestamps of the conversion task,... (2 Replies)
Discussion started by: Krakus
2 Replies
6. Programming
Hello,
I have two files.
File 1 is a list of interested IDs
Ex1
Ex2
Ex3File 2 is the original file with over 8000 columns and 20 millions rows and is a compressed file .gz
Ex1 xx xx xx xx ....
Ex2 xx xx xx xx ....
Ex2 xx xx xx xx ....Now I need to extract the information for all the IDs of... (4 Replies)
Discussion started by: nans
4 Replies
7. Shell Programming and Scripting
Hi Everybody,
I'm a newbie to shell scripting, and I'd appreciate some help. I have a bunch of .txt files that have some unwanted content. I want to remove lines 1-3 and 1028-1098.
#!/bin/bash
for '*.txt' in <path to folder>
do
sed '1,3 d' "$f";
sed '1028,1098 d' "$f";
done
I... (2 Replies)
Discussion started by: BabyNuke
2 Replies
8. Shell Programming and Scripting
We have the data looks like below in a log file.
I want to generat files based on the string between two hash(#) symbol like below
Source:
#ext1#test1.tale2 drop
#ext1#test11.tale21 drop
#ext1#test123.tale21 drop
#ext2#test1.tale21 drop
#ext2#test12.tale21 drop
#ext3#test11.tale21 drop... (5 Replies)
Discussion started by: Sanjeev G
5 Replies
9. UNIX for Beginners Questions & Answers
HI
I have a dsv file that looks like:
<<BOF>>
record_number|id_number|first name|last name|msisdn|network|points|card number|gender
312|9101011234011|Test Junior|Smith|071 123 4321|MTN|73|1241551413214444|M
313|9012023213011|Bob|Smith|27743334321|Vodacom|3|1231233232323244|M... (4 Replies)
Discussion started by: tera
4 Replies
10. Programming
Experts,
I am writing a script and able to write only small piece of code and not able to collect logic to complete this task.
In input file have to look for name like like this (BGL_HSR_901_1AG_A_CR9KTR10) before sh iss neors. Record this (BGL_HSR_901_1AG_A_CR9KTR10) in csv file
Now have to... (0 Replies)
Discussion started by: as7951
0 Replies
LEARN ABOUT DEBIAN
isympy
isympy(1) isympy(1)
NAME
isympy - interactive shell for SymPy
SYNOPSIS
isympy [-c | --console]
isympy [ {-h | --help} | {-v | --version} ]
DESCRIPTION
isympy is a Python shell for SymPy. It is just a normal python shell (ipython shell if you have the ipython package installed) that exe-
cutes the following commands so that you don't have to:
>>> from __future__ import division
>>> from sympy import *
>>> x, y, z = symbols("xyz")
>>> k, m, n = symbols("kmn", integer=True)
So starting isympy is equivalent to starting python (or ipython) and executing the above commands by hand. It is intended for easy and
quick experimentation with SymPy. For more complicated programs, it is recommended to write a script and import things explicitly (using
the "from sympy import sin, log, Symbol, ..." idiom).
OPTIONS
-c shell, --console=shell
Use the specified shell (python or ipython) as console backend instead of the default one (ipython if present or python otherwise).
Example: isympy -c python
FILES
${HOME}/.sympy-history
Saves the history of commands when using the python shell as backend.
BUGS
The upstreams BTS can be found at <http://code.google.com/p/sympy/issues/list> Please report all bugs that you find in there, this will
help improve the overall quality of SymPy.
SEE ALSO
ipython(1), python(1)
2007-10-8 isympy(1)