Filtering files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Filtering files
# 22  
Old 06-09-2012
Or, for a quick-and-dirty fix, tack a ... | awk 'NF>2' on the end to suppress lines without enough fields.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help on filtering

Hi experts, I have a file image.csv as below: COMPUTERNAME,23/07/2013,22/07/2013,21/07/2013,20/07/2013,19/07/2013,18/07/2013,17/07/2013 AED03852180,3,3,3,3,3,3,3 AED03852181,3,3,3,3,3,3,1 AED09020382,3,0,3,0,3,3,3 AED09020383,1,3,3,3,2,1,3 AED09020386,3,3,0,3,3,0,3 ... (4 Replies)
Discussion started by: zaq1xsw2
4 Replies

2. Shell Programming and Scripting

Reading 2 CSV files and filtering data based on group

I have two CSV files in the following format: First file: GroupID, PID:TID, IP, Port Sample data: 0,1000:11,127.0.0.1,445 0,-1:-1,127.0.0.1,800 1,1000:11,127.0.0.1,445 1,-1:-1,127.0.0.1,900 2,1000:11,127.0.0.1,445 2,-1:-1,180.0.0.3,900 Second file: IP,Port,PID Sample data... (6 Replies)
Discussion started by: rakesh_arxmind
6 Replies

3. Shell Programming and Scripting

Filtering files

Hi all, I have some files with different extensions. I want to list the files that doesnt end with particular extension for eg .txt. I want to list all files except .txt. How can I do the same? Thanks Ananth (2 Replies)
Discussion started by: Ananthdoss
2 Replies

4. Programming

need help with shell script filtering files and sort! newbie question?

Hi folks, I would like to get familiar with shell script programing. The first task is: write a shell script that: scans your home-folder + sub-directory for all txt-files that all users of your group are allowed to read and write then output these files sorted by date of last... (4 Replies)
Discussion started by: rollinator
4 Replies

5. Shell Programming and Scripting

Please help me to do some filtering

I have to grep a pattern. scenario is like :- Suppose "/etc/sec/one" is a string, i need to check if this string contains "one" using any utility something like if /etc/sec/one | grep ; then Thanks in advance Renjesh Raju (3 Replies)
Discussion started by: Renjesh
3 Replies

6. Shell Programming and Scripting

Filtering the yesterdays date from log files via script.

hi All, I have this sample text file - access.log: Jan 18 21:34:29 root 209.151.232.70 Jan 18 21:34:40 root 209.151.232.70 Jan 18 21:34:43 root 209.151.232.70 Jan 18 21:34:56 root 209.151.232.70 Jan 18 21:35:10 root 209.151.232.70 Jan 18 21:35:23 root 209.151.232.70 Jan 18 21:36:04 root... (2 Replies)
Discussion started by: linuxgeek
2 Replies

7. Shell Programming and Scripting

Filtering multiple files with variables

Hi, I spend few hours already searching this forum, but cannot find the solution matching exactly my case. I have multiple log files, I need to filter them so I get info about certain event. So we have files: LOGA.txt LOGB.txt LOGC.txt LOGD.txt LOGE.txt 1. I need to grep lines in... (10 Replies)
Discussion started by: Vitoriung
10 Replies

8. Shell Programming and Scripting

Indexing or Filtering code- Pattern Search by comparing two files

So here is goes to the Gurus of shell programming......I have tried a lot of different ways and its a very challenging code to write but i am enjoying it as i troubleshoot and hopefully someone can provide me a better option....Thank you in advance for your time and support....Much appreciated... ... (12 Replies)
Discussion started by: aavam
12 Replies

9. UNIX for Dummies Questions & Answers

Filtering pcap files

Hi, I am new at UNIX and programing in general and only have a basic knowledge of C++. I am helping out with some research at a college and was given the task to sort through captured packets via IP addresses. I was wondering if anyone could help me with writing a code which filters through pcap... (1 Reply)
Discussion started by: hershey101
1 Replies

10. Shell Programming and Scripting

Merging files with AWK filtering and counting lines

Hi there, I have a couple of files I need to merge. I can do a simple merge by concatenating them into one larger file. But then I need to filter the file to get a desired result. The output looks like this: TRNH 0000000010941 ORDH OADR OADR ORDL ENDT 1116399 000000003... (2 Replies)
Discussion started by: Meert
2 Replies
Login or Register to Ask a Question
bup-margin(1)						      General Commands Manual						     bup-margin(1)

NAME
bup-margin - figure out your deduplication safety margin SYNOPSIS
bup margin [options...] DESCRIPTION
bup margin iterates through all objects in your bup repository, calculating the largest number of prefix bits shared between any two entries. This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids. For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45. That means a 46-bit hash would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by its first 46 bits. The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects. Since SHA-1 hashes have 160 bits, that leaves 115 bits of margin. Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits with far fewer objects. If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see if you're getting dangerously close to 160 bits. OPTIONS
--predict Guess the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer from the guess. This is potentially useful for tuning an interpolation search algorithm. --ignore-midx don't use .midx files, use only .idx files. This is only really useful when used with --predict. EXAMPLE
$ bup margin Reading indexes: 100.00% (1612581/1612581), done. 40 40 matching prefix bits 1.94 bits per doubling 120 bits (61.86 doublings) remaining 4.19338e+18 times larger is possible Everyone on earth could have 625878182 data sets like yours, all in one repository, and we would expect 1 object collision. $ bup margin --predict PackIdxList: using 1 index. Reading indexes: 100.00% (1612581/1612581), done. 915 of 1612581 (0.057%) SEE ALSO
bup-midx(1), bup-save(1) BUP
Part of the bup(1) suite. AUTHORS
Avery Pennarun <apenwarr@gmail.com>. Bup unknown- bup-margin(1)