Sponsored Content
Top Forums Shell Programming and Scripting joining multiple files into one while putting the filename in the file Post 302369222 by phil_heath on Saturday 7th of November 2009 12:15:13 AM
Old 11-07-2009
thats perfect!

thanks
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Joining files based on multiple keys

I need a script (perl or awk..anything is fine) to join 3 files based on three key columns. The no of non-key columns can vary in each file. The columns are delimited by semicolon. For example, File1 Dim1;Dim2;Dim3;Fact1;Fact2;Fact3;Fact4;Fact5 ---- data delimited by semicolon --- ... (1 Reply)
Discussion started by: Sebben
1 Replies

2. Shell Programming and Scripting

How to redirect the output to multiple files without putting on console

How to redirect the output to multiple files without putting on console I tried tee but it writes to STDOUT , which I do not want. Test.sh ------------------ #!/bin/ksh echo "Hello " tee -a file1 file2 ---------------------------- $>./Test.sh $> Expected output: -------------------... (2 Replies)
Discussion started by: prashant43
2 Replies

3. UNIX for Dummies Questions & Answers

JOINING MULTIPLE LINES IN A TEXT FILE USING GAWK

sir... am having a data file of customer master., containing some important fields as a set one line after another., what i want is to have one set of these fields(rows) one after another in line.........then the second set... and so on... till the last set completed. I WANT THE DATA... (0 Replies)
Discussion started by: KANNI786
0 Replies

4. UNIX for Dummies Questions & Answers

Joining string on multiple files

Hi guys, I am a forum (and a bit of a unix) newbie, and I currently have a tricky problem lying ahead of me. I have multiple files, and I am looking to join the files on the first column. Example: File 1 andy b 100 amy c 200 amy d 300 File 2 andy c 200 amy c 100 clyde o 50 ... (3 Replies)
Discussion started by: jdr0317
3 Replies

5. Shell Programming and Scripting

Joining multiple files based on one column with different and similar values (shell or perl)

Hi, I have nine files looking similar to file1 & file2 below. File1: 1 ABCA1 1 ABCC8 1 ABR:N 1 ACACB 1 ACAP2 1 ACOT1 1 ACSBG 1 ACTR1 1 ACTRT 1 ADAMT 1 AEN:N 1 AKAP1File2: 1 A4GAL 1 ACTBL 1 ACTL7 (4 Replies)
Discussion started by: seqbiologist
4 Replies

6. Shell Programming and Scripting

Other alternative for joining together columns from multiple files

Hi again, I have monthly one-column files of roughly around 10 years. Is there a more efficient way to concatenate these files column-wise other than using paste command? For instance: file1.txt 12 13 15 12 file2.txt 14 15 18 19 file3.txt 20 21 (8 Replies)
Discussion started by: ida1215
8 Replies

7. Shell Programming and Scripting

Adding filename and line number from multiple files to final file

Hi all, I have 20 files (file001.txt upto file020.txt) and I want to read them from 3rd line upto end of file (line 1002). But in the final file they should appear to start from line 1. I need following kind of output in a single file: Filename Line number 2ndcolumn 4thcolumn I... (14 Replies)
Discussion started by: bioinfo
14 Replies

8. UNIX for Dummies Questions & Answers

Joining different columns from multiple files

Hello again, I am trying to join 3rd column of 3 files into the end on one file and save it separately... my data looks like this file 1 Bob, Green, 80 Mark, Brown, 70 Tina, Smith, 60 file 2 Bob, Green, 70 Mark, Brown, 60 Tina, Smith, 50 file 3 Bob, Green, 50 Mark, Brown,60 Tina,... (6 Replies)
Discussion started by: A-V
6 Replies

9. UNIX for Dummies Questions & Answers

Putting $$ before filename

Hello , I am searching a directory for a file and have to assign the filename to a variable . The variable must have form $$filename So my code is echo "'$$filename='`ls -lrt *PreMatch*.csv| head -1 | nawk '{print $9}'`" however $$ is converting to a number . How could I make it $$... (3 Replies)
Discussion started by: Pratik4891
3 Replies

10. Shell Programming and Scripting

CSV joining and checking multiple files

Hello, For our work we use several scripts to gather/combine data for use in our webshop. Untill now we did not had any problems but since a couple days we noticed some mismatches between imports. It happened that several barcodes where matched even though it was a complete other product. Of... (19 Replies)
Discussion started by: SDohmen
19 Replies
TC(8)                                                                  Linux                                                                 TC(8)

NAME
tbf - Token Bucket Filter SYNOPSIS
tc qdisc ... tbf rate rate burst bytes/cell ( latency ms | limit bytes ) [ mpu bytes [ peakrate rate mtu bytes/cell ] ] burst is also known as buffer and maxburst. mtu is also known as minburst. DESCRIPTION
The Token Bucket Filter is a classful queueing discipline available for traffic control with the tc(8) command. TBF is a pure shaper and never schedules traffic. It is non-work-conserving and may throttle itself, although packets are available, to ensure that the configured rate is not exceeded. It is able to shape up to 1mbit/s of normal traffic with ideal minimal burstiness, send- ing out data exactly at the configured rates. Much higher rates are possible but at the cost of losing the minimal burstiness. In that case, data is on average dequeued at the config- ured rate but may be sent much faster at millisecond timescales. Because of further queues living in network adaptors, this is often not a problem. ALGORITHM
As the name implies, traffic is filtered based on the expenditure of tokens. Tokens roughly correspond to bytes, with the additional con- straint that each packet consumes some tokens, no matter how small it is. This reflects the fact that even a zero-sized packet occupies the link for some time. On creation, the TBF is stocked with tokens which correspond to the amount of traffic that can be burst in one go. Tokens arrive at a steady rate, until the bucket is full. If no tokens are available, packets are queued, up to a configured limit. The TBF now calculates the token deficit, and throttles until the first packet in the queue can be sent. If it is not acceptable to burst out packets at maximum speed, a peakrate can be configured to limit the speed at which the bucket empties. This peakrate is implemented as a second TBF with a very small bucket, so that it doesn't burst. To achieve perfection, the second bucket may contain only a single packet, which leads to the earlier mentioned 1mbit/s limit. This limit is caused by the fact that the kernel can only throttle for at minimum 1 'jiffy', which depends on HZ as 1/HZ. For perfect shap- ing, only a single packet can get sent per jiffy - for HZ=100, this means 100 packets of on average 1000 bytes each, which roughly corre- sponds to 1mbit/s. PARAMETERS
See tc(8) for how to specify the units of these values. limit or latency Limit is the number of bytes that can be queued waiting for tokens to become available. You can also specify this the other way around by setting the latency parameter, which specifies the maximum amount of time a packet can sit in the TBF. The latter calcula- tion takes into account the size of the bucket, the rate and possibly the peakrate (if set). These two parameters are mutually exclusive. burst Also known as buffer or maxburst. Size of the bucket, in bytes. This is the maximum amount of bytes that tokens can be available for instantaneously. In general, larger shaping rates require a larger buffer. For 10mbit/s on Intel, you need at least 10kbyte buffer if you want to reach your configured rate! If your buffer is too small, packets may be dropped because more tokens arrive per timer tick than fit in your bucket. The minimum buffer size can be calculated by dividing the rate by HZ. Token usage calculations are performed using a table which by default has a resolution of 8 packets. This resolution can be changed by specifying the cell size with the burst. For example, to specify a 6000 byte buffer with a 16 byte cell size, set a burst of 6000/16. You will probably never have to set this. Must be an integral power of 2. mpu A zero-sized packet does not use zero bandwidth. For ethernet, no packet uses less than 64 bytes. The Minimum Packet Unit determines the minimal token usage (specified in bytes) for a packet. Defaults to zero. rate The speed knob. See remarks above about limits! See tc(8) for units. Furthermore, if a peakrate is desired, the following parameters are available: peakrate Maximum depletion rate of the bucket. The peakrate does not need to be set, it is only necessary if perfect millisecond timescale shaping is required. mtu/minburst Specifies the size of the peakrate bucket. For perfect accuracy, should be set to the MTU of the interface. If a peakrate is needed, but some burstiness is acceptable, this size can be raised. A 3000 byte minburst allows around 3mbit/s of peakrate, given 1000 byte packets. Like the regular burstsize you can also specify a cell size. EXAMPLE &; USAGE To attach a TBF with a sustained maximum rate of 0.5mbit/s, a peakrate of 1.0mbit/s, a 5kilobyte buffer, with a pre-bucket queue size limit calculated so the TBF causes at most 70ms of latency, with perfect peakrate behaviour, issue: # tc qdisc add dev eth0 handle 10: root tbf rate 0.5mbit burst 5kb latency 70ms peakrate 1mbit minburst 1540 To attach an inner qdisc, for example sfq, issue: # tc qdisc add dev eth0 parent 10:1 handle 100: sfq Without inner qdisc TBF queue acts as bfifo. If the inner qdisc is changed the limit/latency is not effective anymore. SEE ALSO
tc(8) AUTHOR
Alexey N. Kuznetsov, <kuznet@ms2.inr.ac.ru>. This manpage maintained by bert hubert <ahu@ds9a.nl> iproute2 13 December 2001 TC(8)
All times are GMT -4. The time now is 02:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy