Sponsored Content
Top Forums Shell Programming and Scripting Concatenating many files based on a specific column contents Post 302653939 by ks_reddy on Monday 11th of June 2012 03:58:37 AM
Old 06-11-2012
Concatenating many files based on a specific column contents

Dear all,

I have many files(.csv) in a directory.
I want to concatenate the files which have similar entries in a particular column and save into a new file like result_datetime.csv etc.
One example file is like below.

Code:
Sno,Step,Data1,Data2,Data3 etc.
1,0,2,3,4
2,1,3,4,5
3,2,0,1,1

and one more example is like below.

Code:
Sno,Step,Data1,Data2,Data3 etc.
1,0,4,1,1
2,1,4,4,5
3,2,6,1,1

Here both files have similar entries for col2(Step).
So we need to concatenate these files.
Like this we need to scan all the files in the folder and look for specific column entries and concatenate all similar files.

Regards
Sid
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Retrieve line from a file based on a value in specific column

Hi, I have a file that has several values seperated by ":" 2006:John:Student:Football:Portugal:Cinema 2006:James:Engineer:Basket:Poland:Theatre 2007:Lucy:Diver:Gymnastic:England:Music 2007:Smith:Plumber:Basket:Spain:Poker I need make a filter based on the 5th field to find countries that... (1 Reply)
Discussion started by: efernandes
1 Replies

2. UNIX for Dummies Questions & Answers

(cont) Retrieve line from a file based on a value in specific column

HI, Your help was great: awk -F":" '$5 ~ /^P/{print }' file I would like to know what changes need to be done to this line code, so that I can put it in a shell script and call it as the example below. example: countries that start with chacater 'P' > country P Result: ... (0 Replies)
Discussion started by: efernandes
0 Replies

3. Shell Programming and Scripting

Merging files based on the contents

Hi, I have a file f1 having the contents as below select (<condn>) from dual I have another file f2 having the contents as below 1, 2, 3 I want to replace <condn> in f1 with the contents of f2 I tried using sed like this sed "s:<condn>:`cat f2`:g" f1 The above command resulted in sed:... (3 Replies)
Discussion started by: mr_manii
3 Replies

4. Shell Programming and Scripting

Concatenating and appending string based on specific pattern match

Input #GEO-1-type-1-fwd-Initial 890 1519 OPKHIJEFVTEFVHIJEFVOPKHIJTOPKEFVHIJTEFVOPKOPKHIJHIJHIJTTOPKHIJHIJEFVEFVOPKHIJOPKHIJOPKEFVEFVOPKHIJHIJEFVHIJHIJEFVTHIJOPKOPKTEFVEFVEFVOPKHIJOPKOPKHIJTTEFVEFVTEFV #GEO-1-type-2-fwd-Terminal 1572 2030... (7 Replies)
Discussion started by: patrick87
7 Replies

5. Shell Programming and Scripting

Renaming Files Based on Contents

Hello everyone, I currently have a situation which is causing me some issues for keeping up with certain files. I will explain this to the best of my abilities. I have a list of files as follows 50_REPORT_1111 - file contains the word Car 50_REPORT_2222 - file contains the word House... (15 Replies)
Discussion started by: DerangedNick
15 Replies

6. Shell Programming and Scripting

Concatenating contents of a file with members in a directory

Hi, I have a unix file with the below structure - CustId1 CustName1 CustPhn1 /u/home/xmldata/A000001 CustId2 CustName2 CustPhn2 /u/home/xmldata/A000002 CustId3 CustName3 CustPhn3 /u/home/xmldata/A000003 Then I have another unix directory /u/home/xmldata This directory has... (3 Replies)
Discussion started by: Simanto
3 Replies

7. UNIX for Dummies Questions & Answers

How to cut from a text file based on value of a specific column?

Hi, I have a tab delimited text file from which I want to cut out specific columns. If the second column equals one, I want to cut out columns 1 and 5 and 6. If the second column equals two, I want to cut out columns 1 and 5 and 7. How do I go about doing that? Thanks! (4 Replies)
Discussion started by: evelibertine
4 Replies

8. UNIX for Advanced & Expert Users

Appending a files contents to the end of a specific file name in several directories

Here is my dir structure: /tmp/dave/myappend.txt /tmp/dave/dir1/test.txt /tmp/dave/dir2/test.txt /tmp/dave/dir3/test.txt /tmp/dave/dir4/test.txt I want to append the contents of myappend.txt to the end of each file with the name "test.txt" in all dirs in /tmp/dave/ I have tried this:... (2 Replies)
Discussion started by: bigd213
2 Replies

9. Shell Programming and Scripting

How to remove a line based on contents of the first column?

Good day all. Using basic UNIX/Linux tools, how would you delete a line based on a character found in column 1? For example, if the CITY name contains an 'a' or 'A', delete the line: New York City; New York Los Angeles; California Chicago; Illinois Houston; Texas Philadelphia;... (3 Replies)
Discussion started by: BRH
3 Replies

10. Shell Programming and Scripting

Need a Linux command for find/replace column based on specific criteria.

I'm new to shell programming, I have a huge text file in the following format, where columns are separated by single space: ACA MEX 4O_ $98.00 $127.40 $166.60 0:00 0:00 0 ; ACA YUL TS_ $300.00 $390.00 $510.00 0:00 0:00 0 ; ACA YYZ TS_ $300.00 $390.00 $510.00 0:00 0:00 0 ; ADZ YUL TS_ $300.00... (3 Replies)
Discussion started by: transat
3 Replies
LP(4)							   BSD Kernel Interfaces Manual 						     LP(4)

NAME
lp -- printer port Internet Protocol driver SYNOPSIS
ifconfig plip0 myaddress hisaddress [-link0] device ppbus device plip device ppc DESCRIPTION
The lp driver allows a PC parallel printer port to be used as a point-to-point network interface between two similarly configured systems. Data is transferred 4 bits at a time, using the printer status lines for input: hence there is no requirement for special bidirectional hard- ware and any standard AT-compatible printer port with working interrupts may be used. During the boot process, for each plip device which is probed and has an interrupt assigned, a corresponding network device is created. Configuring an lp device with ifconfig(8) causes the corresponding parallel port bus to be reserved for PLIP until the network interface is configured 'down'. The communication protocol is selected by the link0 flag: -link0 (default) Use FreeBSD mode (LPIP). This is the simpler of the two modes and therefore slightly more efficient. link0 Use Crynwr/Linux compatible mode (CLPIP). This mode has a simulated Ethernet packet header, and is easier to interface to other types of equipment. The interface MTU defaults to 1500, but may be set to any value. Both ends of the link must be configured with the same MTU. Cable Connections The cable connecting the two parallel ports should be wired as follows: Pin Pin Description 2 15 Data0 -> ERROR* 3 13 Data1 -> SLCT 4 12 Data2 -> PE 5 10 Data3 -> ACK* 6 11 Data4 -> BUSY 15 2 ERROR* -> Data0 13 3 SLCT -> Data1 12 4 PE -> Data2 10 5 ACK* -> Data3 11 6 BUSY -> Data4 18-25 18-25 Ground Cables with this wiring are widely available as 'Laplink' cables, and are often coloured yellow. The connections are symmetric, and provide 5 lines in each direction (four data plus one handshake). The two modes use the same wiring, but make a different choice of which line to use as handshake. FreeBSD LPIP mode The signal lines are used as follows: Data0 (Pin 2) Data out, bit 0. Data1 (Pin 3) Data out, bit 1. Data2 (Pin 4) Data out, bit 2. Data3 (Pin 5) Handshake out. Data4 (Pin 6) Data out, bit 3. ERROR* (pin 15) Data in, bit 0. SLCT (pin 13) Data in, bit 1. PE (pin 12) Data in, bit 2. BUSY (pin 11) Data in, bit 3. ACK* (pin 10) Handshake in. When idle, all data lines are at zero. Each byte is signalled in four steps: sender writes the 4 most significant bits and raises the hand- shake line; receiver reads the 4 bits and raises its handshake to acknowledge; sender places the 4 least significant bits on the data lines and lowers the handshake; receiver reads the data and lowers its handshake. The packet format has a two-byte header, comprising the fixed values 0x08, 0x00, immediately followed by the IP header and data. The start of a packet is indicated by simply signalling the first byte of the header. The end of the packet is indicated by inverting the data lines (i.e., writing the ones-complement of the previous nibble to be transmitted) without changing the state of the handshake. Note that the end-of-packet marker assumes that the handshake signal and the data-out bits can be written in a single instruction - otherwise certain byte values in the packet data would falsely be interpreted as end-of-packet. This is not a problem for the PC printer port, but requires care when implementing this protocol on other equipment. Crynwr/Linux CLPIP mode The signal lines are used as follows: Data0 (Pin 2) Data out, bit 0. Data1 (Pin 3) Data out, bit 1. Data2 (Pin 4) Data out, bit 2. Data3 (Pin 5) Data out, bit 3. Data4 (Pin 6) Handshake out. ERROR* (pin 15) Data in, bit 0. SLCT (pin 13) Data in, bit 1. PE (pin 12) Data in, bit 2. ACK* (pin 10) Data in, bit 3. BUSY (pin 11) Handshake in. When idle, all data lines are at zero. Each byte is signalled in four steps: sender writes the 4 least significant bits and raises the hand- shake line; receiver reads the 4 bits and raises its handshake to acknowledge; sender places the 4 most significant bits on the data lines and lowers the handshake; receiver reads the data and lowers its handshake. [Note that this is the opposite nibble order to LPIP mode]. Packet format is: Length (least significant byte) Length (most significant byte) 12 bytes of supposed MAC addresses (ignored by FreeBSD). Fixed byte 0x08 Fixed byte 0x00 <IP datagram> Checksum byte. The length includes the 14 header bytes, but not the length bytes themselves nor the checksum byte. The checksum is a simple arithmetic sum of all the bytes (again, including the header but not checksum or length bytes). FreeBSD calculates outgoing checksums, but does not validate incoming ones. The start of packet has to be signalled specially, since the line chosen for handshake-in cannot be used to generate an interrupt. The sender writes the value 0x08 to the data lines, and waits for the receiver to respond by writing 0x01 to its data lines. The sender then starts signalling the first byte of the packet (the length byte). End of packet is deduced from the packet length and is not signalled specially (although the data lines are restored to the zero, idle state to avoid spuriously indicating the start of the next packet). SEE ALSO
ppbus(4), ppc(4), ifconfig(8) BUGS
Busy-waiting loops are used while handshaking bytes, (and worse still when waiting for the receiving system to respond to an interrupt for the start of a packet). Hence a fast system talking to a slow one will consume excessive amounts of CPU. This is unavoidable in the case of CLPIP mode due to the choice of handshake lines; it could theoretically be improved in the case of LPIP mode. Polling timeouts are controlled by counting loop iterations rather than timers, and so are dependent on CPU speed. This is somewhat sta- bilised by the need to perform (slow) ISA bus cycles to actually read the port. BSD
March 4, 1996 BSD
All times are GMT -4. The time now is 05:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy