Sponsored Content
Top Forums Web Development Mysql question: Best way to update a column containing 8 million rows Post 302357837 by Scott on Wednesday 30th of September 2009 06:37:36 PM
Old 09-30-2009
Hi.

Sorry, I'm not so familiar with mySQL, but as you didn't get an answer yet!

This is based on Oracle, but the same ideas should relate.

Firstly it's safe to update a table with however many rows. The worst that could happen is that the update fails and the uncommitted changes are rolled back.


You could:
  • split the updates in to smaller batches
  • add commit statements every few hundred / thousand statements
  • instruct your sql client to commit every few hundred / thousand statements
  • add more redo / undo logs or make your existing ones larger, or both
  • switch off logging on the table (alter table .. nologging if mySQL supports that) to reduce redo / undo (in most cases)
  • disable referential / unique constraint indexes (more for performance)

Last edited by Scott; 09-30-2009 at 07:48 PM..
 

10 More Discussions You Might Find Interesting

1. Web Development

[MYSQL] problem with spaces in rows

Hello. I'm not sure how I can get around this, or what I am doing wrong, but I need some help. :) I want to do an select query looking like this: SELECT venue, SUM( amount ) FROM IWD WHERE venue = 'Foxy Hollow' Unfortunately I need to have spaces in the names in these fields, is... (10 Replies)
Discussion started by: noratx
10 Replies

2. Shell Programming and Scripting

awk command to print only selected rows in a particular column specified by column name

Dear All, I have a data file input.csv like below. (Only five column shown here for example.) Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 3,2,4,5,6 5,3,5,5,6 From this I want the below output Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 where the second column... (4 Replies)
Discussion started by: ks_reddy
4 Replies

3. UNIX for Dummies Questions & Answers

merging rows into new file based on rows and first column

I have 2 files, file01= 7 columns, row unknown (but few) file02= 7 columns, row unknown (but many) now I want to create an output with the first field that is shared in both of them and then subtract the results from the rest of the fields and print there e.g. file 01 James|0|50|25|10|50|30... (1 Reply)
Discussion started by: A-V
1 Replies

4. UNIX for Dummies Questions & Answers

[Solved] Deleting all rows where the first column equals the second column

Hi, I have a tab delimited text file where the first two columns equal numbers. I want to delete all rows where the value in the first column equals the second column. How do I go about doing that? Thanks! Input: 1 1 ABC DEF 2 2 IJK LMN 1 2 ZYX OPW Output: 1 2 ZYX OPW (2 Replies)
Discussion started by: evelibertine
2 Replies

5. Shell Programming and Scripting

Scrape 10 million pages and save the raw html data in mysql database

I have a list of 10 million page urls. I want those pages scraped and saved in the mysql database as raw html. I own a Linux VPS server with 1GB RAM and WHM/cPanel. I would like to scrape at least 100,000 urls in 24 hours. So can anyone give me some sample shell scripting code? (4 Replies)
Discussion started by: Viruthagiri
4 Replies

6. Shell Programming and Scripting

Converting Single Column into Multiple rows, but with strings to specific tab column

Dear fellows, I need your help. I'm trying to write a script to convert a single column into multiple rows. But it need to recognize the beginning of the string and set it to its specific Column number. Each Line (loop) begins with digit (RANGE). At this moment it's kind of working, but it... (6 Replies)
Discussion started by: AK47
6 Replies

7. Shell Programming and Scripting

MySql split rows

Dear community, I have to split string in table and list all values. I'll skip the code and jump directly to mysql query. This is the table: category title ======= ======= 7,3 title 1 1,3 title 2 1,2,3 title 3 Now, what I need is split category into single... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

8. Shell Programming and Scripting

Update a mysql column via bash script

Hello, I want to check the value of all MySQL columns.(column name is "status") via bash script. If value is "0" at I want to make only single column value to "1" I have many "0" values on mysql database(on "status" column) "0" means it is a draft post. I want to publish a post. I... (2 Replies)
Discussion started by: tara123
2 Replies

9. UNIX for Dummies Questions & Answers

Mysql: How to update value in 27000 rows?

Hello, some member created 27000 posts in wrong section (lol :D) so i need to edit all his entries to get new section ID. SELECT * FROM `phpbb_topics` WHERE `topic_first_poster_name` LIKE "%ozerway%"; this will select all his topics... the column with forum id is named "forum_id" and... (3 Replies)
Discussion started by: postcd
3 Replies

10. Programming

MYSQL - trigger update on record insert or update

Right I have a MYSQL database with table1 with 3 columns, colA, colB and colC. I want to combine the data in the 3 columns into a 4th column names col_comb. Here's the SQL command that works: UPDATE table1 SET `col_comb` = CONCAT( `colA` , ' - ', `colB` , ', ', `colC` ); So now I want this... (5 Replies)
Discussion started by: barrydocks
5 Replies
Alzabo::Runtime::JoinCursor(3pm)			User Contributed Perl Documentation			  Alzabo::Runtime::JoinCursor(3pm)

NAME
Alzabo::Runtime::JoinCursor - Cursor that returns arrays of "Alzabo::Runtime::Row" objects SYNOPSIS
use Alzabo::Runtime::JoinCursor; my $cursor = $schema->join( tables => [ $foo, $bar ], where => [ $foo->column('foo_id'), '=', 1 ] ); while ( my @rows = $cursor->next ) { print $rows[0]->select('foo'), " "; print $rows[1]->select('bar'), " "; } DESCRIPTION
Objects in this class are used to return arrays of Alzabo::Runtime::Row objects when requested. The cursor does not preload objects but rather creates them on demand, which is much more efficient. For more details on the rational please see the RATIONALE FOR CURSORS section in Alzabo::Design. INHERITS FROM
"Alzabo::Runtime::Cursor" METHODS
next Returns the next array of "Alzabo::Runtime::Row" objects or an empty list if no more are available. If an individual row could not be fetched, then the array may contain some "undef" values. For outer joins, this is normal behavior, but for regular joins, this probably indicates a data error. all_rows This method fetches all the rows available from the current point onwards. This means that if there are five set of rows that will be returned when the object is created and you call "next()" twice, calling "all_rows()" after it will only return three sets. The return value is an array of array references. Each of these references represents a single set of rows as they would be returned from the "next" method. reset Resets the cursor so that the next "next()" call will return the first row of the set. count Returns the number of rowsets returned by the cursor so far. next_as_hash Returns the next rows in a hash, where the hash keys are the table names and the hash values are the row object. If a table has been included in the join via an outer join, then it is only included in the hash if there is a row for that table. AUTHOR
Dave Rolsky, <autarch@urth.org> perl v5.8.8 2007-12-23 Alzabo::Runtime::JoinCursor(3pm)
All times are GMT -4. The time now is 07:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy