Huge mysql table with Zend Framework
I have a mysql table with over 4 million of data; well the problem is that SOME queries WORK and SOME DON'T it depends on the search term, if the search term has a big volume of data in the table than I get the following error:
Fatal error: Allowed memory size of 1048576000 bytes exhausted (tried to allocate 75 bytes) in /home/****/public_html/Zend/Db/Statement/Pdo.php on line 290
I currently have Zend Framework cache for metadata enabled, I have index on all the fields from that table.The site is running on a dedicated server with 2gb of ram.
I've also set memory limit to: ini_set("memory_limit","1000M");
Any other things that I can optimize?
Those are the types of query that I'm currently using:
$do = $this->select()
->where('branche LIKE ?','%'.mysql_escape_string($branche).'%')
->order('premium DESC');
}
//For name
if(empty($branche) && empty($plz))
{
$do = $this->select("MATCH(`name`) AGAINST ('{$theString}') AS score")
->where('MATCH(`name`) AGAINST( ? IN BOOLEAN MODE)', $theString)
->order('premium DESC, score');
}
And a few other, but they are pretty much the same.
Best Regards
//LE
ZEND_PAGINATOR CODE
$d = $firmen->doSearch($finalType,$theKeyword,$thePLZ,$theBranche,false,false,false,$theOrder);
if ($d !== false) {
$paginator = Zend_Paginator::factory($d);
$paginator->setItemCountPerPage(5)
->setPageRange(10)
->setCurrentPageNumber($pag);
$this->view->data = $paginator;
//MYSQL EXPLAIN RESULTS
mysql> EXPLAIN select * from `wirtscha_ksw`.`firmen` WHERE `name` LIKE '%gmbh%';ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 32911
Current database: *** NONE ***
+----+-------------+--------+------+---------------+------+---------+------+---------+-------------+
| id | select_t开发者_开发百科ype | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+------+---------------+------+---------+------+---------+-------------+
| 1 | SIMPLE | firmen | ALL | NULL | NULL | NULL | NULL | 3749155 | Using where |
+----+-------------+--------+------+---------------+------+---------+------+---------+- ------------+
1 row in set (0.03 sec
Do you really need to load all records at once? I'd recommend you to use LIMIT in those queries. In case you need to present the data, also consider using Zend_Paginator.
UPDATE: The approach you're taking is to pass Zend_Paginator all the results, which is overkilling with large resultsets. A more optimal approach in those cases is to pass it just the query, and it will then take care of fetching only the data that's needed to display the page (this includes counting the number of records and limiting the query to the number of results per page), e.g.:
$paginator = new Zend_Paginator(
// $query is an instance of Zend_Db_Select
new Zend_Paginator_Adapter_DbSelect($query);
);
$paginator->setItemCountPerPage(5)
->setPageRange(10)
->setCurrentPageNumber($pag);
Zend paginator is memory exhaustive out-of-the-box as it is. I've had to increase the default memory_limit x4 to 512M already and that's with a test database that is not as large as the live version ultimately will be.
精彩评论