PHP数组插入/操作降级迭代

PHP数组插入/操作降级迭代

问题描述:

I am in the process of transferring data from one database to another. They are different dbs (mssql to mysql) so I cant do direct queries and am using PHP as an intermediary. Consider the following code. For some reason, each time it goes through the while loop it takes twice as much time as the time before.

$continue = true;
$limit = 20000;

while($continue){
    $i = 0;
    $imp->endTimer();
    $imp->startTimer("Fetching Apps");

    $qry = "THIS IS A BASIC SELECT QUERY";
    $data = $imp->src->dbQuery($qry, array(), PDO::FETCH_ASSOC);

    $inserts = array();
    $continue = (count($data) == $limit);

    $imp->endTimer();
    $imp->startTimer("Processing Apps " . memory_get_usage() );

    if($data == false){
        $continue = false;
    }
    else{
        foreach($data AS $row){

            // THERE IS SOME EXTREMELY BASIC IF STATEMENTS HERE

            $inserts[] = array(
                "paymentID"=>$paymentID,
                "ticketID"=>$ticketID,
                "applicationLink"=>$row{'ApplicationID'},
                "paymentLink"=>(int)($paymentLink),
                "ticketLink"=>(int)($ticketLink),
                "dateApplied"=>$row{'AddDate'},
                "appliedBy"=>$adderID,
                "appliedAmount"=>$amount,
                "officeID"=>$imp->officeID,
                "customerID"=>-1,
                "taxCollected"=>0
            );
            $i++;
            $minID = $row{'ApplicationID'};
        }
    }


    $imp->endTimer();
    $imp->startTimer("Inserting $i Apps");

    if(count($inserts) > 0){
         $imp->dest->dbBulkInsert("appliedPayments", $inserts);
    }

    unset($data);
    unset($inserts);
    echo "Inserted $i Apps<BR>";
}  

It doesn't matter what I set the limit to, the processing portion takes twice as long each time. I am logging each portion of the loop and selecting the data from the old database and inserting it into the new one take no time at all. The "processing portion" is doubling every time. Why? Here are the logs, if you do some quick math on the timestamps, each step labeled "Processing Apps" takes twice as long as the one before... (I stopped it a little early on this one, but it was taking a significantly longer time on the final iteration)

logs

Well - so I don't know why this works, but if I move everything inside the while loop into a separate function, it DRAMATICALLY increases performance. Im guessing its a garbage collection / memory management issue and that having a function call end helps the Garbage collector know it can release the memory. Now when I log the memory usage, the memory usage stays constant between calls instead of growing... Dirty php...