嵌套数组中子项的递归计数+通过引用更改值

I know this is basic recursion but I get stuck anyway :(

I need to count how many elements each element has below it (children, grandchildren,...) and write that value into original array.

My example array:

$table = [
1 => [
    'id' => 1,
    'children_count' => 0
],
2 => [
    'id' => 2,
    'children_count' => 0,
    'children' => [
        3 => [
            'id' => 3,
            'children_count' => 0,
            'children' => [
                4 => [
                    'id' => 4,
                    'children_count' => 0,
                    'children' => [
                        5 => [
                            'id' => 5,
                            'children_count' => 0
                        ],

                        6 => [
                            'id' => 6,
                            'children_count' => 0
                        ]
                    ]
                ]
            ]
        ]
    ]
]    
];

My recursion code:

function count_children(&$array){
    foreach($array['children'] as &$child){
        if(isset($child['children'])){
            $array['children_count'] += count_children($child);
        }
        else return 1;
    }
}

Call for recursion:

//call for each root element
foreach($table as &$element){
    if(isset($element['children'])) count_children($element);
}

Expected output:

$table = [
1 => [
'id' => 1,
'children_count' => 0
 ],
 2 => [
'id' => 2,
'children_count' => 4,
'children' => [
    3 => [
        'id' => 3,
        'children_count' => 3,
        'children' => [
            4 => [
                'id' => 4,
                'children_count' => 2,
                'children' => [
                    5 => [
                        'id' => 5,
                        'children_count' => 0
                    ],

                    6 => [
                        'id' => 6,
                        'children_count' => 0
                    ]
                ]
            ]
        ]
    ]
]
]    
];

Where did I got it wrong? My function does something, element 3 gets value 1, but thats about it.

Here is the ideone link: http://ideone.com/LOnl3G