php分裂字符串在块ngrams unicode char问题

问题描述：

I try to generate n grams from string in PHP For that I use this function from : https://gist.github.com/Xeoncross/5366393

function Bigrams($word){
    $ngrams = array();
    $len = strlen($word);
    for($i=0;$i+1<$len;$i++){
        $ngrams[$i]=$word[$i].$word[$i+1];
    }
    return $ngrams;
}

$word = "abcdefg";

print_r(Bigrams($word));

That OK return as expected ngrams :

[0] => ab
[1] => bc
[2] => cd
[3] => de
[4] => ef
[5] => fg

But for certain Unicode characters not return as expected:

Ex: for $word = "Lòria" return:

[0] => L�
[1] => ò
[2] => �r
[3] => ri

Or for $word = "пожалуйста" return:

[0] => п
[1] => ��
[2] => о
[3] => ��
[4] => ж
[5] => ��
[6] => а
[7] => ��
[8] => л

Any idea how to solve this?

我尝试在PHP中从字符串生成n克为此我使用以下函数： https://gist.github.com/Xeoncross/5366393 p>

  function Bigrams（$ word）{
 $ ngrams = array（）; 
 $ len = strlen（$ word）; 
 for（$ i = 0; $ i + 1＆lt; $ len; $ i ++）{\  n $ ngrams [$ i] = $ word [$ i]。$ word [$ i + 1]; 
} 
返回$ ngrams; 
} 
 
 $ word =“abcdefg”; 
 \  nprint_r（Bigrams（$ word））; 
  code>  pre> 
 
 可以按预期返回ngrams： p> 
 
 
  [0]  =＆GT;  ab 
 [1] =＆gt;  bc 
 [2] =＆gt;  cd 
 [3] =＆gt;  de 
 [4] =＆gt;  ef 
 [5] =＆gt;  fg 
  code>  pre> 
 
 但是对于某些Unicode字符不能按预期返回： p> 
 
 
 Ex：for $ word =“Lòria”return：   p> 
 
 
  [0] =＆gt;  L�
 [1] =＆gt;  ò
 [2] =＆gt;  �r
 [3] =＆gt;  ri 
  code>  pre> 
 
 或$ word =“пожалуйста”返回： p> 
 
 
  [0] =＆gt;  п
 [1] =＆gt;  ��
 [2] =＆gt;  о
 [3] =＆gt;  ��
 [4] =＆gt;  ж
 [5] =＆gt;  ��
 [6] =＆gt;  а
 [7] =＆gt;  ��
 [8] =＆gt;  л
  code>  pre> 
 
 知道如何解决这个问题吗？ p> 
  div>

答

use unicode oriented string functions

function Bigrams($word){
    $ngrams = array();
    $len = mb_strlen($word);
    for($i=0;$i+1<$len;$i++){
        $ngrams[$i]=mb_substr($word, $i, 2);
    }
    return $ngrams;
}

$word = "пожалуйста";

print_r(Bigrams($word));

result

Array
(
    [0] => по
    [1] => ож
    [2] => жа
    [3] => ал
    [4] => лу
    [5] => уй
    [6] => йс
    [7] => ст
    [8] => та
)

php分裂字符串在块ngrams unicode char问题

相关推荐