memcached鍏ㄩ潰鍓栨瀽鈥? memcached鐨勫簲鐢ㄥ拰鍏煎绋嬪簭

鍙戣〃鏃ワ細2008/7/30
浣滆€咃細闀块噹闆呭箍(Masahiro Nagano)
鍘熸枃閾炬帴锛?a href="http://gihyo.jp/dev/feature/01/memcached/0005">http://gihyo.jp/dev/feature/01/memcached/0005

<!-- end Pukiwiki generated code-->

鎴戞槸Mixi鐨勯暱閲庛€俶emcached鐨勮繛杞界粓浜庤缁撴潫浜嗐€傚埌涓婃涓烘锛屾垜浠粙缁嶄簡涓巑emcached鐩存帴鐩稿叧鐨勮瘽棰橈紝鏈浠嬬粛涓€浜沵ixi鐨勬渚嬪拰瀹為檯搴旂敤涓婄殑璇濋锛屽苟浠嬬粛涓€浜涗笌memcached鍏煎鐨勭▼搴忋€?/p>

<!-- begin Pukiwiki generated code-->

mixi妗堜緥鐮旂┒

mixi鍦ㄦ彁渚涙湇鍔＄殑鍒濇湡闃舵灏变娇鐢ㄤ簡memcached銆傞殢鐫€缃戠珯璁块棶閲忕殑鎬ュ墽澧炲姞锛屽崟绾负鏁版嵁搴撴坊鍔爏lave宸叉棤娉曟弧瓒抽渶瑕侊紝鍥犳寮曞叆浜唌emcached銆傛澶栵紝鎴戜滑涔熶粠澧炲姞鍙墿灞曟€х殑鏂归潰杩涜浜嗛獙璇侊紝璇佹槑浜唌emcached鐨勯€熷害鍜岀ǔ瀹氭€ч兘鑳芥弧瓒抽渶瑕併€傜幇鍦紝memcached宸叉垚涓簃ixi鏈嶅姟涓潪甯搁噸瑕佺殑缁勬垚閮ㄥ垎銆?/p>

鍥? 鐜板湪鐨勭郴缁熺粍浠?/p>

鏈嶅姟鍣ㄩ厤缃拰鏁伴噺

mixi浣跨敤浜嗚璁稿澶氭湇鍔″櫒锛屽鏁版嵁搴撴湇鍔″櫒銆佸簲鐢ㄦ湇鍔″櫒銆佸浘鐗囨湇鍔″櫒銆佸弽鍚戜唬鐞嗘湇鍔″櫒绛夈€傚崟鍗昺emcached灏辨湁灏嗚繎200鍙版湇鍔″櫒鍦ㄨ繍琛屻€?memcached鏈嶅姟鍣ㄧ殑鍏稿瀷閰嶇疆濡備笅锛?/p>

• CPU锛欼ntel Pentium 4 2.8GHz
• 鍐呭瓨锛?GB
• 纭洏锛?46GB SCSI
• 鎿嶄綔绯荤粺锛歀inux锛坸86_64锛?/li>

杩欎簺鏈嶅姟鍣ㄤ互鍓嶆浘鐢ㄤ簬鏁版嵁搴撴湇鍔″櫒绛夈€傞殢鐫€CPU鎬ц兘鎻愬崌銆佸唴瀛樹环鏍间笅闄嶏紝鎴戜滑绉瀬鍦板皢鏁版嵁搴撴湇鍔″櫒銆佸簲鐢ㄦ湇鍔″櫒绛夋崲鎴愪簡鎬ц兘鏇村己澶с€佸唴瀛樻洿澶氱殑鏈嶅姟鍣ㄣ€傝繖鏍凤紝鍙互鎶戝埗mixi鏁翠綋浣跨敤鐨勬湇鍔″櫒鏁伴噺鐨勬€ュ墽澧炲姞锛岄檷浣庣鐞嗘垚鏈€傜敱浜巑emcached鏈嶅姟鍣ㄥ嚑涔庝笉鍗犵敤CPU锛屽氨灏嗘崲涓嬫潵鐨勬湇鍔″櫒鐢ㄤ綔memcached鏈嶅姟鍣ㄤ簡銆?/p>

memcached杩涚▼

姣忓彴memcached鏈嶅姟鍣ㄤ粎鍚姩涓€涓猰emcached杩涚▼銆傚垎閰嶇粰memcached鐨勫唴瀛樹负3GB锛屽惎鍔ㄥ弬鏁板涓嬶細

/usr/bin/memcached -p 11211 -u nobody -m 3000 -c 30720

鐢变簬浣跨敤浜唜86_64鐨勬搷浣滅郴缁燂紝鍥犳鑳藉垎閰?GB浠ヤ笂鐨勫唴瀛樸€?2浣嶆搷浣滅郴缁熶腑锛屾瘡涓繘绋嬫渶澶氬彧鑳戒娇鐢?GB鍐呭瓨銆備篃鏇剧粡鑰冭檻杩囧惎鍔ㄥ涓垎閰?GB浠ヤ笅鍐呭瓨鐨勮繘绋嬶紝浣嗚繖鏍蜂竴鍙版湇鍔″櫒涓婄殑TCP杩炴帴鏁板氨浼氭垚鍊嶅鍔狅紝绠＄悊涓婁篃鍙樺緱澶嶆潅锛屾墍浠ixi灏辩粺涓€浣跨敤浜?4浣嶆搷浣滅郴缁熴€?/p>

鍙﹀锛岃櫧鐒舵湇鍔″櫒鐨勫唴瀛樹负4GB锛屽嵈浠呭垎閰嶄簡3GB锛屾槸鍥犱负鍐呭瓨鍒嗛厤閲忚秴杩囪繖涓€硷紝灏辨湁鍙兘瀵艰嚧鍐呭瓨浜ゆ崲(swap)銆傝繛杞界殑绗?娆?/a>涓墠鍧傝瑙ｈ繃浜唌emcached鐨勫唴瀛樺瓨鍌ㄢ€渟lab allocator鈥濓紝褰撴椂璇磋繃锛宮emcached鍚姩鏃舵寚瀹氱殑鍐呭瓨鍒嗛厤閲忔槸memcached鐢ㄤ簬淇濆瓨鏁版嵁鐨勯噺锛屾病鏈夊寘鎷€渟lab allocator鈥濇湰韬崰鐢ㄧ殑鍐呭瓨銆佷互鍙婁负浜嗕繚瀛樻暟鎹€岃缃殑绠＄悊绌洪棿銆傚洜姝わ紝memcached杩涚▼鐨勫疄闄呭唴瀛樺垎閰嶉噺瑕佹瘮鎸囧畾鐨勫閲忚澶э紝杩欎竴鐐瑰簲褰撴敞鎰忋€?/p>

mixi淇濆瓨鍦╩emcached涓殑鏁版嵁澶ч儴鍒嗛兘姣旇緝灏忋€傝繖鏍凤紝杩涚▼鐨勫ぇ灏忚姣旀寚瀹氱殑瀹归噺澶у緢澶氥€傚洜姝わ紝鎴戜滑鍙嶅鏀瑰彉鍐呭瓨鍒嗛厤閲忚繘琛岄獙璇侊紝纭浜?GB鐨勫ぇ灏忎笉浼氬紩鍙憇wap锛岃繖灏辨槸鐜板湪搴旂敤鐨勬暟鍊笺€?/p>

memcached浣跨敤鏂规硶鍜屽鎴风

鐜板湪锛宮ixi鐨勬湇鍔″皢200鍙板乏鍙崇殑memcached鏈嶅姟鍣ㄤ綔涓轰竴涓猵ool浣跨敤銆傛瘡鍙版湇鍔″櫒鐨勫閲忎负3GB锛岄偅涔堝叏浣撳氨鏈変簡灏嗚繎600GB鐨勫法澶х殑鍐呭瓨鏁版嵁搴撱€傚鎴风绋嬪簭搴撲娇鐢ㄤ簡鏈繛杞戒腑澶氭鎻愬埌杞︾殑Cache::Memcached::Fast锛屼笌鏈嶅姟鍣ㄨ繘琛屼氦浜掋€傚綋鐒讹紝缂撳瓨鐨勫垎甯冨紡绠楁硶浣跨敤鐨勬槸 绗?娆?/a>浠嬬粛杩囩殑 Consistent Hashing绠楁硶銆?/p>

• Cache::Memcached::Fast - search.cpan.org

搴旂敤灞備笂memcached鐨勪娇鐢ㄦ柟娉曠敱寮€鍙戝簲鐢ㄧ▼搴忕殑宸ョ▼甯堣嚜琛屽喅瀹氬苟瀹炵幇銆備絾鏄紝涓轰簡闃叉杞﹁疆鍐嶉€犮€侀槻姝ache::Memcached::Fast涓婄殑鏁欒鍐嶆鍙戠敓锛屾垜浠彁渚涗簡Cache::Memcached::Fast鐨剋rap妯″潡骞朵娇鐢ㄣ€?/p>

閫氳繃Cache::Memcached::Fast缁存寔杩炴帴

Cache::Memcached鐨勬儏鍐典笅锛屼笌memcached鐨勮繛鎺ワ紙鏂囦欢鍙ユ焺锛変繚瀛樺湪Cache::Memcached鍖呭唴鐨勭被鍙橀噺涓€傚湪mod_perl鍜孎astCGI绛夌幆澧冧笅锛屽寘鍐呯殑鍙橀噺涓嶄細鍍廋GI閭ｆ牱闅忔椂閲嶆柊鍚姩锛岃€屾槸鍦ㄨ繘绋嬩腑涓€鐩翠繚鎸併€傚叾缁撴灉灏辨槸涓嶄細鏂紑涓巑emcached鐨勮繛鎺ワ紝鍑忓皯浜員CP杩炴帴寤虹珛鏃剁殑寮€閿€锛屽悓鏃朵篃鑳介槻姝㈢煭鏃堕棿鍐呭弽澶嶈繘琛孴CP杩炴帴銆佹柇寮€鑰屽鑷寸殑TCP绔彛璧勬簮鏋銆?/p>

浣嗘槸锛孋ache::Memcached::Fast娌℃湁杩欎釜鍔熻兘锛屾墍浠ラ渶瑕佸湪妯″潡涔嬪灏咰ache::Memcached::Fast瀵硅薄淇濇寔鍦ㄧ被鍙橀噺涓紝浠ヤ繚璇佹寔涔呰繛鎺ャ€?/p>

package Gihyo::Memcached;

use strict;
use warnings;
use Cache::Memcached::Fast;

my @server_list = qw/192.168.1.1:11211 192.168.1.1:11211/;
my $fast;  ## 鐢ㄤ簬淇濇寔瀵硅薄

sub new {
    my $self  = bless {}, shift;
    if ( !$fast ) {
        $fast = Cache::Memcached::Fast->new({ servers => \@server_list });
    }
    $self->{_fast} = $fast;
    return $self;
}

sub get {
   my $self = shift;
   $self->{_fast}->get(@_);
}

涓婇潰鐨勪緥瀛愪腑锛孋ache::Memcached::Fast瀵硅薄淇濆瓨鍒扮被鍙橀噺$fast涓€?/p>

鍏叡鏁版嵁鐨勫鐞嗗拰rehash

璇稿mixi鐨勪富椤典笂鐨勬柊闂昏繖鏍风殑鎵€鏈夌敤鎴峰叡浜殑缂撳瓨鏁版嵁銆佽缃俊鎭瓑鏁版嵁锛屼細鍗犵敤璁稿椤碉紝璁块棶娆℃暟涔熼潪甯稿銆傚湪杩欑鏉′欢涓嬶紝璁块棶寰堝鏄撻泦涓埌鏌愬彴memcached鏈嶅姟鍣ㄤ笂銆傝闂泦涓湰韬苟涓嶆槸闂锛屼絾鏄竴鏃﹁闂泦涓殑閭ｅ彴鏈嶅姟鍣ㄥ彂鐢熸晠闅滃鑷磎emcached鏃犳硶杩炴帴锛屽氨浼氫骇鐢熷法澶х殑闂銆?/p>

杩炶浇鐨?a href="http://tech.idv2.com/2008/07/24/memcached-004/">绗?娆?/a> 涓彁鍒帮紝Cache::Memcached鎷ユ湁rehash鍔熻兘锛屽嵆鍦ㄦ棤娉曡繛鎺ヤ繚瀛樻暟鎹殑鏈嶅姟鍣ㄧ殑鎯呭喌涓嬶紝浼氬啀娆¤绠梙ash鍊硷紝杩炴帴鍏朵粬鐨勬湇鍔″櫒銆?/p>

浣嗘槸锛孋ache::Memcached::Fast娌℃湁杩欎釜鍔熻兘銆備笉杩囷紝瀹冭兘澶熷湪杩炴帴鏈嶅姟鍣ㄥけ璐ユ椂锛岀煭鏃堕棿鍐呬笉鍐嶈繛鎺ヨ鏈嶅姟鍣ㄧ殑鍔熻兘銆?/p>

my $fast = Cache::Memcached::Fast->new({
    max_failures     => 3,
    failure_timeout  => 1
});

鍦╢ailure_timeout绉掑唴鍙戠敓max_failures浠ヤ笂娆¤繛鎺ュけ璐ワ紝灏变笉鍐嶈繛鎺ヨmemcached鏈嶅姟鍣ㄣ€傛垜浠殑璁剧疆鏄?绉掗挓3娆′互涓娿€?/p>

姝ゅ锛宮ixi杩樹负鎵€鏈夌敤鎴峰叡浜殑缂撳瓨鏁版嵁鐨勯敭鍚嶈缃懡鍚嶈鍒欙紝绗﹀悎鍛藉悕瑙勫垯鐨勬暟鎹細鑷姩淇濆瓨鍒板鍙癿emcached鏈嶅姟鍣ㄤ腑锛屽彇寰楁椂浠庝腑浠呴€夊彇涓€鍙版湇鍔″櫒銆傚垱寤鸿鍑芥暟搴撳悗锛屽氨鍙互浣縨emcached鏈嶅姟鍣ㄦ晠闅滀笉鍐嶄骇鐢熷叾浠栧奖鍝嶃€?/p>

memcached搴旂敤缁忛獙

鍒版涓烘浠嬬粛浜唌emcached鍐呴儴鏋勯€犲拰鍑芥暟搴擄紝鎺ヤ笅鏉ヤ粙缁嶄竴浜涘叾浠栫殑搴旂敤缁忛獙銆?/p>

閫氳繃daemontools鍚姩

閫氬父鎯呭喌涓媘emcached杩愯寰楃浉褰撶ǔ瀹氾紝浣唌ixi鐜板湪浣跨敤鐨勬渶鏂扮増1.2.5 鏇剧粡鍙戠敓杩囧嚑娆emcached杩涚▼姝绘帀鐨勬儏鍐点€傛灦鏋勪笂淇濊瘉浜嗗嵆浣挎湁鍑犲彴memcached鏁呴殰涔熶笉浼氬奖鍝嶆湇鍔★紝涓嶈繃瀵逛簬memcached杩涚▼姝绘帀鐨勬湇鍔″櫒锛屽彧瑕侀噸鏂板惎鍔╩emcached锛屽氨鍙互姝ｅ父杩愯锛屾墍浠ラ噰鐢ㄤ簡鐩戣memcached杩涚▼骞惰嚜鍔ㄥ惎鍔ㄧ殑鏂规硶銆備簬鏄娇鐢ㄤ簡daemontools銆?/p>

daemontools鏄痲mail鐨勪綔鑰匘JB寮€鍙戠殑UNIX鏈嶅姟绠＄悊宸ュ叿闆嗭紝鍏朵腑鍚嶄负supervise鐨勭▼搴忓彲鐢ㄤ簬鏈嶅姟鍚姩銆佸仠姝㈢殑鏈嶅姟閲嶅惎绛夈€?/p>

• daemontools

杩欓噷涓嶄粙缁峝aemontools鐨勫畨瑁呬簡銆俶ixi浣跨敤浜嗕互涓嬬殑run鑴氭湰鏉ュ惎鍔╩emcached銆?/p>

#!/bin/sh

if [ -f /etc/sysconfig/memcached ];then
        . /etc/sysconfig/memcached
fi

exec 2>&1
exec /usr/bin/memcached -p $PORT -u $USER  -m $CACHESIZE -c $MAXCONN $OPTIONS

鐩戣

mixi浣跨敤浜嗗悕涓衡€渘agios鈥濈殑寮€婧愮洃瑙嗚蒋浠舵潵鐩戣memcached銆?/p>

• Nagios: Home

鍦╪agios涓彲浠ョ畝鍗曞湴寮€鍙戞彃浠讹紝鍙互璇︾粏鍦扮洃瑙唌emcached鐨刧et銆乤dd绛夊姩浣溿€備笉杩噈ixi浠呴€氳繃stats鍛戒护鏉ョ‘璁emcached鐨勮繍琛岀姸鎬併€?/p>

define command {
command_name                   check_memcached
command_line                   $USER1$/check_tcp -H $HOSTADDRESS$ -p 11211 -t 5 -E -s 'stats\r\nquit\r\n' -e 'uptime' -M crit
}

姝ゅ锛宮ixi灏唖tats鐩綍鐨勭粨鏋滈€氳繃rrdtool杞寲鎴愬浘褰紝杩涜鎬ц兘鐩戣锛屽苟灏嗘瘡澶╃殑鍐呭瓨浣跨敤閲忓仛鎴愭姤琛紝閫氳繃閭欢涓庡紑鍙戣€呭叡浜€?/p>

memcached鐨勬€ц兘

杩炶浇涓凡浠嬬粛杩囷紝memcached鐨勬€ц兘鍗佸垎浼樼銆傛垜浠潵鐪嬬湅mixi鐨勫疄闄呮渚嬨€傝繖閲屼粙缁嶇殑鍥捐〃鏄湇鍔℃墍浣跨敤鐨勮闂渶涓洪泦涓殑memcached鏈嶅姟鍣ㄣ€?/p>

鍥? 璇锋眰鏁?/p>

鍥? 娴侀噺

鍥? TCP杩炴帴鏁?/p>

浠庝笂鑷充笅渚濇涓鸿姹傛暟銆佹祦閲忓拰TCP杩炴帴鏁般€傝姹傛暟鏈€澶т负15000qps锛屾祦閲忚揪鍒?00Mbps锛岃繖鏃剁殑杩炴帴鏁板凡瓒呰繃浜?0000涓€傝鏈嶅姟鍣ㄦ病鏈夌壒鍒殑纭欢锛屽氨鏄紑澶翠粙缁嶇殑鏅€氱殑memcached鏈嶅姟鍣ㄣ€傛鏃剁殑CPU鍒╃敤鐜囦负锛?/p>

鍥? CPU鍒╃敤鐜?/p>

鍙锛屼粛鐒舵湁idle鐨勯儴鍒嗐€傚洜姝わ紝memcached鐨勬€ц兘闈炲父楂橈紝鍙互浣滀负Web搴旂敤绋嬪簭寮€鍙戣€呮斁蹇冨湴淇濆瓨涓存椂鏁版嵁鎴栫紦瀛樻暟鎹殑鍦版柟銆?/p>

鍏煎搴旂敤绋嬪簭

memcached鐨勫疄鐜板拰鍗忚閮藉崄鍒嗙畝鍗曪紝鍥犳鏈夊緢澶氫笌memcached鍏煎鐨勫疄鐜般€備竴浜涘姛鑳藉己澶х殑鎵╁睍鍙互灏唌emcached鐨勫唴瀛樻暟鎹啓鍒扮鐩樹笂锛屽疄鐜版暟鎹殑鎸佷箙鎬у拰鍐椾綑銆傝繛杞?a href="http://tech.idv2.com/2008/07/16/memcached-003/">绗?娆?/a> 浠嬬粛杩囷紝浠ュ悗鐨刴emcached鐨勫瓨鍌ㄥ眰灏嗗彉鎴愬彲鎵╁睍鐨勶紙pluggable锛夛紝閫愭笎鏀寔杩欎簺鍔熻兘銆?/p>

杩欓噷浠嬬粛鍑犱釜涓巑emcached鍏煎鐨勫簲鐢ㄧ▼搴忋€?/p>