九瀛︿範鍒嗗竷寮忔灦鏋?KafKa浠嬬粛 Dubbo zookeeper springmvc mybatis shiro maven

鏍稿績鎶€鏈細Maven,Springmvc mybatis shiro, Druid, Restful, Dubbo, ZooKeeper,Redis,FastDFS,ActiveMQ,Nginx
浠嬬粛

Kafka鏄竴涓垎甯冨紡鐨勩€佸彲鍒嗗尯鐨勩€佸彲澶嶅埗鐨勬秷鎭郴缁熴€傚畠鎻愪緵浜嗘櫘閫氭秷鎭郴缁熺殑鍔熻兘锛屼絾鍏锋湁鑷繁鐙壒鐨勮璁°€傝繖涓嫭鐗圭殑璁捐鏄粈涔堟牱鐨勫憿锛?/span>

棣栧厛璁╂垜浠湅鍑犱釜鍩烘湰鐨勬秷鎭郴缁熸湳璇細

• Kafka灏嗘秷鎭互topic涓哄崟浣嶈繘琛屽綊绾炽€?/span>

• 灏嗗悜Kafka topic鍙戝竷娑堟伅鐨勭▼搴忔垚涓?strong>producers.

• 灏嗛璁opics骞舵秷璐规秷鎭殑绋嬪簭鎴愪负consumer.

• Kafka浠ラ泦缇ょ殑鏂瑰紡杩愯锛屽彲浠ョ敱涓€涓垨澶氫釜鏈嶅姟缁勬垚锛屾瘡涓湇鍔″彨鍋氫竴涓?strong>broker.

producers閫氳繃缃戠粶灏嗘秷鎭彂閫佸埌Kafka闆嗙兢锛岄泦缇ゅ悜娑堣垂鑰呮彁渚涙秷鎭紝濡備笅鍥炬墍绀猴細

瀹㈡埛绔拰鏈嶅姟绔€氳繃TCP鍗忚閫氫俊銆侹afka鎻愪緵浜咼ava瀹㈡埛绔紝骞朵笖瀵瑰绉嶈瑷€閮芥彁渚涗簡鏀寔銆?/span>

聽Topics 鍜孡ogs

鍏堟潵鐪嬩竴涓婯afka鎻愪緵鐨勪竴涓娊璞℃蹇?topic.

涓€涓猼opic鏄涓€缁勬秷鎭殑褰掔撼銆傚姣忎釜topic锛孠afka 瀵瑰畠鐨勬棩蹇楄繘琛屼簡鍒嗗尯锛屽涓嬪浘鎵€绀猴細

姣忎釜鍒嗗尯閮界敱涓€绯诲垪鏈夊簭鐨勩€佷笉鍙彉鐨勬秷鎭粍鎴愶紝杩欎簺娑堟伅琚繛缁殑杩藉姞鍒板垎鍖轰腑銆傚垎鍖轰腑鐨勬瘡涓秷鎭兘鏈変竴涓繛缁殑搴忓垪鍙峰彨鍋歰ffset,鐢ㄦ潵鍦ㄥ垎鍖轰腑鍞竴鐨勬爣璇嗚繖涓秷鎭€?/span>

聽

鍦ㄤ竴涓彲閰嶇疆鐨勬椂闂存鍐咃紝Kafka闆嗙兢淇濈暀鎵€鏈夊彂甯冪殑娑堟伅锛屼笉绠¤繖浜涙秷鎭湁娌℃湁 琚秷璐广€傛瘮濡傦紝濡傛灉娑堟伅鐨勪繚瀛樼瓥鐣ヨ璁剧疆涓?澶╋紝閭ｄ箞鍦ㄤ竴涓秷 鎭鍙戝竷鐨勪袱澶╂椂闂村唴锛屽畠閮芥槸鍙互琚秷璐圭殑銆備箣鍚庡畠灏嗚涓㈠純浠ラ噴鏀剧┖闂淬€侹afka鐨勬€ц兘鏄拰鏁版嵁閲忔棤鍏崇殑甯搁噺绾х殑锛屾墍浠ヤ繚鐣欏お澶氱殑鏁版嵁骞朵笉鏄棶棰樸€?/span>

聽

瀹為檯涓婃瘡涓猚onsumer鍞竴闇€瑕佺淮鎶ょ殑鏁版嵁鏄秷鎭湪鏃ュ織涓殑浣嶇疆锛屼篃灏辨槸 offset.杩欎釜offset鏈塩onsumer鏉ョ淮鎶わ細涓€鑸儏 鍐典笅闅忕潃consumer涓嶆柇鐨勮鍙栨秷鎭紝杩檕ffset鐨勫€间笉鏂鍔狅紝浣嗗叾瀹瀋onsumer鍙互浠ヤ换鎰忕殑椤哄簭璇诲彇娑堟伅锛屾瘮濡傚畠鍙互灏唎ffset璁剧疆 鎴愪负涓€涓棫鐨勫€兼潵閲嶈涔嬪墠鐨勬秷鎭€?/span>

聽

浠ヤ笂鐗圭偣鐨勭粨鍚堬紝浣縆afka consumers闈炲父鐨勮交閲忕骇锛氬畠浠彲浠ュ湪涓嶅闆嗙兢鍜屽叾浠朿onsumer閫犳垚褰卞搷鐨勬儏鍐典笅璇诲彇娑堟伅銆備綘鍙互浣跨敤鍛戒护琛屾潵"tail"娑堟伅鑰屼笉浼氬鍏朵粬姝ｅ湪娑堣垂娑堟伅鐨刢onsumer閫犳垚褰卞搷銆?/span>

聽

灏嗘棩蹇楀垎鍖哄彲浠ヨ揪鍒颁互涓嬬洰鐨勶細棣栧厛杩欎娇寰楁瘡涓棩蹇楃殑鏁伴噺涓嶄細澶ぇ锛屽彲浠ュ湪鍗曚釜鏈嶅姟涓婁繚瀛樸€傚彟澶栨瘡涓垎鍖哄彲浠ュ崟鐙彂甯冨拰娑堣垂锛屼负骞跺彂鎿嶄綔topic鎻愪緵浜嗕竴绉嶅彲鑳姐€?/span>

鍒嗗竷寮?/span>

姣忎釜鍒嗗尯鍦↘afka闆嗙兢鐨勮嫢骞叉湇鍔′腑閮芥湁鍓湰锛岃繖鏍疯繖浜涙寔鏈夊壇鏈殑鏈嶅姟鍙互鍏卞悓澶勭悊鏁版嵁鍜岃姹傦紝鍓湰鏁伴噺鏄彲浠ラ厤缃殑銆傚壇鏈娇Kafka鍏峰浜嗗閿欒兘鍔涖€?/span>

姣忎釜鍒嗗尯閮界敱涓€涓湇鍔″櫒浣滀负鈥渓eader鈥濓紝闆舵垨鑻ュ共鏈嶅姟鍣ㄤ綔涓?鈥渇ollowers鈥?leader璐熻矗澶勭悊娑堟伅鐨勮鍜?鍐欙紝followers鍒欏幓澶嶅埗leader.濡傛灉leader down浜嗭紝followers涓殑涓€鍙板垯浼氳嚜鍔ㄦ垚涓簂eader銆傞泦缇や腑鐨勬瘡涓湇鍔￠兘浼氬悓鏃舵壆婕斾袱涓鑹诧細浣滀负瀹冩墍鎸佹湁鐨勪竴閮ㄥ垎鍒嗗尯鐨刲eader锛屽悓 鏃朵綔涓哄叾浠栧垎鍖虹殑followers锛岃繖鏍烽泦缇ゅ氨浼氭嵁鏈夎緝濂界殑璐熻浇鍧囪　銆?/span>

Producers

Producer灏嗘秷鎭彂甯冨埌瀹冩寚瀹氱殑topic涓?骞惰礋璐ｅ喅瀹氬彂甯冨埌鍝釜鍒嗗尯銆傞€氬父绠€鍗曠殑鐢辫礋杞藉潎琛℃満鍒堕殢鏈洪€夋嫨鍒嗗尯锛屼絾涔熷彲浠ラ€氳繃鐗瑰畾鐨勫垎鍖哄嚱鏁伴€夋嫨鍒嗗尯銆備娇鐢ㄧ殑鏇村鐨勬槸绗簩绉嶃€?/span>

Consumers

鍙戝竷娑堟伅閫氬父鏈変袱绉嶆ā寮忥細闃熷垪妯″紡鍜屽彂甯?璁㈤槄妯″紡銆傞槦鍒楁ā寮忎腑锛宑onsumers鍙互鍚屾椂浠庢湇鍔＄璇诲彇娑堟伅锛屾瘡涓秷鎭彧琚叾涓竴涓猚onsumer璇诲埌锛涘彂甯?璁㈤槄妯″紡涓秷鎭骞挎挱鍒版墍鏈夌殑consumer涓€?/span>

Consumers鍙互鍔犲叆涓€涓?consumer 缁勶紝鍏卞悓绔炰簤涓€涓猼opic锛宼opic涓殑娑堟伅灏嗚鍒嗗彂鍒扮粍涓殑涓€涓垚鍛樹腑銆傚悓涓€缁勪腑鐨刢onsumer鍙互鍦ㄤ笉鍚岀殑绋嬪簭涓紝涔熷彲浠ュ湪涓嶅悓鐨勬満鍣ㄤ笂銆傚 鏋滄墍鏈夌殑consumer閮藉湪涓€涓粍涓紝杩欏氨鎴愪负浜嗕紶缁熺殑闃熷垪妯″紡锛屽湪鍚刢onsumer涓疄鐜拌礋杞藉潎琛°€?/span>

濡傛灉鎵€鏈夌殑consumer閮戒笉鍦ㄤ笉鍚岀殑缁勪腑锛岃繖灏辨垚涓轰簡鍙戝竷-璁㈤槄妯″紡锛屾墍鏈夌殑娑堟伅閮借鍒嗗彂鍒版墍鏈夌殑consumer涓€?/span>

鏇村父瑙佺殑鏄紝姣忎釜topic閮芥湁鑻ュ共鏁伴噺鐨刢onsumer缁勶紝姣忎釜缁勯兘鏄竴涓€昏緫涓婄殑鈥滆闃呰€呪€濓紝涓轰簡瀹归敊鍜屾洿濂界殑绋冲畾鎬э紝姣忎釜缁勭敱鑻ュ共consumer缁勬垚銆傝繖鍏跺疄灏辨槸涓€涓彂甯?璁㈤槄妯″紡锛屽彧涓嶈繃璁㈤槄鑰呮槸涓粍鑰屼笉鏄崟涓猚onsumer銆?/span>

鐢变袱涓満鍣ㄧ粍鎴愮殑闆嗙兢鎷ユ湁4涓垎鍖?(P0-P3) 2涓猚onsumer缁? A缁勬湁涓や釜consumerB缁勬湁4涓?/span>

聽

鐩告瘮浼犵粺鐨勬秷鎭郴缁燂紝Kafka鍙互寰堝ソ鐨勪繚璇佹湁搴忔€с€?/span>

浼犵粺鐨勯槦鍒楀湪鏈嶅姟鍣ㄤ笂淇濆瓨鏈夊簭鐨勬秷鎭紝濡傛灉澶氫釜consumers鍚屾椂浠庤繖涓湇鍔?鍣ㄦ秷璐规秷鎭紝鏈嶅姟鍣ㄥ氨浼氫互娑堟伅瀛樺偍鐨勯『搴忓悜consumer鍒?鍙戞秷鎭€傝櫧鐒舵湇鍔″櫒鎸夐『搴忓彂甯冩秷鎭紝浣嗘槸娑堟伅鏄寮傛鐨勫垎鍙戝埌鍚刢onsumer涓婏紝鎵€浠ュ綋娑堟伅鍒拌揪鏃跺彲鑳藉凡缁忓け鍘讳簡鍘熸潵鐨勯『搴忥紝杩欐剰鍛崇潃骞跺彂娑堣垂灏嗗鑷?椤哄簭閿欎贡銆備负浜嗛伩鍏嶆晠闅滐紝杩欐牱鐨勬秷鎭郴缁熼€氬父浣跨敤鈥滀笓鐢╟onsumer鈥濈殑姒傚康锛屽叾瀹炲氨鏄彧鍏佽涓€涓秷璐硅€呮秷璐规秷鎭紝褰撶劧杩欏氨鎰忓懗鐫€澶卞幓浜嗗苟鍙戞€с€?/span>

聽

鍦ㄨ繖鏂归潰Kafka鍋氱殑鏇村ソ锛岄€氳繃鍒嗗尯鐨勬蹇碉紝Kafka鍙互鍦ㄥ涓?consumer缁勫苟鍙戠殑鎯呭喌涓嬫彁渚涜緝濂界殑鏈夊簭鎬у拰璐熻浇鍧囪　銆傚皢姣忎釜鍒嗗尯鍒?鍙垎鍙戠粰涓€涓猚onsumer缁勶紝杩欐牱涓€涓垎鍖哄氨鍙杩欎釜缁勭殑涓€涓猚onsumer娑堣垂锛屽氨鍙互椤哄簭鐨勬秷璐硅繖涓垎鍖虹殑娑堟伅銆傚洜涓烘湁澶氫釜鍒嗗尯锛屼緷鐒跺彲浠ュ湪澶?涓猚onsumer缁勪箣闂磋繘琛岃礋杞藉潎琛°€傛敞鎰廲onsumer缁勭殑鏁伴噺涓嶈兘澶氫簬鍒嗗尯鐨勬暟閲忥紝涔熷氨鏄湁澶氬皯鍒嗗尯灏卞厑璁稿灏戝苟鍙戞秷璐广€?/span>

聽

Kafka鍙兘淇濊瘉涓€涓垎鍖轰箣鍐呮秷鎭殑鏈夊簭鎬э紝鍦ㄤ笉鍚岀殑鍒嗗尯涔嬮棿鏄笉鍙互鐨勶紝杩欏凡缁忓彲浠ユ弧瓒冲ぇ閮ㄥ垎搴旂敤鐨勯渶姹傘€傚鏋滈渶瑕乼opic涓墍鏈夋秷鎭殑鏈夊簭鎬э紝閭ｅ氨鍙兘璁╄繖涓猼opic鍙湁涓€涓垎鍖猴紝褰撶劧涔熷氨鍙湁涓€涓猚onsumer缁勬秷璐瑰畠銆?/span>