Hadoop源码分析30 JobInProgress 的 TaskInProgress 执行情况

 

提交命令行:hadoop  jar /opt/hadoop-1.0.0/hadoop-examples-1.0.0.jar wordcount /user/admin/in/yellow.txt /user/admin/out/3

JobTracker已生成JobInProgress(job_201404200630_0001),包括以下TaskInProgress

maps[0](task_201404200630_0001_m_000000)

maps[1](task_201404200630_0001_m_000001)

maps[2](task_201404200630_0001_m_000002)

reduces[0](task_201404200630_0001_r_000000)

reduces[1](task_201404200630_0001_r_000001)

cleanup[0]task_201404200630_0001_m_000003mapcleanup

cleanup[1]task_201404200630_0001_r_000002reducecleanup

setup[0]task_201404200630_0001_m_000004mapsetup

setup[1]task_201404200630_0001_r_000003reducesetup

 

RPC请求:heartbeat from10.1.1.102

处理:获得setup[0]TaskInProgressattempt_201404200630_0001_m_000004,封装成MapTaskattempt_201404200630_0001_m_000004_0),再封装成LaunchTaskAction

返回:1LaunchTaskAction(setup[0])

 

RPC请求:heartbeat from10.1.1.103

处理:没有获得Task

返回: []

 

RPC请求:heartbeat from10.1.1.102

处理: Task attempt_201404200630_0001_m_000004)成功——setup[0]task_201404200630_0001_m_000004成功

获得maps[0](task_201404200630_0001_m_000000),封装成MapTaskattempt_201404200630_0001_m_000000_0,再封装成LaunchTaskAction

获得maps[1](task_201404200630_0001_m_000001),封装成MapTaskattempt_201404200630_0001_m_000001_0,再封装成LaunchTaskAction

获得setup[0](attempt_201404200630_0001_m_000004_0),再封装成KillTaskAction

返回:2LaunchTaskActionmaps[0],maps[1]),1KillTaskActionsetup[0]

 

RPC请求:heartbeat from10.1.1.103

处理:获得maps[2](task_201404200630_0001_m_000002),封装成MapTaskattempt_201404200630_0001_m_000002_0,再封装成LaunchTaskAction

返回: 1LaunchTaskActionmaps[2])

 

RPC请求:heartbeat from10.1.1.102

处理: Task attempt_201404200630_0001_m_000000_0)成功——maps[0]task_201404200630_0001_m_000009成功

Task attempt_201404200630_0001_m_000001_0)成功——maps[1]task_201404200630_0001_m_000001成功

获得maps[2](task_201404200630_0001_m_000002),封装成MapTaskattempt_201404200630_0001_m_000002_1),再封装成LaunchTaskAction

获得reduces[0]task_201404200630_0001_r_000000),封装成ReduceTaskattempt_201404200630_0001_r_000000_0),再封装成LaunchTaskAction

返回:2LaunchTaskActionmaps[2]reduces[0])

 

RPC请求:heartbeat from10.1.1.103

处理:Task attempt_201404200630_0001_m_000002_0)成功——maps[2]task_201404200630_0001_m_000002成功

获得reduces[1]task_201404200630_0001_r_000001),封装成ReduceTaskattempt_201404200630_0001_r_000001_0),再封装成LaunchTaskAction

返回:1LaunchTaskActionreduces[1])

 

RPC请求:heartbeat from10.1.1.102

处理:Task attempt_201404200630_0001_m_000002_1)成功——maps[2](task_201404200630_0001_m_000002)已经成功,故需要将其KILL

Task attempt_201404200630_0001_r_000000_0 RUNNING —— reduces[0](task_201404200630_0001_r_000000)RUNNING

返回:1KillTaskActionmaps[2]attempt_201404200630_0001_m_000002_1

 

RPC请求:heartbeat from10.1.1.103

处理:Task(attempt_201404200630_0001_r_000001_0COMMIT_PENDING--reduces[1]task_201404200630_0001_r_000001COMMIT_PENDING

返回:1CommitTaskAction(reduces[1],attempt_201404200630_0001_r_000001_0)

 

RPC请求:heartbeat from10.1.1.102

处理:Task(attempt_201404200630_0001_r_000000_0COMMIT_PENDING--reduces[0]task_201404200630_0001_r_000000COMMIT_PENDING

返回:1CommitTaskAction(reduces[0],attempt_201404200630_0001_r_000000_0)

 

RPC请求:heartbeat from10.1.1.103

处理:Task(attempt_201404200630_0001_r_000001_0SUCCEEDED--reduces[1]task_201404200630_0001_r_000001SUCCEEDED

返回:1KillTaskAction(reduces[1],attempt_201404200630_0001_r_000001_0)

 

RPC请求:heartbeat from10.1.1.102

处理:Task(attempt_201404200630_0001_r_000000_0SUCCEEDED--reduces[0]task_201404200630_0001_r_000000SUCCEEDED

返回:1LaunchTaskAction(cleanup[0],attempt_201404200630_0001_m_000003_0),1KillTaskAction(reduces[0],attempt_201404200630_0001_r_000000_0)

 

RPC请求:heartbeat from 10.1.1.103

处理:

返回:[]

 

RPC请求:heartbeat from10.1.1.102

处理:Task(attempt_201404200630_0001_m_000003_0SUCCEEDED--cleanup[0]task_201404200630_0001_m_000003SUCCEEDED

返回:KillJobActionKILL_JOB:job_201404200630_0001

 

RPC请求:heartbeat from10.1.1.103

处理:

返回:KillJobActionKILL_JOB:job_201404200630_0001

 

可见10.1.1.102执行了以下Task

setup[0]MapsetupLaunch-RUNNING-SUCCEED-KILL

maps[0]:Launch-RUNNING- SUCCEED

maps[1]:Launch-RUNNING- SUCCEED

maps[2]:Launch-RUNNING- SUCCEED-KILL

reduces[0]:Launch-RUNNING-COMMIT_PENDING-COMMIT-SUCCEED-KILL

cleanup[0]Mapcleanup:Launch-RUNNING-SUCCEED

Job: KILL

 

10.1.2.103执行了以下Task

maps[2]:Launch-RUNNING- SUCCEED

reduces[1]:Launch-RUNNING-COMMIT_PENDING-COMMIT-SUCCEED-KILL

Job: KILL

 

setup[1]Reducesetupcleanup[1]Reducecleanup)没有执行。