单线程池与每个任务一个线程池
我想使用Java中的并发性向在线API发出请求,下载并解析响应文档,然后将结果数据加载到数据库中.
I want to use concurrency in Java to make requests to an online API, download and parse the response documents, and load the resulting data into a database.
在每个线程中请求,解析和加载一个线程池是否是标准做法?换句话说,只有一个类实现Runnable
.或者说,拥有三个不同的线程池,效率更高,第一个线程池发出请求并将其推入队列,第二个线程池从第一个队列中轮询,解析并推入解析的数据到第二个队列,最后是第三个池,轮询第二个队列中的数据并加载到数据库中?在这种情况下,我将编写三个实现Runnable
的不同类.
Is it standard to have one pool of threads in which each thread requests, parses, and loads? In other words, only one class implements Runnable
. Or is it more efficient to have, say, three different pools of threads, with the first pool of threads making the requests and pushing them to a queue, the second pool of threads polling from the first queue, parsing, and pushing the parsed data to a second queue, and finally the third pool polling the data from the second queue and loading into the database? In this case, I'd write three different classes that implement Runnable
.
您必须考虑处理的哪些部分将从并行性中受益.在线API通信很可能是候选方法,因为将涉及套接字和网络等待.与DB交互也是如此.如果有多个 可用 CPU内核,多线程分析可能只会提高性能.
You have to consider which parts of the processing will benefit from parallelism. The online API communication will most likely be a candidate, since there will be sockets and network waits involved. Likewise with the DB interaction. Multithreaded parsing will probably only improve performance if there are multiple available CPU cores.
将整个过程分为3个单独的类肯定会增加凝聚力,这意味着每个类的职责都会减少,这是一件好事.另一方面,将每个这些类设置为Runnable
并具有多个队列将增加应用程序的复杂性(可能不必要).
Splitting the entire process into 3 separate classes will definitely increase the cohesion, meaning each class will have less responsibilities, which is a good thing. On the other hand, making each of these classes a Runnable
and having several queues will increase the complexity (possibly unecessarily) of the application.
我建议制作3个单独的类,但不要将它们设为Runnable
.然后将contains
和orchestrates
划分为3个类的Runnable
,即一个单线程池.如果您发现这样做似乎不够快(并且经过了一些性能分析),请尝试将可运行对象分成2个线程池:下载和解析以及db访问.
I would suggest making 3 separate classes, but dont make them Runnable
. Then make a Runnable
that contains
and orchestrates
the 3 classes, that is one single thread pool. If you see that this doesnt seem to be fast enough (and after some profiling), try splitting the runnable into 2 thread pools: a download and parse, and a db access.
重点是,从简单开始,并根据需要增加复杂性.
The point being, start simple and add complexity as needed.