我们如何在Memcached读取中阻止Ruby on Rails,以免占用CPU?
我们有一个守护进程,该守护进程扫描表中的脏位,然后将脏行分批调度到delay_job中.为了避免常量select from data where dirty = 1
,我们设置了一个memcached屏障,该屏障包装了表格扫描,例如
We have a daemon which scans a table for dirty bits and then schedules dirty rows to a delayed_job in batches. In order to avoid a constant select from data where dirty = 1
, we set up a memcached barrier, which wraps the table scan, like
loop do # daemon
until Rails.cache.fetch("have_dirty_rows") do end
page = 1
loop do # paginate dirty rows
dirty_batch = paginate(#:select => "*",
:order => "id",
:per_page => DIRTY_GET_BATCH_SIZE,
:conditions => {:dirty => 1},
:page => page)
if dirty_batch.empty?
Rails.cache.write("have_dirty_rows",false)
break
end
...
page = page.next
end
end
除非我添加一些sleep 0.0001
或类似的内容,否则循环仍然会消耗100%的CPU. Ruby/Rails中是否存在一种有效的机制,该机制会阻止诸如memcached值之类的东西,或者我们可以从memcached值中获取数据,因此它不会一直轮询?
Unless I add some sleep 0.0001
or such, the loop eats 100% CPU still. Is there an efficient mechanism in Ruby/Rails which will block on something like the memcached value, or which we can feed from a memcached value, so it's not polling all the time?
主动轮询很糟糕! 脏点从哪里来? 如果此进程使用消息队列机制(例如RabbitMQ)来通知其他进程会更好.数据库中发生了某些变化.
Active polling is BAD ! Where are the dirty bits comin from? It woul be better if this process uses a message queue mechanism (eg RabbitMQ ) to notify other processes. That something has changed in the database.