我们如何在Memcached读取中阻止Ruby on Rails,以免占用CPU?

问题描述:

我们有一个守护进程,该守护进程扫描表中的脏位,然后将脏行分批调度到delay_job中.为了避免常量select from data where dirty = 1,我们设置了一个memcached屏障,该屏障包装了表格扫描,例如

We have a daemon which scans a table for dirty bits and then schedules dirty rows to a delayed_job in batches. In order to avoid a constant select from data where dirty = 1, we set up a memcached barrier, which wraps the table scan, like

   loop do # daemon
     until Rails.cache.fetch("have_dirty_rows") do end
     page = 1
     loop do # paginate dirty rows
       dirty_batch = paginate(#:select     => "*",
                           :order      => "id",
                           :per_page   => DIRTY_GET_BATCH_SIZE,
                           :conditions => {:dirty => 1},
                           :page       => page)
       if dirty_batch.empty?
         Rails.cache.write("have_dirty_rows",false)
         break
       end
       ...
       page = page.next
     end
   end

除非我添加一些sleep 0.0001或类似的内容,否则循环仍然会消耗100%的CPU. Ruby/Rails中是否存在一种有效的机制,该机制会阻止诸如memcached值之类的东西,或者我们可以从memcached值中获取数据,因此它不会一直轮询?

Unless I add some sleep 0.0001 or such, the loop eats 100% CPU still. Is there an efficient mechanism in Ruby/Rails which will block on something like the memcached value, or which we can feed from a memcached value, so it's not polling all the time?

主动轮询很糟糕! 脏点从哪里来? 如果此进程使用消息队列机制(例如RabbitMQ)来通知其他进程会更好.数据库中发生了某些变化.

Active polling is BAD ! Where are the dirty bits comin from? It woul be better if this process uses a message queue mechanism (eg RabbitMQ ) to notify other processes. That something has changed in the database.