IT박스

고착 된 / 사실적인 Resque 직원을 어떻게 정리합니까?

itboxs 2020. 6. 29. 08:15
반응형

고착 된 / 사실적인 Resque 직원을 어떻게 정리합니까?


첨부 된 이미지에서 볼 수 있듯이 갇힌 것처럼 보이는 두 명의 작업자가 있습니다. 이러한 프로세스는 몇 초 이상 걸리지 않아야합니다.

여기에 이미지 설명을 입력하십시오

왜 그들이 명확하지 않은지 또는 수동으로 제거하는 방법을 모르겠습니다.

Redis-to-Go 및 HireFire와 함께 Resque를 사용하여 자동으로 작업자를 조정하는 Heroku를 사용하고 있습니다.


이 솔루션 중 어느 것도 나를 위해 일한 적이 없으며 redis-web에서 여전히 볼 수 있습니다.

0 out of 10 Workers Working

마지막으로 모든 작업자를 지우는 데 도움이되었습니다.

Resque.workers.each {|w| w.unregister_worker}

콘솔에서 :

queue_name = "process_numbers"
Resque.redis.del "queue:#{queue_name}"

그렇지 않으면 다음과 같이 제거하기 위해 가짜로 시도 할 수 있습니다.

Resque::Worker.working.each {|w| w.done_working}

편집하다

많은 사람들 이이 답변을 찬성했으며 사람들이 대기열에서 작업자를 등록 취소하는 hagope 솔루션을 시도하는 것이 중요하다고 생각하지만 위의 코드는 대기열을 삭제합니다. 당신이 그들을 가짜로 기뻐한다면, 시원하십시오.


resque gem이 설치되어 있으므로 콘솔을 열고 현재 작업자를 얻을 수 있습니다.

Resque.workers

작업자 목록을 반환합니다.

#=> [#<Worker infusion.local:40194-0:JAVA_DYNAMIC_QUEUES,index_migrator,converter,extractor>]

작업자를 선택하고 prune_dead_workers예를 들어 첫 번째

Resque.workers.first.prune_dead_workers

hagope의 답변에 덧붙여서, 나는 일정 시간 동안 달리고 있던 근로자 만 등록을 취소하고 싶었습니다. 아래 코드는 300 초 (5 분) 이상 근무하는 근로자 만 등록을 취소합니다.

Resque.workers.each {|w| w.unregister_worker if w.processing['run_at'] && Time.now - w.processing['run_at'].to_time > 300}

https://gist.github.com/ewherrmann/8809350에 추가 한 Resque 관련 Rake 작업 컬렉션이 진행 중입니다.


서버를 시작하기 위해 명령을 실행할 때마다이 명령을 실행하십시오.

$ ps -e -o pid,command | grep [r]esque

다음과 같이 보일 것입니다 :

92102 resque: Processing ProcessNumbers since 1253142769

내 예제에서 PID (프로세스 ID)를 기록하십시오.

그런 다음 두 가지 방법 중 하나의 프로세스를 종료 할 수 있습니다.

  • 정상적으로 사용 QUIT 92102

  • 강제 사용 TERM 92102

* I'm not sure of the syntax it's either QUIT 92102 or QUIT -92102

Let me know if you have any trouble.


I just did:

% rails c production
irb(main):001:0>Resque.workers

Got the list of workers.

irb(main):002:0>Resque.remove_worker(Resque.workers[n].id)

... where n is the zero based index of the unwanted worker.


I had a similar problem that Redis saved the DB to disk that included invalid (non running) workers. Each time Redis/resque was started they appeared.

Fix this using:

Resque::Worker.working.each {|w| w.done_working}
Resque.redis.save # Save the DB to disk without ANY workers

Make sure you restart Redis and your Resque workers.


Here's how you can purge them from Redis by hostname. This happens to me when I decommission a server and workers do not exit gracefully.

Resque.workers.each { |w| w.unregister_worker if w.id.start_with?(hostname) }

I ran into this issue and started down the path of implementing a lot of the suggestions here. However, I discovered the root cause that was creating this issue was that I was using the gem redis-rb 3.3.0. Downgrading to redis-rb 3.2.2 prevented these workers from getting stuck in the first place.


Started working on https://github.com/shaiguitar/resque_stuck_queue/ recently. It's not a solution to how to fix stuck workers but it addresses the issue of resque hanging/being stuck, so I figured it could be helpful for people on this thread. From README:

"If resque doesn't run jobs within a certain timeframe, it will trigger a pre-defined handler of your choice. You can use this to send an email, pager duty, add more resque workers, restart resque, send you a txt...whatever suits you."

Been used in production and works pretty well for me thus far.


I had stuck/stale resque workers here too, or should I say 'jobs', because the worker is actually still there and running fine, it's the forked process that is stuck.

I chose the brutal solution of killing the forked process "Processing" since more than 5min, via a bash script, then the worker just spawn the next in queue, and everything keeps on going

have a look at my script here: https://gist.github.com/jobwat/5712437


I've cleared them out from redis-cli directly. Luckily redistogo.com allows access from environments outside heroku. Get dead worker ID from the list. Mine was

55ba6f3b-9287-4f81-987a-4e8ae7f51210:2

Run this command in redis directly.

del "resque:worker:55ba6f3b-9287-4f81-987a-4e8ae7f51210:2:*"

You can monitor redis db to see what it's doing behind the scenes.

redis xxx.redistogo.com> MONITOR
OK
1380274567.540613 "MONITOR"
1380274568.345198 "incrby" "resque:stat:processed" "1"
1380274568.346898 "incrby" "resque:stat:processed:c65c8e2b-555a-4a57-aaa6-477b27d6452d:2:*" "1"
1380274568.346920 "del" "resque:worker:c65c8e2b-555a-4a57-aaa6-477b27d6452d:2:*"
1380274568.348803 "smembers" "resque:queues"

Second last line deletes the worker.


If you are using newer versions of Resque, you'll need to use the following command as the internal APIs have changed...

Resque::WorkerRegistry.working.each {|work| Resque::WorkerRegistry.remove(work.id)}

This avoids the problem as long as you have a resque version newer than 1.26.0:

resque: env QUEUE=foo TERM_CHILD=1 bundle exec rake resque:work

Keep in mind that it does not let the currently running job finish.


you can also use below command to stop all rescue worker

sudo kill -9  `ps aux | grep resque | grep -v grep | cut -c 10-16`

reference from this link

참고URL : https://stackoverflow.com/questions/7416318/how-do-i-clear-stuck-stale-resque-workers

반응형