Skip to content

Commit 8d0bf30

Browse files
behradbehrad
committed
bump to 0.7.0
2 parents f131518 + f440b7b commit 8d0bf30

54 files changed

Lines changed: 4602 additions & 1622 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

100644100755
Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
1+
.idea
12
.DS_Store
23
node_modules
34
*.sock
4-
testing
55
lib/http/public/stylesheets/main.css
6+
*.rdb
7+
test/incomplete

‎.npmignore‎

100644100755
File mode changed.

‎.travis.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
language: node_js
2+
node_js:
3+
- "0.10"
4+
services:
5+
- redis-server

‎History.md‎

100644100755
Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,22 @@
11

2-
0.6.2 / 2013-04-03
2+
0.7.0 / 2014-01-24
3+
==================
4+
5+
* Suppress "undefined" messages on String errors. Closes #230
6+
* Fix cannot read property id of undefined errors. Closes #252
7+
* Parameterize limit of jobs checked in promotion cycles. Closes #244
8+
* Graceful shutdown
9+
* Worker pause/resume ability, Closes #163
10+
* Ensure event subscription before job save. Closes #179
11+
* Fix Queue singleton
12+
* Fix failed event being called in first attempt. Closes #142
13+
* Disable search (Search index memory leaks). See #58 & #218
14+
* Emit error events on both kue and job
15+
* JS/Coffeescript tests added (Mocha+Should)
16+
* Travis support added
17+
18+
19+
0.6.2 / 2013-04-03
320
==================
421

522
* Fix redirection to active for mounted apps

‎Makefile‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
REPORTER = spec
2+
3+
all: build
4+
5+
build:
6+
@./node_modules/coffee-script/bin/coffee \
7+
-c \
8+
-o lib src
9+
10+
test-tdd:
11+
@./node_modules/.bin/mocha \
12+
--reporter $(REPORTER) \
13+
--ui tdd \
14+
test/tdd/*.js
15+
16+
test-bdd:
17+
@./node_modules/.bin/mocha \
18+
--reporter $(REPORTER) \
19+
--require should \
20+
--ui bdd \
21+
test/*.js
22+
23+
test-bdd-coffee:
24+
@./node_modules/.bin/mocha \
25+
--compilers coffee:coffee-script \
26+
--reporter $(REPORTER) \
27+
--require should \
28+
--ui bdd \
29+
test/*.coffee
30+
31+
32+
test-all: test-bdd test-bdd-coffee
33+
34+
.PHONY: test-all

‎Readme.md‎

100644100755
Lines changed: 61 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,26 @@
1-
# Kue
1+
# Kue [![Build Status](https://travis-ci.org/behrad/kue.png)](https://travis-ci.org/behrad/kue)
22

33
Kue is a priority job queue backed by [redis](http://redis.io), built for [node.js](http://nodejs.org).
44

55
## Installation
66

77
$ npm install kue
88

9+
[![NPM](https://nodei.co/npm/kue.png?downloads=true&stars=true)](https://nodei.co/npm/kue/)
10+
911
## Features
1012

11-
- delayed jobs
12-
- job event and progress pubsub
13-
- rich integrated UI
14-
- infinite scrolling
13+
- Delayed jobs
14+
- Job event and progress pubsub
15+
- Rich integrated UI
16+
- Infinite scrolling
1517
- UI progress indication
16-
- job specific logging
17-
- powered by Redis
18-
- optional retries
19-
- full-text search capabilities
18+
- Job specific logging
19+
- Powered by Redis
20+
- Optional retries
21+
- Full-text search capabilities
2022
- RESTful JSON API
23+
- Graceful workers shutdown
2124

2225
## Overview
2326

@@ -32,6 +35,7 @@
3235
- [Processing Jobs](#processing-jobs)
3336
- [Processing Concurrency](#processing-concurrency)
3437
- [Updating Progress](#updating-progress)
38+
- [Graceful Shutdown](#graceful-shutdown)
3539
- [Redis Connection Settings](#redis-connection-settings)
3640
- [User-Interface](#user-interface)
3741
- [JSON API](#json-api)
@@ -117,7 +121,8 @@ job.progress(frames, totalFrames);
117121

118122
Job-specific events are fired on the `Job` instances via Redis pubsub. The following events are currently supported:
119123

120-
- `failed` the job has failed
124+
- `failed` the job has failed and has no remaining attempts
125+
- 'failed attempt' the job has failed, but has remaining attempts yet
121126
- `complete` the job has completed
122127
- `promotion` the job (when delayed) is now queued
123128
- `progress` the job's progress ranging from 0-100
@@ -167,12 +172,12 @@ var email = jobs.create('email', {
167172
title: 'Account renewal required'
168173
, to: 'tj@learnboost.com'
169174
, template: 'renewal-email'
170-
}).delay(minute)
175+
}).delay(milliseconds)
171176
.priority('high')
172177
.save();
173178
```
174179

175-
When using delayed jobs, we must also check the delayed jobs with a timer, promoting them if the scheduled delay has been exceeded. This `setInterval` is defined within `Queue#promote(ms)`, defaulting to a check every 5 seconds.
180+
When using delayed jobs, we must also check the delayed jobs with a timer, promoting them if the scheduled delay has been exceeded. This `setInterval` is defined within `Queue#promote(ms,limit)`, defaulting to a check of top 200 jobs every 5 seconds.
176181

177182
```js
178183
jobs.promote();
@@ -182,7 +187,8 @@ jobs.promote();
182187

183188
Processing jobs is simple with Kue. First create a `Queue` instance much like we do for creating jobs, providing us access to redis etc, then invoke `jobs.process()` with the associated type.
184189

185-
In the following example we pass the callback `done` to `email`, if this function responds with an error it will be displayed in the UI and the job will be marked as a failure.
190+
In the following example we pass the callback `done` to `email`, When an error occurs we invoke `done(err)` to tell Kue something happened, otherwise we invoke `done()` only when the job is complete.
191+
if this function responds with an error it will be displayed in the UI and the job will be marked as a failure.
186192

187193
```js
188194
var kue = require('kue')
@@ -214,7 +220,7 @@ jobs.create('slideshow pdf', {
214220
});
215221
```
216222

217-
We can access this same arbitrary data within a separate process while processing, via the `job.data` property. In the example we render each slide one-by-one, updating the job's log and process. When an error occurs we invoke `done(err)` to tell Kue something happened, otherwise we invoke `done()` only when the job is complete.
223+
We can access this same arbitrary data within a separate process while processing, via the `job.data` property. In the example we render each slide one-by-one, updating the job's log and process.
218224

219225
```js
220226
jobs.process('slideshow pdf', 5, function(job, done){
@@ -236,9 +242,40 @@ jobs.process('slideshow pdf', 5, function(job, done){
236242
});
237243
```
238244

245+
### Graceful Shutdown
246+
247+
As of Kue 0.7.0, a `Queue#shutdown(fn, timeout)` is added which signals all workers to stop processing after
248+
their current active job is done. Workers will wait `timeout` milliseconds for their active job's done to be called
249+
or mark the active job `failed` with shutdown error reason. When all workers tell Kue they are stopped `fn` is called.
250+
251+
```javascript
252+
process.once( 'SIGTERM', function ( sig ) {
253+
queue.shutdown(function(err) {
254+
console.log( 'Kue is shut down.', err||'' );
255+
process.exit( 0 );
256+
}, 5000 );
257+
});
258+
```
259+
239260
## Redis Connection Settings
240261

241-
By default, Kue will connect to Redis using the client default settings (port defaults to `6379`, host defaults to `127.0.0.1`). Redis client connection settings can be set by overriding the `kue.redis.createClient` function.
262+
By default, Kue will connect to Redis using the client default settings (port defaults to `6379`, host defaults to `127.0.0.1`).
263+
`Queue#createQueue(options)` accepts redis connection options in `options.redis` key.
264+
265+
```javascript
266+
var kue = require('kue');
267+
q = kue.createQueue({
268+
redis: {
269+
port: 1234,
270+
host: '10.0.50.20'
271+
options: {
272+
// look for more redis options in [node_redis](https://github.com/mranney/node_redis)
273+
}
274+
}
275+
});
276+
```
277+
278+
For backward compatibility to `Kue < 0.7.0`, monkey-patch-styled Redis client connection settings can be set by overriding the `kue.redis.createClient` function.
242279

243280
For example, to create a Redis client that connects to `192.168.1.2` on port `1234` that requires authentication, use the following:
244281

@@ -280,6 +317,15 @@ kue.app.set('title', 'My Application');
280317

281318
```js
282319
["5", "7", "10"]
320+
```
321+
You may disable search indexes in `Kue >= 0.7.0` for memory optimization or to avoid probable
322+
memory leak reported in issue list:
323+
324+
```javascript
325+
var kue = require('kue');
326+
q = kue.createQueue({
327+
disableSearch: true
328+
});
283329
```
284330

285331
### GET /stats

‎examples/delayed.js‎

100644100755
Lines changed: 14 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
21
var kue = require('../');
32

43
// create our job queue
@@ -10,36 +9,32 @@ var jobs = kue.createQueue();
109
var minute = 60000;
1110

1211
var email = jobs.create('email', {
13-
title: 'Account renewal required'
14-
, to: 'tj@learnboost.com'
15-
, template: 'renewal-email'
12+
title: 'Account renewal required', to: 'tj@learnboost.com', template: 'renewal-email'
1613
}).delay(minute)
17-
.priority('high')
18-
.save();
14+
.priority('high')
15+
.save();
1916

2017

21-
email.on('promotion', function(){
22-
console.log('renewal job promoted');
18+
email.on('promotion', function () {
19+
console.log('renewal job promoted');
2320
});
2421

25-
email.on('complete', function(){
26-
console.log('renewal job completed');
22+
email.on('complete', function () {
23+
console.log('renewal job completed');
2724
});
2825

2926
jobs.create('email', {
30-
title: 'Account expired'
31-
, to: 'tj@learnboost.com'
32-
, template: 'expired-email'
27+
title: 'Account expired', to: 'tj@learnboost.com', template: 'expired-email'
3328
}).delay(minute * 10)
34-
.priority('high')
35-
.save();
29+
.priority('high')
30+
.save();
3631

3732
jobs.promote();
3833

39-
jobs.process('email', 10, function(job, done){
40-
setTimeout(function(){
41-
done();
42-
}, Math.random() * 5000);
34+
jobs.process('email', 10, function (job, done) {
35+
setTimeout(function () {
36+
done();
37+
}, Math.random() * 5000);
4338
});
4439

4540
// start the UI

‎examples/events.js‎

100644100755
Lines changed: 32 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
21
var kue = require('../');
32

43
// create our job queue
@@ -13,49 +12,47 @@ var jobs = kue.createQueue();
1312
// user input etc.
1413

1514
function create() {
16-
var name = ['tobi', 'loki', 'jane', 'manny'][Math.random() * 4 | 0];
17-
var job = jobs.create('video conversion', {
18-
title: 'converting ' + name + '\'s to avi'
19-
, user: 1
20-
, frames: 200
21-
});
22-
23-
job.on('complete', function(){
24-
console.log(" Job complete");
25-
}).on('failed', function(){
26-
console.log(" Job failed");
27-
}).on('progress', function(progress){
28-
process.stdout.write('\r job #' + job.id + ' ' + progress + '% complete');
29-
});
30-
31-
job.save();
32-
33-
setTimeout(create, Math.random() * 2000 | 0);
15+
var name = ['tobi', 'loki', 'jane', 'manny'][Math.random() * 4 | 0];
16+
var job = jobs.create('video conversion', {
17+
title: 'converting ' + name + '\'s to avi', user: 1, frames: 200
18+
});
19+
20+
job.on('complete',function () {
21+
console.log(" Job complete");
22+
}).on('failed',function () {
23+
console.log(" Job failed");
24+
}).on('progress', function (progress) {
25+
process.stdout.write('\r job #' + job.id + ' ' + progress + '% complete');
26+
});
27+
28+
job.save();
29+
30+
setTimeout(create, Math.random() * 2000 | 0);
3431
}
3532

3633
create();
3734

3835
// process video conversion jobs, 1 at a time.
3936

40-
jobs.process('video conversion', 1, function(job, done){
41-
var frames = job.data.frames;
42-
43-
function next(i) {
44-
// pretend we are doing some work
45-
convertFrame(i, function(err){
46-
if (err) return done(err);
47-
// report progress, i/frames complete
48-
job.progress(i, frames);
49-
if (i >= frames) done()
50-
else next(i + Math.random() * 10);
51-
});
52-
}
53-
54-
next(0);
37+
jobs.process('video conversion', 1, function (job, done) {
38+
var frames = job.data.frames;
39+
40+
function next(i) {
41+
// pretend we are doing some work
42+
convertFrame(i, function (err) {
43+
if (err) return done(err);
44+
// report progress, i/frames complete
45+
job.progress(i, frames);
46+
if (i >= frames) done()
47+
else next(i + Math.random() * 10);
48+
});
49+
}
50+
51+
next(0);
5552
});
5653

5754
function convertFrame(i, fn) {
58-
setTimeout(fn, Math.random() * 50);
55+
setTimeout(fn, Math.random() * 50);
5956
}
6057

6158
// start the UI

0 commit comments

Comments
 (0)