Skip to content

Queue tutorials: should we use channel#get rather than channel#consume? #38

@SimonWoolf

Description

@SimonWoolf

Inspired by intercom conversation just now.

Given we're putting emphasis on the queues as a way for people to use multiple independent workers to each pop messages off to process, rather than for streaming, should we consider having the tutorials use channel#get ('pop one message off') rather than channel#consume ('give me messages as fast as they come')?

(I know my queue-demo used consume(), but that was so we could show off how fast messages got into the queue (and 'processing time' was ~0 as it was just printing them to the console), so the streaming method was fine).

We could even showcase some basic limited concurrency, e.g. something like:

function processItem(item, cb) {
  console.log("Processing item: ", item && String(item.content))
  setTimeout(cb, 5000);
}

function getMessage(channel) {
  channel.get(QUEUE, {}, (err, item) => {
    if(!item) {
      setTimeout(() => getMessage(channel), 1000);
      return;
    }
    processItem(item, (err) => {
      if(err) {
        channel.nack(item);
      } else {
        channel.ack(item);
      }
      // Processing finished, so get another message
      getMessage(channel);
    });
  });
}

const MAX_CONCURRENCY = 5;

amqp.connect(URL, (err, conn) => {
  if (err) bail(err);
  console.log("Connected");

  conn.createChannel((err, ch) => {
    for(let i=0; i<MAX_CONCURRENCY; i++) {
      getMessage(ch);
    }
  });
});

WDYT?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions