Skip to content

making a through stream and a writable stream appear as a through stream, and append a chunk to the end #81

Description

@m59peacemaker

I have a transform stream and a writable stream that both need to read from stdin. The transform stream transforms chunks and emits them (as usual). The writable stream collects some information from the stream and has a getInfo() function that can be called to retrieve that information. This is what my current code looks like, with mergeStreams being a hacky mess to make it work:

const myStream = () => {
  const transformStream = foo()
  const writableStream = bar()
  const stream = mergeStreams([transformStream, writableStream], (push, cb) => { // flush function
    // add some output to the end of the stream
    push('\n' + writableStream.getInfo() + '\n')
    cb() // stream is done
  })
  stream.push('\n') // start with a blank line
  return stream
}

process.stdin
  .pipe(myStream())
  .pipe(process.stdout)

// sample output:
/*                // blank line
 * output from transformStream
 * output from transformStream
 * and so on from transformStream
 * and finally the result from writableStream.getInfo()
 */

and here's a rundown of the stream logic

/* stream is writable/readable
  on write, chunks go to transformStream and writableStream
  transformStream receives, transforms, and emits values
  when the writer is finished writing to the stream
    and transformStream is done reading/writing
    and writableStream is done reading,
    emit result of writableStream.getInfo()
*/

I made this work with node streams by proxying the chunks to the other streams and using the flush function and events. It's complex and awkward behind the scenes. I hope that with pull-stream, I can get rid of that hacky mergeStreams function and do this with some pull-stream modules. I can't find a way to return a single stream that writes to both transformStream and writableStream and to inject the results of getInfo() at the end.

Thanks for any advice on this!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions