conduit-1.3.5: Streaming data processing library.

Safe Haskell | Safe-Inferred
Language | Haskell2010

Data.Conduit.Combinators

Contents

- Producers
  - Pure
  - Monadic
  - I/O
  - Filesystem
- Consumers
  - Pure
  - Monadic
  - I/O
- Transformers
  - Pure
  - Monadic
  - Textual
  - Builders
- Special

Description

This module is meant as a replacement for Data.Conduit.List. That module follows a naming scheme which was originally inspired by its enumerator roots. This module is meant to introduce a naming scheme which encourages conduit best practices.

There are two versions of functions in this module. Those with a trailing E work in the individual elements of a chunk of data, e.g., the bytes of a ByteString, the Chars of a Text, or the Ints of a Vector Int. Those without a trailing E work on unchunked streams.

FIXME: discuss overall naming, usage of mono-traversable, etc

Mention take (Conduit) vs drop (Consumer)

Synopsis

- yieldMany :: (Monad m, MonoFoldable mono) => mono -> ConduitT i (Element mono) m ()
- unfold :: Monad m => (b -> Maybe (a, b)) -> b -> ConduitT i a m ()
- enumFromTo :: (Monad m, Enum a, Ord a) => a -> a -> ConduitT i a m ()
- iterate :: Monad m => (a -> a) -> a -> ConduitT i a m ()
- repeat :: Monad m => a -> ConduitT i a m ()
- replicate :: Monad m => Int -> a -> ConduitT i a m ()
- sourceLazy :: (Monad m, LazySequence lazy strict) => lazy -> ConduitT i strict m ()
- repeatM :: Monad m => m a -> ConduitT i a m ()
- repeatWhileM :: Monad m => m a -> (a -> Bool) -> ConduitT i a m ()
- replicateM :: Monad m => Int -> m a -> ConduitT i a m ()
- sourceFile :: MonadResource m => FilePath -> ConduitT i ByteString m ()
- sourceFileBS :: MonadResource m => FilePath -> ConduitT i ByteString m ()
- sourceHandle :: MonadIO m => Handle -> ConduitT i ByteString m ()
- sourceHandleUnsafe :: MonadIO m => Handle -> ConduitT i ByteString m ()
- sourceIOHandle :: MonadResource m => IO Handle -> ConduitT i ByteString m ()
- stdin :: MonadIO m => ConduitT i ByteString m ()
- withSourceFile :: (MonadUnliftIO m, MonadIO n) => FilePath -> (ConduitM i ByteString n () -> m a) -> m a
- sourceDirectory :: MonadResource m => FilePath -> ConduitT i FilePath m ()
- sourceDirectoryDeep :: MonadResource m => Bool -> FilePath -> ConduitT i FilePath m ()
- drop :: Monad m => Int -> ConduitT a o m ()
- dropE :: (Monad m, IsSequence seq) => Index seq -> ConduitT seq o m ()
- dropWhile :: Monad m => (a -> Bool) -> ConduitT a o m ()
- dropWhileE :: (Monad m, IsSequence seq) => (Element seq -> Bool) -> ConduitT seq o m ()
- fold :: (Monad m, Monoid a) => ConduitT a o m a
- foldE :: (Monad m, MonoFoldable mono, Monoid (Element mono)) => ConduitT mono o m (Element mono)
- foldl :: Monad m => (a -> b -> a) -> a -> ConduitT b o m a
- foldl1 :: Monad m => (a -> a -> a) -> ConduitT a o m (Maybe a)
- foldlE :: (Monad m, MonoFoldable mono) => (a -> Element mono -> a) -> a -> ConduitT mono o m a
- foldMap :: (Monad m, Monoid b) => (a -> b) -> ConduitT a o m b
- foldMapE :: (Monad m, MonoFoldable mono, Monoid w) => (Element mono -> w) -> ConduitT mono o m w
- foldWhile :: Monad m => (a -> s -> Either e s) -> s -> ConduitT a o m (Either e s)
- all :: Monad m => (a -> Bool) -> ConduitT a o m Bool
- allE :: (Monad m, MonoFoldable mono) => (Element mono -> Bool) -> ConduitT mono o m Bool
- any :: Monad m => (a -> Bool) -> ConduitT a o m Bool
- anyE :: (Monad m, MonoFoldable mono) => (Element mono -> Bool) -> ConduitT mono o m Bool
- and :: Monad m => ConduitT Bool o m Bool
- andE :: (Monad m, MonoFoldable mono, Element mono ~ Bool) => ConduitT mono o m Bool
- or :: Monad m => ConduitT Bool o m Bool
- orE :: (Monad m, MonoFoldable mono, Element mono ~ Bool) => ConduitT mono o m Bool
- asum :: (Monad m, Alternative f) => ConduitT (f a) o m (f a)
- elem :: (Monad m, Eq a) => a -> ConduitT a o m Bool
- elemE :: (Monad m, IsSequence seq, Eq (Element seq)) => Element seq -> ConduitT seq o m Bool
- notElem :: (Monad m, Eq a) => a -> ConduitT a o m Bool
- notElemE :: (Monad m, IsSequence seq, Eq (Element seq)) => Element seq -> ConduitT seq o m Bool
- sinkLazy :: (Monad m, LazySequence lazy strict) => ConduitT strict o m lazy
- sinkList :: Monad m => ConduitT a o m [a]
- sinkVector :: (Vector v a, PrimMonad m) => ConduitT a o m (v a)
- sinkVectorN :: (Vector v a, PrimMonad m) => Int -> ConduitT a o m (v a)
- sinkLazyBuilder :: Monad m => ConduitT Builder o m ByteString
- sinkNull :: Monad m => ConduitT a o m ()
- awaitNonNull :: (Monad m, MonoFoldable a) => ConduitT a o m (Maybe (NonNull a))
- head :: Monad m => ConduitT a o m (Maybe a)
- headDef :: Monad m => a -> ConduitT a o m a
- headE :: (Monad m, IsSequence seq) => ConduitT seq o m (Maybe (Element seq))
- peek :: Monad m => ConduitT a o m (Maybe a)
- peekE :: (Monad m, MonoFoldable mono) => ConduitT mono o m (Maybe (Element mono))
- last :: Monad m => ConduitT a o m (Maybe a)
- lastDef :: Monad m => a -> ConduitT a o m a
- lastE :: (Monad m, IsSequence seq) => ConduitT seq o m (Maybe (Element seq))
- length :: (Monad m, Num len) => ConduitT a o m len
- lengthE :: (Monad m, Num len, MonoFoldable mono) => ConduitT mono o m len
- lengthIf :: (Monad m, Num len) => (a -> Bool) -> ConduitT a o m len
- lengthIfE :: (Monad m, Num len, MonoFoldable mono) => (Element mono -> Bool) -> ConduitT mono o m len
- maximum :: (Monad m, Ord a) => ConduitT a o m (Maybe a)
- maximumE :: (Monad m, IsSequence seq, Ord (Element seq)) => ConduitT seq o m (Maybe (Element seq))
- minimum :: (Monad m, Ord a) => ConduitT a o m (Maybe a)
- minimumE :: (Monad m, IsSequence seq, Ord (Element seq)) => ConduitT seq o m (Maybe (Element seq))
- null :: Monad m => ConduitT a o m Bool
- nullE :: (Monad m, MonoFoldable mono) => ConduitT mono o m Bool
- sum :: (Monad m, Num a) => ConduitT a o m a
- sumE :: (Monad m, MonoFoldable mono, Num (Element mono)) => ConduitT mono o m (Element mono)
- product :: (Monad m, Num a) => ConduitT a o m a
- productE :: (Monad m, MonoFoldable mono, Num (Element mono)) => ConduitT mono o m (Element mono)
- find :: Monad m => (a -> Bool) -> ConduitT a o m (Maybe a)
- mapM_ :: Monad m => (a -> m ()) -> ConduitT a o m ()
- mapM_E :: (Monad m, MonoFoldable mono) => (Element mono -> m ()) -> ConduitT mono o m ()
- foldM :: Monad m => (a -> b -> m a) -> a -> ConduitT b o m a
- foldME :: (Monad m, MonoFoldable mono) => (a -> Element mono -> m a) -> a -> ConduitT mono o m a
- foldMapM :: (Monad m, Monoid w) => (a -> m w) -> ConduitT a o m w
- foldMapME :: (Monad m, MonoFoldable mono, Monoid w) => (Element mono -> m w) -> ConduitT mono o m w
- sinkFile :: MonadResource m => FilePath -> ConduitT ByteString o m ()
- sinkFileCautious :: MonadResource m => FilePath -> ConduitM ByteString o m ()
- sinkTempFile :: MonadResource m => FilePath -> String -> ConduitM ByteString o m FilePath
- sinkSystemTempFile :: MonadResource m => String -> ConduitM ByteString o m FilePath
- sinkFileBS :: MonadResource m => FilePath -> ConduitT ByteString o m ()
- sinkHandle :: MonadIO m => Handle -> ConduitT ByteString o m ()
- sinkIOHandle :: MonadResource m => IO Handle -> ConduitT ByteString o m ()
- print :: (Show a, MonadIO m) => ConduitT a o m ()
- stdout :: MonadIO m => ConduitT ByteString o m ()
- stderr :: MonadIO m => ConduitT ByteString o m ()
- withSinkFile :: (MonadUnliftIO m, MonadIO n) => FilePath -> (ConduitM ByteString o n () -> m a) -> m a
- withSinkFileBuilder :: (MonadUnliftIO m, MonadIO n) => FilePath -> (ConduitM Builder o n () -> m a) -> m a
- withSinkFileCautious :: (MonadUnliftIO m, MonadIO n) => FilePath -> (ConduitM ByteString o n () -> m a) -> m a
- sinkHandleBuilder :: MonadIO m => Handle -> ConduitM Builder o m ()
- sinkHandleFlush :: MonadIO m => Handle -> ConduitM (Flush ByteString) o m ()
- map :: Monad m => (a -> b) -> ConduitT a b m ()
- mapE :: (Monad m, Functor f) => (a -> b) -> ConduitT (f a) (f b) m ()
- omapE :: (Monad m, MonoFunctor mono) => (Element mono -> Element mono) -> ConduitT mono mono m ()
- concatMap :: (Monad m, MonoFoldable mono) => (a -> mono) -> ConduitT a (Element mono) m ()
- concatMapE :: (Monad m, MonoFoldable mono, Monoid w) => (Element mono -> w) -> ConduitT mono w m ()
- take :: Monad m => Int -> ConduitT a a m ()
- takeE :: (Monad m, IsSequence seq) => Index seq -> ConduitT seq seq m ()
- takeWhile :: Monad m => (a -> Bool) -> ConduitT a a m ()
- takeWhileE :: (Monad m, IsSequence seq) => (Element seq -> Bool) -> ConduitT seq seq m ()
- takeExactly :: Monad m => Int -> ConduitT a b m r -> ConduitT a b m r
- takeExactlyE :: (Monad m, IsSequence a) => Index a -> ConduitT a b m r -> ConduitT a b m r
- concat :: (Monad m, MonoFoldable mono) => ConduitT mono (Element mono) m ()
- filter :: Monad m => (a -> Bool) -> ConduitT a a m ()
- filterE :: (IsSequence seq, Monad m) => (Element seq -> Bool) -> ConduitT seq seq m ()
- mapWhile :: Monad m => (a -> Maybe b) -> ConduitT a b m ()
- conduitVector :: (Vector v a, PrimMonad m) => Int -> ConduitT a (v a) m ()
- scanl :: Monad m => (a -> b -> a) -> a -> ConduitT b a m ()
- mapAccumWhile :: Monad m => (a -> s -> Either s (s, b)) -> s -> ConduitT a b m s
- concatMapAccum :: Monad m => (a -> accum -> (accum, [b])) -> accum -> ConduitT a b m ()
- intersperse :: Monad m => a -> ConduitT a a m ()
- slidingWindow :: (Monad m, IsSequence seq, Element seq ~ a) => Int -> ConduitT a seq m ()
- chunksOfE :: (Monad m, IsSequence seq) => Index seq -> ConduitT seq seq m ()
- chunksOfExactlyE :: (Monad m, IsSequence seq) => Index seq -> ConduitT seq seq m ()
- mapM :: Monad m => (a -> m b) -> ConduitT a b m ()
- mapME :: (Monad m, Traversable f) => (a -> m b) -> ConduitT (f a) (f b) m ()
- omapME :: (Monad m, MonoTraversable mono) => (Element mono -> m (Element mono)) -> ConduitT mono mono m ()
- concatMapM :: (Monad m, MonoFoldable mono) => (a -> m mono) -> ConduitT a (Element mono) m ()
- filterM :: Monad m => (a -> m Bool) -> ConduitT a a m ()
- filterME :: (Monad m, IsSequence seq) => (Element seq -> m Bool) -> ConduitT seq seq m ()
- iterM :: Monad m => (a -> m ()) -> ConduitT a a m ()
- scanlM :: Monad m => (a -> b -> m a) -> a -> ConduitT b a m ()
- mapAccumWhileM :: Monad m => (a -> s -> m (Either s (s, b))) -> s -> ConduitT a b m s
- concatMapAccumM :: Monad m => (a -> accum -> m (accum, [b])) -> accum -> ConduitT a b m ()
- encodeUtf8 :: (Monad m, Utf8 text binary) => ConduitT text binary m ()
- decodeUtf8 :: MonadThrow m => ConduitT ByteString Text m ()
- decodeUtf8Lenient :: Monad m => ConduitT ByteString Text m ()
- line :: (Monad m, IsSequence seq, Element seq ~ Char) => ConduitT seq o m r -> ConduitT seq o m r
- lineAscii :: (Monad m, IsSequence seq, Element seq ~ Word8) => ConduitT seq o m r -> ConduitT seq o m r
- unlines :: (Monad m, IsSequence seq, Element seq ~ Char) => ConduitT seq seq m ()
- unlinesAscii :: (Monad m, IsSequence seq, Element seq ~ Word8) => ConduitT seq seq m ()
- takeExactlyUntilE :: (Monad m, IsSequence seq) => (Element seq -> Bool) -> ConduitT seq o m r -> ConduitT seq o m r
- linesUnbounded :: (Monad m, IsSequence seq, Element seq ~ Char) => ConduitT seq seq m ()
- linesUnboundedAscii :: (Monad m, IsSequence seq, Element seq ~ Word8) => ConduitT seq seq m ()
- splitOnUnboundedE :: (Monad m, IsSequence seq) => (Element seq -> Bool) -> ConduitT seq seq m ()
- builderToByteString :: PrimMonad m => ConduitT Builder ByteString m ()
- unsafeBuilderToByteString :: PrimMonad m => ConduitT Builder ByteString m ()
- builderToByteStringWith :: PrimMonad m => BufferAllocStrategy -> ConduitT Builder ByteString m ()
- builderToByteStringFlush :: PrimMonad m => ConduitT (Flush Builder) (Flush ByteString) m ()
- builderToByteStringWithFlush :: PrimMonad m => BufferAllocStrategy -> ConduitT (Flush Builder) (Flush ByteString) m ()
- type BufferAllocStrategy = (IO Buffer, Int -> Buffer -> IO (IO Buffer))
- allNewBuffersStrategy :: Int -> BufferAllocStrategy
- reuseBufferStrategy :: IO Buffer -> BufferAllocStrategy
- vectorBuilder :: (PrimMonad m, PrimMonad n, Vector v e, PrimState m ~ PrimState n) => Int -> ((e -> n ()) -> ConduitT i Void m r) -> ConduitT i (v e) m r
- mapAccumS :: Monad m => (a -> s -> ConduitT b Void m s) -> s -> ConduitT () b m () -> ConduitT a Void m s
- peekForever :: Monad m => ConduitT i o m () -> ConduitT i o m ()
- peekForeverE :: (Monad m, MonoFoldable i) => ConduitT i o m () -> ConduitT i o m ()

# Producers

## Pure

yieldMany :: (Monad m, MonoFoldable mono) => mono -> ConduitT i (Element mono) m () #

Yield each of the values contained by the given `MonoFoldable`.

This will work on many data structures, including lists, `ByteString`s, and `Vector`s.

Subject to fusion

Since: 1.3.0

unfold :: Monad m => (b -> Maybe (a, b)) -> b -> ConduitT i a m () #

Generate a producer from a seed value.

Subject to fusion

Since: 1.3.0

enumFromTo :: (Monad m, Enum a, Ord a) => a -> a -> ConduitT i a m () #

Enumerate from a value to a final value, inclusive, via `succ`.

This is generally more efficient than using `Prelude`'s `enumFromTo` and combining with `sourceList` since this avoids any intermediate data structures.

Subject to fusion

Since: 1.3.0

iterate :: Monad m => (a -> a) -> a -> ConduitT i a m () #

Produces an infinite stream of repeated applications of f to x.

Subject to fusion

Since: 1.3.0

repeat :: Monad m => a -> ConduitT i a m () #

Produce an infinite stream consisting entirely of the given value.

Subject to fusion

Since: 1.3.0

replicate :: Monad m => Int -> a -> ConduitT i a m () #

Produce a finite stream consisting of n copies of the given value.

Subject to fusion

Since: 1.3.0

sourceLazy :: (Monad m, LazySequence lazy strict) => lazy -> ConduitT i strict m () #

Generate a producer by yielding each of the strict chunks in a `LazySequence`.

For more information, see `toChunks`.

Subject to fusion

Since: 1.3.0

## Monadic

repeatM :: Monad m => m a -> ConduitT i a m () #

Repeatedly run the given action and yield all values it produces.

Subject to fusion

Since: 1.3.0

repeatWhileM :: Monad m => m a -> (a -> Bool) -> ConduitT i a m () #

Repeatedly run the given action and yield all values it produces, until the provided predicate returns `False`.

Subject to fusion

Since: 1.3.0

replicateM :: Monad m => Int -> m a -> ConduitT i a m () #

Perform the given action n times, yielding each result.

Subject to fusion

Since: 1.3.0

## I/O

sourceFile :: MonadResource m => FilePath -> ConduitT i ByteString m () #

Stream the contents of a file as binary data.

Since: 1.3.0

sourceFileBS :: MonadResource m => FilePath -> ConduitT i ByteString m () #

Same as `sourceFile`. The alternate name is a holdover from an older version, when `sourceFile` was more polymorphic than it is today.

Since: 1.3.0

sourceHandle :: MonadIO m => Handle -> ConduitT i ByteString m () #

Stream the contents of a `Handle` as binary data. Note that this function will not automatically close the `Handle` when processing completes, since it did not acquire the `Handle` in the first place.

Since: 1.3.0

sourceHandleUnsafe :: MonadIO m => Handle -> ConduitT i ByteString m () #

Same as `sourceHandle`, but instead of allocating a new buffer for each incoming chunk of data, reuses the same buffer. Therefore, the `ByteString`s yielded by this function are not referentially transparent between two different `yield`s.

This function will be slightly more efficient than `sourceHandle` by avoiding allocations and reducing garbage collections, but should only be used if you can guarantee that you do not reuse a `ByteString` (or any slice thereof) between two calls to `await`.

Since: 1.3.0

sourceIOHandle :: MonadResource m => IO Handle -> ConduitT i ByteString m () #

An alternative to `sourceHandle`. Instead of taking a pre-opened `Handle`, it takes an action that opens a `Handle` (in read mode), so that it can open it only when needed and close it as soon as possible.

Since: 1.3.0

stdin :: MonadIO m => ConduitT i ByteString m () #

`sourceHandle` applied to `stdin`.

Subject to fusion

Since: 1.3.0

withSourceFile :: (MonadUnliftIO m, MonadIO n) => FilePath -> (ConduitM i ByteString n () -> m a) -> m a #

Like `withBinaryFile`, but provides a source to read bytes from.

Since: 1.3.0

## Filesystem

sourceDirectory :: MonadResource m => FilePath -> ConduitT i FilePath m () #

Stream the contents of the given directory, without traversing deeply.

This function will return all of the contents of the directory, whether they be files, directories, etc.

Note that the generated filepaths will be the complete path, not just the filename. In other words, if you have a directory `foo` containing files `bar` and `baz`, and you use `sourceDirectory` on `foo`, the results will be `foo/bar` and `foo/baz`.

Since: 1.3.0

sourceDirectoryDeep #

Arguments

:: MonadResource m |
=> Bool | Follow directory symlinks
-> FilePath | Root directory
-> ConduitT i FilePath m () |

Deeply stream the contents of the given directory.

This works the same as `sourceDirectory`, but will not return directories at all. This function also takes an extra parameter to indicate whether symlinks will be followed.

Since: 1.3.0

# Consumers

## Pure

drop :: Monad m => Int -> ConduitT a o m () #

Ignore a certain number of values in the stream.

Note: since this function doesn't produce anything, you probably want to use it with (`>>`) instead of directly plugging it into a pipeline:

    >>> runConduit $ yieldMany [1..5] .| drop 2 .| sinkList
    []
    >>> runConduit $ yieldMany [1..5] .| (drop 2 >> sinkList)
    [3,4,5]

Since: 1.3.0

dropE :: (Monad m, IsSequence seq) => Index seq -> ConduitT seq o m () #

Drop a certain number of elements from a chunked stream.

Note: you likely want to use it with monadic composition. See the docs for `drop`.

Since: 1.3.0

dropWhile :: Monad m => (a -> Bool) -> ConduitT a o m () #

Drop all values which match the given predicate.

Note: you likely want to use it with monadic composition. See the docs for `drop`.

Since: 1.3.0

dropWhileE :: (Monad m, IsSequence seq) => (Element seq -> Bool) -> ConduitT seq o m () #

Drop all elements in the chunked stream which match the given predicate.

Note: you likely want to use it with monadic composition. See the docs for `drop`.

Since: 1.3.0

fold :: (Monad m, Monoid a) => ConduitT a o m a #

Monoidally combine all values in the stream.

Subject to fusion

Since: 1.3.0

foldE :: (Monad m, MonoFoldable mono, Monoid (Element mono)) => ConduitT mono o m (Element mono) #

Monoidally combine all elements in the chunked stream.

Subject to fusion

Since: 1.3.0

foldl :: Monad m => (a -> b -> a) -> a -> ConduitT b o m a #

A strict left fold.

Subject to fusion

Since: 1.3.0

foldl1 :: Monad m => (a -> a -> a) -> ConduitT a o m (Maybe a) #

A strict left fold with no starting value. Returns `Nothing` when the stream is empty.

Subject to fusion

foldlE :: (Monad m, MonoFoldable mono) => (a -> Element mono -> a) -> a -> ConduitT mono o m a #

A strict left fold on a chunked stream.

Subject to fusion

Since: 1.3.0

foldMap :: (Monad m, Monoid b) => (a -> b) -> ConduitT a o m b #

Apply the provided mapping function and monoidal combine all values.

Subject to fusion

Since: 1.3.0

foldMapE :: (Monad m, MonoFoldable mono, Monoid w) => (Element mono -> w) -> ConduitT mono o m w #

Apply the provided mapping function and monoidal combine all elements of the chunked stream.

Subject to fusion

Since: 1.3.0

foldWhile :: Monad m => (a -> s -> Either e s) -> s -> ConduitT a o m (Either e s) #

Specialized version of `mapAccumWhile` that does not provide values downstream.

Since: 1.3.4

all :: Monad m => (a -> Bool) -> ConduitT a o m Bool #

Check that all values in the stream return True.

Subject to shortcut logic: at the first False, consumption of the stream will stop.

Subject to fusion

Since: 1.3.0

allE :: (Monad m, MonoFoldable mono) => (Element mono -> Bool) -> ConduitT mono o m Bool #

Check that all elements in the chunked stream return True.

Subject to shortcut logic: at the first False, consumption of the stream will stop.

Subject to fusion

Since: 1.3.0

any :: Monad m => (a -> Bool) -> ConduitT a o m Bool #

Check that at least one value in the stream returns True.

Subject to shortcut logic: at the first True, consumption of the stream will stop.

Subject to fusion

Since: 1.3.0

anyE :: (Monad m, MonoFoldable mono) => (Element mono -> Bool) -> ConduitT mono o m Bool #

Check that at least one element in the chunked stream returns True.

Subject to shortcut logic: at the first True, consumption of the stream will stop.

Subject to fusion

Since: 1.3.0

and :: Monad m => ConduitT Bool o m Bool #

Are all values in the stream True?

Consumption stops once the first False is encountered.

Subject to fusion

Since: 1.3.0

andE :: (Monad m, MonoFoldable mono, Element mono ~ Bool) => ConduitT mono o m Bool #

Are all elements in the chunked stream True?

Consumption stops once the first False is encountered.

Subject to fusion

Since: 1.3.0

or :: Monad m => ConduitT Bool o m Bool #

Are any values in the stream True?

Consumption stops once the first True is encountered.

Subject to fusion

Since: 1.3.0

orE :: (Monad m, MonoFoldable mono, Element mono ~ Bool) => ConduitT mono o m Bool #

Are any elements in the chunked stream True?

Consumption stops once the first True is encountered.

Subject to fusion

Since: 1.3.0

asum :: (Monad m, Alternative f) => ConduitT (f a) o m (f a) #

`Alternative`ly combine all values in the stream.

Since: 1.3.0

elem :: (Monad m, Eq a) => a -> ConduitT a o m Bool #

Are any values in the stream equal to the given value?

Stops consuming as soon as a match is found.

Subject to fusion

Since: 1.3.0

elemE :: (Monad m, IsSequence seq, Eq (Element seq)) => Element seq -> ConduitT seq o m Bool #

Are any elements in the chunked stream equal to the given element?

Stops consuming as soon as a match is found.

Subject to fusion

Since: 1.3.0

notElem :: (Monad m, Eq a) => a -> ConduitT a o m Bool #

Are no values in the stream equal to the given value?

Stops consuming as soon as a match is found.

Subject to fusion

Since: 1.3.0

notElemE :: (Monad m, IsSequence seq, Eq (Element seq)) => Element seq -> ConduitT seq o m Bool #

Are no elements in the chunked stream equal to the given element?

Stops consuming as soon as a match is found.

Subject to fusion

Since: 1.3.0

sinkLazy :: (Monad m, LazySequence lazy strict) => ConduitT strict o m lazy #

Consume all incoming strict chunks into a lazy sequence. Note that the entirety of the sequence will be resident at memory.

This can be used to consume a stream of strict ByteStrings into a lazy ByteString, for example.

Subject to fusion

Since: 1.3.0

sinkList :: Monad m => ConduitT a o m [a] #

Consume all values from the stream and return as a list. Note that this will pull all values into memory.

Subject to fusion

Since: 1.3.0

sinkVector :: (Vector v a, PrimMonad m) => ConduitT a o m (v a) #

Sink incoming values into a vector, growing the vector as necessary to fit more elements.

Note that using this function is more memory efficient than `sinkList` and then converting to a `Vector`, as it avoids intermediate list constructors.

Subject to fusion

Since: 1.3.0

sinkVectorN #

Arguments

:: (Vector v a, PrimMonad m) |
=> Int | maximum allowed size
-> ConduitT a o m (v a) |

Sink incoming values into a vector, up until size `maxSize`. Subsequent values will be left in the stream. If there are less than `maxSize` values present, returns a `Vector` of smaller size.

Note that using this function is more memory efficient than `sinkList` and then converting to a `Vector`, as it avoids intermediate list constructors.

Subject to fusion

Since: 1.3.0

sinkLazyBuilder :: Monad m => ConduitT Builder o m ByteString #

Same as `sinkBuilder`, but afterwards convert the builder to its lazy representation.

Alternatively, this could be considered an alternative to `sinkLazy`, with the following differences:

- This function will allow multiple input types, not just the strict version of the lazy structure.
- Some buffer copying may occur in this version.

Subject to fusion

Since: 1.3.0

sinkNull :: Monad m => ConduitT a o m () #

Consume and discard all remaining values in the stream.

Subject to fusion

Since: 1.3.0

awaitNonNull :: (Monad m, MonoFoldable a) => ConduitT a o m (Maybe (NonNull a)) #

Same as `await`, but discards any leading `onull` values.

Since: 1.3.0

head :: Monad m => ConduitT a o m (Maybe a) #

Take a single value from the stream, if available.

Since: 1.3.0

headDef :: Monad m => a -> ConduitT a o m a #

Same as `head`, but returns a default value if none are available from the stream.

Since: 1.3.0

headE :: (Monad m, IsSequence seq) => ConduitT seq o m (Maybe (Element seq)) #

Get the next element in the chunked stream.

Since: 1.3.0

peek :: Monad m => ConduitT a o m (Maybe a) #

View the next value in the stream without consuming it.

Since: 1.3.0

peekE :: (Monad m, MonoFoldable mono) => ConduitT mono o m (Maybe (Element mono)) #

View the next element in the chunked stream without consuming it.

Since: 1.3.0

last :: Monad m => ConduitT a o m (Maybe a) #

Retrieve the last value in the stream, if present.

Subject to fusion

Since: 1.3.0

lastDef :: Monad m => a -> ConduitT a o m a #

Same as `last`, but returns a default value if none are available from the stream.

Since: 1.3.0

lastE :: (Monad m, IsSequence seq) => ConduitT seq o m (Maybe (Element seq)) #

Retrieve the last element in the chunked stream, if present.

Subject to fusion

Since: 1.3.0

length :: (Monad m, Num len) => ConduitT a o m len #

Count how many values are in the stream.

Subject to fusion

Since: 1.3.0

lengthE :: (Monad m, Num len, MonoFoldable mono) => ConduitT mono o m len #

Count how many elements are in the chunked stream.

Subject to fusion

Since: 1.3.0

lengthIf :: (Monad m, Num len) => (a -> Bool) -> ConduitT a o m len #

Count how many values in the stream pass the given predicate.

Subject to fusion

Since: 1.3.0

lengthIfE :: (Monad m, Num len, MonoFoldable mono) => (Element mono -> Bool) -> ConduitT mono o m len #

Count how many elements in the chunked stream pass the given predicate.

Subject to fusion

Since: 1.3.0

maximum :: (Monad m, Ord a) => ConduitT a o m (Maybe a) #

Get the largest value in the stream, if present.

Subject to fusion

Since: 1.3.0

maximumE :: (Monad m, IsSequence seq, Ord (Element seq)) => ConduitT seq o m (Maybe (Element seq)) #

Get the largest element in the chunked stream, if present.

Subject to fusion

Since: 1.3.0

minimum :: (Monad m, Ord a) => ConduitT a o m (Maybe a) #

Get the smallest value in the stream, if present.

Subject to fusion

Since: 1.3.0

minimumE :: (Monad m, IsSequence seq, Ord (Element seq)) => ConduitT seq o m (Maybe (Element seq)) #

Get the smallest element in the chunked stream, if present.

Subject to fusion

Since: 1.3.0

null :: Monad m => ConduitT a o m Bool #

True if there are no values in the stream.

This function does not modify the stream.

Since: 1.3.0

nullE :: (Monad m, MonoFoldable mono) => ConduitT mono o m Bool #

True if there are no elements in the chunked stream.

This function may remove empty leading chunks from the stream, but otherwise will not modify it.

Since: 1.3.0

sum :: (Monad m, Num a) => ConduitT a o m a #

Get the sum of all values in the stream.

Subject to fusion

Since: 1.3.0

sumE :: (Monad m, MonoFoldable mono, Num (Element mono)) => ConduitT mono o m (Element mono) #

Get the sum of all elements in the chunked stream.

Subject to fusion

Since: 1.3.0

product :: (Monad m, Num a) => ConduitT a o m a #

Get the product of all values in the stream.

Subject to fusion

Since: 1.3.0

productE :: (Monad m, MonoFoldable mono, Num (Element mono)) => ConduitT mono o m (Element mono) #

Get the product of all elements in the chunked stream.

Subject to fusion

Since: 1.3.0

find :: Monad m => (a -> Bool) -> ConduitT a o m (Maybe a) #

Find the first matching value.

Subject to fusion

Since: 1.3.0

## Monadic

mapM_ :: Monad m => (a -> m ()) -> ConduitT a o m () #

Apply the action to all values in the stream.

Note: if you want to pass the values instead of consuming them, use `iterM` instead.

Subject to fusion

Since: 1.3.0

mapM_E :: (Monad m, MonoFoldable mono) => (Element mono -> m ()) -> ConduitT mono o m () #

Apply the action to all elements in the chunked stream.

Note: the same caveat as with `mapM_` applies. If you don't want to consume the values, you can use `iterM`:

    iterM (omapM_ f)

Subject to fusion

Since: 1.3.0

foldM :: Monad m => (a -> b -> m a) -> a -> ConduitT b o m a #

A monadic strict left fold.

Subject to fusion

Since: 1.3.0

foldME :: (Monad m, MonoFoldable mono) => (a -> Element mono -> m a) -> a -> ConduitT mono o m a #

A monadic strict left fold on a chunked stream.

Subject to fusion

Since: 1.3.0

foldMapM :: (Monad m, Monoid w) => (a -> m w) -> ConduitT a o m w #

Apply the provided monadic mapping function and monoidal combine all values.

Subject to fusion

Since: 1.3.0

foldMapME :: (Monad m, MonoFoldable mono, Monoid w) => (Element mono -> m w) -> ConduitT mono o m w #

Apply the provided monadic mapping function and monoidal combine all elements in the chunked stream.

Subject to fusion

Since: 1.3.0

## I/O

sinkFile :: MonadResource m => FilePath -> ConduitT ByteString o m () #

Stream all incoming data to the given file.

Since: 1.3.0

sinkFileCautious :: MonadResource m => FilePath -> ConduitM ByteString o m () #

Cautious version of `sinkFile`. The idea here is to stream the values to a temporary file in the same directory of the destination file, and only on successfully writing the entire file, moves it atomically to the destination path.

In the event of an exception occurring, the temporary file will be deleted and no move will be made. If the application shuts down without running exception handling (such as machine failure or a SIGKILL), the temporary file will remain and the destination file will be untouched.

Since: 1.3.0

sinkTempFile #

Arguments

:: MonadResource m |
=> FilePath | temp directory
-> String | filename pattern
-> ConduitM ByteString o m FilePath |

Stream data into a temporary file in the given directory with the given filename pattern, and return the temporary filename. The temporary file will be automatically deleted when exiting the active `ResourceT` block, if it still exists.

Since: 1.3.0

sinkSystemTempFile #

Arguments

:: MonadResource m |
=> String | filename pattern
-> ConduitM ByteString o m FilePath |

Same as `sinkTempFile`, but will use the default temp file directory for the system as the first argument.

Since: 1.3.0

sinkFileBS :: MonadResource m => FilePath -> ConduitT ByteString o m () #

`sinkFile` specialized to `ByteString` to help with type inference.

Since: 1.3.0

sinkHandle :: MonadIO m => Handle -> ConduitT ByteString o m () #

Stream all incoming data to the given `Handle`. Note that this function does not flush and will not close the `Handle` when processing completes.

Since: 1.3.0

sinkIOHandle :: MonadResource m => IO Handle -> ConduitT ByteString o m () #

An alternative to `sinkHandle`. Instead of taking a pre-opened `Handle`, it takes an action that opens a `Handle` (in write mode), so that it can open it only when needed and close it as soon as possible.

Since: 1.3.0

print :: (Show a, MonadIO m) => ConduitT a o m () #

Print all incoming values to stdout.

Subject to fusion

Since: 1.3.0

stdout :: MonadIO m => ConduitT ByteString o m () #

`sinkHandle` applied to `stdout`.

Subject to fusion

Since: 1.3.0

stderr :: MonadIO m => ConduitT ByteString o m () #

`sinkHandle` applied to `stderr`.

Subject to fusion

Since: 1.3.0

withSinkFile :: (MonadUnliftIO m, MonadIO n) => FilePath -> (ConduitM ByteString o n () -> m a) -> m a #

Like `withBinaryFile`, but provides a sink to write bytes to.

Since: 1.3.0

withSinkFileBuilder :: (MonadUnliftIO m, MonadIO n) => FilePath -> (ConduitM Builder o n () -> m a) -> m a #

Same as `withSinkFile`, but lets you use a `Builder`.

Since: 1.3.0

withSinkFileCautious :: (MonadUnliftIO m, MonadIO n) => FilePath -> (ConduitM ByteString o n () -> m a) -> m a #

Like `sinkFileCautious`, but uses the `with` pattern instead of `MonadResource`.

Since: 1.3.0

sinkHandleBuilder :: MonadIO m => Handle -> ConduitM Builder o m () #

Stream incoming builders, executing them directly on the buffer of the given `Handle`. Note that this function does not automatically close the `Handle` when processing completes. Pass `flush` to flush the buffer.

Since: 1.3.0

sinkHandleFlush :: MonadIO m => Handle -> ConduitM (Flush ByteString) o m () #

Stream incoming `Flush`es, executing them on `IO.Handle` Note that this function does not automatically close the `Handle` when processing completes

Since: 1.3.0

# Transformers

## Pure

map :: Monad m => (a -> b) -> ConduitT a b m () #

Apply a transformation to all values in a stream.

Subject to fusion

Since: 1.3.0

mapE :: (Monad m, Functor f) => (a -> b) -> ConduitT (f a) (f b) m () #

Apply a transformation to all elements in a chunked stream.

Subject to fusion

Since: 1.3.0

omapE :: (Monad m, MonoFunctor mono) => (Element mono -> Element mono) -> ConduitT mono mono m () #

Apply a monomorphic transformation to all elements in a chunked stream.

Unlike `mapE`, this will work on types like `ByteString` and `Text` which are `MonoFunctor` but not `Functor`.

Subject to fusion

Since: 1.3.0

concatMap :: (Monad m, MonoFoldable mono) => (a -> mono) -> ConduitT a (Element mono) m () #

Apply the function to each value in the stream, resulting in a foldable value (e.g., a list). Then yield each of the individual values in that foldable value separately.

Generalizes concatMap, mapMaybe, and mapFoldable.

Subject to fusion

Since: 1.3.0

concatMapE :: (Monad m, MonoFoldable mono, Monoid w) => (Element mono -> w) -> ConduitT mono w m () #

Apply the function to each element in the chunked stream, resulting in a foldable value (e.g., a list). Then yield each of the individual values in that foldable value separately.

Generalizes concatMap, mapMaybe, and mapFoldable.

Subject to fusion

Since: 1.3.0

take :: Monad m => Int -> ConduitT a a m () #

Stream up to n number of values downstream.

Note that, if downstream terminates early, not all values will be consumed. If you want to force exactly the given number of values to be consumed, see `takeExactly`.

Subject to fusion

Since: 1.3.0

takeE :: (Monad m, IsSequence seq) => Index seq -> ConduitT seq seq m () #

Stream up to n number of elements downstream in a chunked stream.

Note that, if downstream terminates early, not all values will be consumed. If you want to force exactly the given number of values to be consumed, see `takeExactlyE`.

Since: 1.3.0

takeWhile :: Monad m => (a -> Bool) -> ConduitT a a m () #

Stream all values downstream that match the given predicate.

Same caveats regarding downstream termination apply as with `take`.

Since: 1.3.0

takeWhileE :: (Monad m, IsSequence seq) => (Element seq -> Bool) -> ConduitT seq seq m () #

Stream all elements downstream that match the given predicate in a chunked stream.

Same caveats regarding downstream termination apply as with `takeE`.

Since: 1.3.0

takeExactly :: Monad m => Int -> ConduitT a b m r -> ConduitT a b m r #

Consume precisely the given number of values and feed them downstream.

This function is in contrast to `take`, which will only consume up to the given number of values, and will terminate early if downstream terminates early. This function will discard any additional values in the stream if they are unconsumed.

Note that this function takes a downstream `ConduitT` as a parameter, as opposed to working with normal fusion. For more information, see http://www.yesodweb.com/blog/2013/10/core-flaw-pipes-conduit, the section titled "pipes and conduit: isolate".

Since: 1.3.0

takeExactlyE :: (Monad m, IsSequence a) => Index a -> ConduitT a b m r -> ConduitT a b m r #

Same as `takeExactly`, but for chunked streams.

Since: 1.3.0

concat :: (Monad m, MonoFoldable mono) => ConduitT mono (Element mono) m () #

Flatten out a stream by yielding the values contained in an incoming `MonoFoldable` as individually yielded values.

Subject to fusion

Since: 1.3.0

filter :: Monad m => (a -> Bool) -> ConduitT a a m () #

Keep only values in the stream passing a given predicate.

Subject to fusion

Since: 1.3.0

filterE :: (IsSequence seq, Monad m) => (Element seq -> Bool) -> ConduitT seq seq m () #

Keep only elements in the chunked stream passing a given predicate.

Subject to fusion

Since: 1.3.0

mapWhile :: Monad m => (a -> Maybe b) -> ConduitT a b m () #

Map values as long as the result is `Just`.

Since: 1.3.0

conduitVector #

Arguments

:: (Vector v a, PrimMonad m) |
=> Int | maximum allowed size
-> ConduitT a (v a) m () |

Break up a stream of values into vectors of size n. The final vector may be smaller than n if the total number of values is not a strict multiple of n. No empty vectors will be yielded.

Since: 1.3.0

scanl :: Monad m => (a -> b -> a) -> a -> ConduitT b a m () #

Analog of `scanl` for lists.

Subject to fusion

Since: 1.3.0

mapAccumWhile :: Monad m => (a -> s -> Either s (s, b)) -> s -> ConduitT a b m s #

`mapWhile` with a break condition dependent on a strict accumulator. Equivalently, `mapAccum` as long as the result is `Right`. Instead of producing a leftover, the breaking input determines the resulting accumulator via `Left`.

Subject to fusion

concatMapAccum :: Monad m => (a -> accum -> (accum, [b])) -> accum -> ConduitT a b m () #

`concatMap` with an accumulator.

Subject to fusion

Since: 1.3.0

intersperse :: Monad m => a -> ConduitT a a m () #

Insert the given value between each two values in the stream.

Subject to fusion

Since: 1.3.0

slidingWindow :: (Monad m, IsSequence seq, Element seq ~ a) => Int -> ConduitT a seq m () #

Sliding window of values 1,2,3,4,5 with window size 2 gives [1,2],[2,3],[3,4],[4,5]

Best used with structures that support O(1) snoc.

Subject to fusion

Since: 1.3.0

chunksOfE :: (Monad m, IsSequence seq) => Index seq -> ConduitT seq seq m () #

Split input into chunk of size `chunkSize`

The last element may be smaller than the `chunkSize` (see also `chunksOfExactlyE` which will not yield this last element)

Since: 1.3.0

chunksOfExactlyE :: (Monad m, IsSequence seq) => Index seq -> ConduitT seq seq m () #

Split input into chunk of size `chunkSize`

If the input does not split into chunks exactly, the remainder will be leftover (see also `chunksOfE`)

Since: 1.3.0

## Monadic

mapM :: Monad m => (a -> m b) -> ConduitT a b m () #

Apply a monadic transformation to all values in a stream.

If you do not need the transformed values, and instead just want the monadic side-effects of running the action, see `mapM_`.

Subject to fusion

Since: 1.3.0

mapME :: (Monad m, Traversable f) => (a -> m b) -> ConduitT (f a) (f b) m () #

Apply a monadic transformation to all elements in a chunked stream.

Subject to fusion

Since: 1.3.0

omapME :: (Monad m, MonoTraversable mono) => (Element mono -> m (Element mono)) -> ConduitT mono mono m () #

Apply a monadic monomorphic transformation to all elements in a chunked stream.

Unlike `mapME`, this will work on types like `ByteString` and `Text` which are `MonoFunctor` but not `Functor`.

Subject to fusion

Since: 1.3.0

concatMapM :: (Monad m, MonoFoldable mono) => (a -> m mono) -> ConduitT a (Element mono) m () #

Apply the monadic function to each value in the stream, resulting in a foldable value (e.g., a list). Then yield each of the individual values in that foldable value separately.

Generalizes concatMapM, mapMaybeM, and mapFoldableM.

Subject to fusion

Since: 1.3.0

filterM :: Monad m => (a -> m Bool) -> ConduitT a a m () #

Keep only values in the stream passing a given monadic predicate.

Subject to fusion

Since: 1.3.0

filterME :: (Monad m, IsSequence seq) => (Element seq -> m Bool) -> ConduitT seq seq m () #

Keep only elements in the chunked stream passing a given monadic predicate.

Subject to fusion

Since: 1.3.0

iterM :: Monad m => (a -> m ()) -> ConduitT a a m () #

Apply a monadic action on all values in a stream.

This `Conduit` can be used to perform a monadic side-effect for every value, whilst passing the value through the `Conduit` as-is.

    iterM f = mapM (\a -> f a >>= \() -> return a)

Subject to fusion

Since: 1.3.0

scanlM :: Monad m => (a -> b -> m a) -> a -> ConduitT b a m () #

Analog of `scanl` for lists, monadic.

Subject to fusion

Since: 1.3.0

mapAccumWhileM :: Monad m => (a -> s -> m (Either s (s, b))) -> s -> ConduitT a b m s #

Monadic `mapAccumWhile`.

Subject to fusion

concatMapAccumM :: Monad m => (a -> accum -> m (accum, [b])) -> accum -> ConduitT a b m () #

`concatMapM` with an accumulator.

Subject to fusion

Since: 1.3.0

## Textual

encodeUtf8 :: (Monad m, Utf8 text binary) => ConduitT text binary m () #

Encode a stream of text as UTF8.

Subject to fusion

Since: 1.3.0

decodeUtf8 :: MonadThrow m => ConduitT ByteString Text m () #

Decode a stream of binary data as UTF8.

Since: 1.3.0

decodeUtf8Lenient :: Monad m => ConduitT ByteString Text m () #

Decode a stream of binary data as UTF8, replacing any invalid bytes with the Unicode replacement character.

Since: 1.3.0

line :: (Monad m, IsSequence seq, Element seq ~ Char) => ConduitT seq o m r -> ConduitT seq o m r #

Stream in the entirety of a single line.

Like `takeExactly`, this will consume the entirety of the line regardless of the behavior of the inner Conduit.

Since: 1.3.0

lineAscii :: (Monad m, IsSequence seq, Element seq ~ Word8) => ConduitT seq o m r -> ConduitT seq o m r #

Same as `line`, but operates on ASCII/binary data.

Since: 1.3.0

unlines :: (Monad m, IsSequence seq, Element seq ~ Char) => ConduitT seq seq m () #

Insert a newline character after each incoming chunk of data.

Subject to fusion

Since: 1.3.0

unlinesAscii :: (Monad m, IsSequence seq, Element seq ~ Word8) => ConduitT seq seq m () #

Same as `unlines`, but operates on ASCII/binary data.

Subject to fusion

Since: 1.3.0

takeExactlyUntilE :: (Monad m, IsSequence seq) => (Element seq -> Bool) -> ConduitT seq o m r -> ConduitT seq o m r #

Stream in the chunked input until an element matches a predicate.

Like `takeExactly`, this will consume the entirety of the prefix regardless of the behavior of the inner Conduit.

linesUnbounded :: (Monad m, IsSequence seq, Element seq ~ Char) => ConduitT seq seq m () #

Convert a stream of arbitrarily-chunked textual data into a stream of data where each chunk represents a single line. Note that, if you have unknown or untrusted input, this function is unsafe, since it would allow an attacker to form lines of massive length and exhaust memory.

Subject to fusion

Since: 1.3.0

linesUnboundedAscii :: (Monad m, IsSequence seq, Element seq ~ Word8) => ConduitT seq seq m () #

Same as `linesUnbounded`, but for ASCII/binary data.

Subject to fusion

Since: 1.3.0

splitOnUnboundedE :: (Monad m, IsSequence seq) => (Element seq -> Bool) -> ConduitT seq seq m () #

Split a stream of arbitrarily-chunked data, based on a predicate on elements. Elements that satisfy the predicate will cause chunks to be split, and aren't included in these output chunks. Note that, if you have unknown or untrusted input, this function is unsafe, since it would allow an attacker to form chunks of massive length and exhaust memory.

## Builders

builderToByteString :: PrimMonad m => ConduitT Builder ByteString m () #

Incrementally execute builders and pass on the filled chunks as bytestrings.

Since: 1.3.0

unsafeBuilderToByteString :: PrimMonad m => ConduitT Builder ByteString m () #

Incrementally execute builders on the given buffer and pass on the filled chunks as bytestrings. Note that, if the given buffer is too small for the execution of a build step, a larger one will be allocated.

WARNING: This conduit yields bytestrings that are NOT referentially transparent. Their content will be overwritten as soon as control is returned from the inner sink!

Since: 1.3.0

builderToByteStringWith :: PrimMonad m => BufferAllocStrategy -> ConduitT Builder ByteString m () #

A conduit that incrementally executes builders and passes on the filled chunks as bytestrings to an inner sink.

INV: All bytestrings passed to the inner sink are non-empty.

Since: 1.3.0

builderToByteStringFlush :: PrimMonad m => ConduitT (Flush Builder) (Flush ByteString) m () #

Same as `builderToByteString`, but input and output are wrapped in `Flush`.

Since: 1.3.0

builderToByteStringWithFlush :: PrimMonad m => BufferAllocStrategy -> ConduitT (Flush Builder) (Flush ByteString) m () #

Since: 1.3.0

type BufferAllocStrategy = (IO Buffer, Int -> Buffer -> IO (IO Buffer)) #

A buffer allocation strategy `(buf0, nextBuf)` specifies the initial buffer to use and how to compute a new buffer `nextBuf minSize buf` with at least size `minSize` from a filled buffer `buf`. The double nesting of the `IO` monad helps to ensure that the reference to the filled buffer `buf` is lost as soon as possible, but the new buffer doesn't have to be allocated too early.

Since: 1.3.0

allNewBuffersStrategy :: Int -> BufferAllocStrategy #

The simplest buffer allocation strategy: whenever a buffer is requested, allocate a new one that is big enough for the next build step to execute.

NOTE that this allocation strategy may spill quite some memory upon direct insertion of a bytestring by the builder. Thats no problem for garbage collection, but it may lead to unreasonably high memory consumption in special circumstances.

Since: 1.3.0

reuseBufferStrategy :: IO Buffer -> BufferAllocStrategy #

An unsafe, but possibly more efficient buffer allocation strategy: reuse the buffer, if it is big enough for the next build step to execute.

Since: 1.3.0

# Special

vectorBuilder #

Arguments

:: (PrimMonad m, PrimMonad n, Vector v e, PrimState m ~ PrimState n) |
=> Int | size
-> ((e -> n ()) -> ConduitT i Void m r) |
-> ConduitT i (v e) m r |

Generally speaking, yielding values from inside a Conduit requires some allocation for constructors. This can introduce an overhead, similar to the overhead needed to represent a list of values instead of a vector. This overhead is even more severe when talking about unboxed values.

This combinator allows you to overcome this overhead, and efficiently fill up vectors. It takes two parameters. The first is the size of each mutable vector to be allocated. The second is a function. The function takes an argument which will yield the next value into a mutable vector.

Under the surface, this function uses a number of tricks to get high performance. For more information on both usage and implementation, please see: https://www.schoolofhaskell.com/user/snoyberg/library-documentation/vectorbuilder

Since: 1.3.0

mapAccumS :: Monad m => (a -> s -> ConduitT b Void m s) -> s -> ConduitT () b m () -> ConduitT a Void m s #

Consume a source with a strict accumulator, in a way piecewise defined by a controlling stream. The latter will be evaluated until it terminates.

    >>> let f a s = liftM (:s) $ mapC (*a) =$ CL.take a
    >>> reverse $ runIdentity $ yieldMany [0..3] $$ mapAccumS f [] (yieldMany [1..])
    [[],[1],[4,6],[12,15,18]] :: [[Int]]

peekForever :: Monad m => ConduitT i o m () -> ConduitT i o m () #

Run a consuming conduit repeatedly, only stopping when there is no more data available from upstream.

Since: 1.3.0

peekForeverE :: (Monad m, MonoFoldable i) => ConduitT i o m () -> ConduitT i o m () #

Run a consuming conduit repeatedly, only stopping when there is no more data available from upstream.

In contrast to `peekForever`, this function will ignore empty chunks of data. So for example, if a stream of data contains an empty `ByteString`, it is still treated as empty, and the consuming function is not called.

Since: 1.3.0