What is Stream Processing
Java developers should be very familiar with Stream API in Java, which greatly improves the ability to handle data collections.
int sum = widgets.stream()
.filter(w -> w.getColor() == RED)
.mapToInt(w -> w.getWeight())
.sum();
The idea of a Stream is to abstract the data processing into a data stream and return a new stream for use after each process.
Stream Function Definition in Golang
The most important step is to think through the requirements before writing the code, so let’s try to put ourselves in the author’s shoes and think about the flow of the component. First, let’s put the underlying implementation logic aside and try to define the stream function from scratch.
Stream’s workflow is part of the production-consumer model, and the whole process is very similar to the production process in a factory.
- creation phase/data acquisition (raw material)
- processing phase/intermediate processing (pipeline processing)
- aggregation stage/final operation (final product)
The API is defined around the three life cycles of a stream.
Creation Phase
To create the abstract object stream, it can be understood as a constructor.
We support three ways of constructing streams: slicing conversion, channel conversion, and functional conversion.
Note that the methods in this phase are normal public methods and are not bound to the stream object.
// Create stream by variable parameter pattern
func Just(items ... .interface{}) Stream// Create a stream via channel
func Range(source <-chan interface{}) Stream// Create stream by function
func From(generate GenerateFunc) Stream// Concatenate a stream
func Concat(s Stream, others . . Stream) Stream
Processing phase
The operations required in the processing phase often correspond to our business logic, such as conversion, filtering, de-duplication, sorting, and so on.
The API for this phase is a method that needs to be bound to a Stream object.
The following definition is combined with common business scenarios.
