The best “Application programmers” are cultured to focus on business requirements and typically don’t always factor in the need for concurrency — and that’s the way it should be. We don’t think of concurrency first — we think of “business requirements first”.

Often the complexities involved in writing concurrent code are abstracted inside a Go library or a third-party package (e.g. HTTP server handlers) but it is not always the case. Now and then, there is a requirement to scale up functionality in a way that cannot leverage existing libraries. Every programmer must know to write concurrent code — a functionally correct code is useless if it cannot scale up — and vice versa.

In this blog post, I will provide step-by-step instructions to make “sequential” code “concurrent.” I will start with a sequential code and then refactor it to make it concurrent. This is how I do it in real life — I focus on business functionality first, write a sequential correct version of my code and then refactor to make it more concurrent.

Addressing the business functionality — The sequential code

Step number one is to write sequential code right. Let us encapsulate business functionality into one function.

businessFunctionality() response

Breaking down the business functionality

Come up with some intelligent way to break the functionality into small chunks. You cannot parallelize it if you cannot modularize it.

For exploring data, an option to explore data based on dates is worth exploring. For example, a business functionality can explore data from Jan 1 to Dec 31. This can be broken into 365 (or 366) jobs — where each job takes care of exploring data for one day.

func businessFunctionalityJob(startTime, endTime time.Time) JobResponse
{
    ...
}

And the overall flow will be something like this:

func businessFunctionality() response
{
    jobStartTime := //startTime
    jobEndTime := //endTime
    jobWindow := 24 * time.Hour    var jobResponses []JobResponse
    for currentTime := jobStartTime; currentTime.Before(jobEndTime);        
           currentTime = jobStartTime.Add(jobWindow) {
        jobResponses = append(jobResponses,  
                  businessFunctionality(jobStartTime, jobEndtime))    }    return combineResponses(jobResp)
}

Alternatively, you can look at breaking the business functionality with some number range

func businessFunctionalityJob(start, end int64) response {
    ...
}

More generically,

type JobInput struct {
   startTime time.Time
   endTime   time.Time
}type JobResponse struct { //keeping this empty for this example. Real-world will be more complex.
}type Response struct {
   finalOutput string
}func businessFunctionality() Response {
   var jobInputs []JobInput //Create job input for each job
   var jobResponses []JobResponse
   for _, jobInput := range jobInputs {
      jobResponses = append(jobResponses,  
                      businessFunctionalityJob(jobInput))
   }
   return combineResponses(jobResponses)
}func combineResponses(jobResponses []JobResponse) Response {
   return Response{finalOutput: "Well done!!"}
}func businessFunctionalityJob(jobInput JobInput) JobResponse {
   return JobResponse{}
}func main() {
   fmt.Println(businessFunctionality().finalOutput)
}

Making it parallel

So far, we have broken down the functionality but have not made any progress in improving performance. We can easily make it parallel by starting multiple go routines.