Temporal Workers with Go: a roadmap
Why this matters, in 30 seconds
The Temporal Service never runs your code. Workers do. They poll Task Queues, run your Workflows and Activities, and report back. When a Workflow is "stuck", the cause is almost always on the Worker side: nobody polls that Task Queue, the type isn't registered, the Worker is out of slots, or a deploy killed it mid-Activity.
Getting Workers right decides:
- Whether work is picked up at all: same Task Queue name, same registrations.
- How fast and how much: slots, pollers, rate limits, the Workflow cache.
- Where it runs: dedicated queues, per-host routing, Sessions.
- What survives a deploy: graceful shutdown and Worker Versioning.
The roadmap
Stage 1: Foundations (core concepts, must-know)
| # | Chapter | You'll be able to… |
|---|---|---|
| 01 | What is a Worker? | Tell a Worker Program, Process and Entity apart, explain why the Service never runs your code, and run a first Worker. |
| 02 | Tasks | Name the three kinds of Tasks, say when a Workflow Task is scheduled, and tell a Workflow Task failure from a Workflow Execution failure. |
| 03 | Task Queues | Explain how Workers find work, apply the identical-registration rule, set Task Queues in the right places, and catch a name mismatch. |
Stage 2: Daily use (running and sizing Workers)
| # | Chapter | You'll be able to… |
|---|---|---|
| 04 | Running a Worker in Go | Create, register and run Workers, host several Task Queues in one process, connect to Cloud, and set a useful Worker Identity. |
| 05 | Worker options | Size slots and pollers, rate-limit per Worker or per Task Queue, and run Activity-only Workers. |
| 06 | Sticky Execution | Explain the sticky queue and the Workflow cache, and tune StickyScheduleToStartTimeout and the cache size. |
Stage 3: Advanced (routing and cross-cutting code)
| # | Chapter | You'll be able to… |
|---|---|---|
| 07 | Task Routing | Send Activities to special environments, pin a pipeline to one host, and handle priorities. |
| 08 | Worker Sessions | Run a series of Activities on one Worker with the Session API, and handle Worker loss correctly. |
| 09 | Interceptors | Add auditing, metrics and logging around every Client call, Workflow and Activity, and test it. |
Stage 4: Production
| # | Chapter | You'll be able to… |
|---|---|---|
| 10 | Graceful shutdown | Set WorkerStopTimeout, write Activities that checkpoint on shutdown, and wire shutdown into your platform. |
| 11 | Worker Versioning | Ship new Workflow code with Pinned and Auto-Upgrade Workflows, ramp a new Version and drain the old one. |
| 12 | Serverless Workers | Know when Temporal can start Workers for you, and run a Worker on AWS Lambda or GCP Cloud Run. |
Core concepts vs. implementation details
| Core concepts: understand these first | Implementation details, options & sub-features: learn when needed |
|---|---|
| Workers poll; the Service only hands out Tasks (01) | Worker Program / Process / Entity vocabulary, Data Converter (01) |
| Workflow, Activity and Nexus Tasks (02) | Workflow Task failure vs Workflow Execution failure (02) |
| Task Queues, created on demand (03) | Partitions, ordering, sync match (03) |
| Same Task Queue ⇒ same registrations (03) | Custom names, DisableRegistrationAliasing (04) |
worker.New → Register… → Run (04) | Start/Stop, several Workers per process, Identity, OnFatalError (04) |
| Slots and pollers (05) | Rate limits, poller autoscaling, eager Activities, Tuner (05) |
| Sticky Execution (06) | Cache size, StickyScheduleToStartTimeout (06) |
| Task Routing with dedicated queues (07) | Per-host queues, priorities and fairness (07), Sessions (08) |
| — | Interceptors (09) |
| Graceful shutdown (10) | Worker stop channel, heartbeat checkpoints, Local Activities on shutdown (10) |
| Worker Deployments and Versions (11) | Ramping, inheritance rules, WorkerDeploymentClient (11) |
| — | Serverless Workers, lambdaworker, Cloud Run (12) |
Which Worker setup do I need?
text
Is a Workflow stuck in Running with no progress?
├── No pollers on its Task Queue? → Task Queue name mismatch / no Worker → chapter 03
├── "unable to find workflow type"? → register it on every Worker of that queue → chapters 03, 04
└── Backlog grows, Workers busy? → slots, pollers, more Workers → chapter 05
Must some Activities run in a special place?
├── A kind of machine (GPU, PCI zone) → a dedicated Task Queue + Worker pool → chapter 07
└── The same host as a previous Activity
├── Go, simple → Worker Sessions → chapter 08
└── Full control → per-host Task Queue → chapter 07
Must a downstream API not be flooded?
├── Per Worker → WorkerActivitiesPerSecond → chapter 05
└── Across all Workers → TaskQueueActivitiesPerSecond → chapter 05
Will you deploy Workers while Workflows are running?
├── Activities in flight → WorkerStopTimeout + heartbeats → chapter 10
└── Workflow code changes → Worker Versioning (or patching) → chapter 11
Don't want to run Worker processes at all? → Serverless Workers (Public Preview) → chapter 12Go API cheat sheet
| I want to… | Go API | Chapter |
|---|---|---|
| Create a Worker | worker.New(c, taskQueue, worker.Options{}) | 04 |
| Register code | w.RegisterWorkflow(fn), w.RegisterActivity(&Activities{…}), …WithOptions(fn, workflow.RegisterOptions{Name: …}) | 04 |
| Run until SIGINT/SIGTERM | w.Run(worker.InterruptCh()) | 04, 10 |
| Run inside a service | w.Start() … w.Stop() | 04, 10 |
| Name the Worker | worker.Options{Identity: …} (default pid@hostname@) | 04 |
| Size concurrency | MaxConcurrentActivityExecutionSize, MaxConcurrentWorkflowTaskExecutionSize, MaxConcurrent…TaskPollers | 05 |
| Rate-limit Activities | WorkerActivitiesPerSecond (per Worker), TaskQueueActivitiesPerSecond (whole queue) | 05 |
| Activity-only Worker | worker.Options{DisableWorkflowWorker: true} | 05 |
| Tune the Workflow cache | worker.SetStickyWorkflowCacheSize(n), StickyScheduleToStartTimeout | 06 |
| Route an Activity | workflow.ActivityOptions{TaskQueue: …} | 03, 07 |
| Prioritize a Task | Priority: temporal.Priority{PriorityKey: …, FairnessKey: …} | 07 |
| Keep Activities on one Worker | EnableSessionWorker: true, workflow.CreateSession, workflow.CompleteSession | 08 |
| Detect a lost Session | workflow.GetSessionInfo(ctx).SessionState == workflow.SessionStateFailed | 08 |
| Add an interceptor | client.Options{Interceptors: …}, worker.Options{Interceptors: …}, interceptor.WorkerInterceptorBase | 09 |
| Shut down gracefully | worker.Options{WorkerStopTimeout: …}, activity.GetWorkerStopChannel(ctx) | 10 |
| Run a versioned Worker | DeploymentOptions{UseVersioning: true, Version: worker.WorkerDeploymentVersion{…}}, RegisterOptions{VersioningBehavior: …} | 11 |
| Promote a Version | temporal worker deployment set-current-version, c.WorkerDeploymentClient().GetHandle(…).SetCurrentVersion(…) | 11 |
| Run on AWS Lambda | lambdaworker.RunWorker(version, func(o *lambdaworker.Options) error {…}) | 12 |
About the code samples
- Each chapter is a self-contained module named
github.com/acme/shop. Every Go block starts with// file: <path>so you can paste it into that path. - All samples were compiled,
gofmt-checked andgo vet-checked against Go SDK v1.50.0 (go.temporal.io/sdk),go.temporal.io/sdk/contrib/envconfigv1.0.2 andgo.temporal.io/sdk/contrib/aws/lambdaworkerv0.1.1. The interceptor unit test in chapter 09 passes. - Behaviors were checked on a dev server (Temporal CLI 1.9.1, Server 1.32.0): unknown types and Task Queue typos (03), registration panics (04, 11), the sticky → normal queue fallback (06), per-host routing and its fallback (07), Sessions including Worker loss and Continue-As-New (08), the three
WorkerStopTimeoutcases and heartbeat resume (10), and a Pinned vs Auto-Upgrade rollout from v1 to v2 (11). - External systems are interfaces or fakes. Run everything locally with
temporal server start-dev(UI at http://localhost:8233).
Notes from checking the docs against the SDK
WorkerStopTimeoutdefaults to 0. The Go "Run a Worker" page says the Worker waits for in-flight Tasks "up to theWorkerStopTimeout", but not that the default is no wait. With the default,Stop()returns at once, the Activity context is canceled and the attempt is retried (chapter 10).- "Replays from the beginning". The Tasks page says every Workflow Task replays the whole history. That only happens on a cache miss; with Sticky Execution the Worker applies just the new events (chapters 02 and 06).
- Default Worker Identity. The Encyclopedia says
pid@hostname. In Go it ispid@hostname@(trailing@), inherited from the Client (chapter 04). - Registration without a versioning behavior. The Go page says it fails at registration. The SDK panics only when
UseVersioningand aVersionare set.DefaultVersioningBehaviorwithoutUseVersioningpanics earlier, inworker.New(chapter 11). - Session failure handling. When the Session's Worker disappears during an Activity, that Activity returns a canceled error, not
ErrSessionFailed; only later Activities getErrSessionFailed. Retrying only onErrSessionFailedfails the Workflow (seen on a dev server). CheckGetSessionInfo(…).SessionStateinstead. A graceful stop fails the Session immediately (chapter 08). - Session limit.
MaxConcurrentSessionExecutionSizedefaults to 1000, not "a very large value", and applies per Worker, not per process (chapter 08). - Sessions and Worker Versioning. The SDK says Sessions are not compatible with versioning, but only rejects the deprecated
UseBuildIDForVersioning. WithDeploymentOptions.UseVersioningthere is no check. Test before combining them (chapter 08). - Server-side rate limit disables eager Activities. Setting
TaskQueueActivitiesPerSecondturns off eager Activity execution (SDK doc comment). The Task Routing page doesn't say so (chapter 05). - No option reference. The Go "Run a Worker" page links
worker.Optionsto theinternalpackage and says only that an empty struct works. Chapter 05 has a defaults table read from the source. - Fatal errors. After startup only
NamespaceNotFound,InvalidArgumentandClientVersionNotSupportedstop a Worker (and callOnFatalError). A Worker with nothing registered starts without error (chapter 04). lambdaworker.ShutdownDeadlineBufferis computed from the defaultWorkerStopTimeout(5s) before your configure callback runs, so raisingWorkerStopTimeoutthere leaves the buffer at 7s. Set both. The package also readsTEMPORAL_TASK_QUEUEand sets the identity torequestID@functionARN; neither is on the Go page (chapter 12).- CLI rollout.
set-current-versionandset-ramping-versionask for confirmation; use--yesin CI (chapter 11).
Sources
Encyclopedia: Workers · Tasks · Task Queues · Task Queue naming · Task Routing · Sticky Execution · Worker shutdown · Worker Versioning · Serverless Workers
Go SDK: Run a Worker · Worker Sessions · Interceptors · Serverless Workers · linked from it: AWS Lambda · GCP Cloud Run