Skip to content

Temporal Workers with Go: a roadmap ​

Why this matters, in 30 seconds ​

The Temporal Service never runs your code. Workers do. They poll Task Queues, run your Workflows and Activities, and report back. When a Workflow is "stuck", the cause is almost always on the Worker side: nobody polls that Task Queue, the type isn't registered, the Worker is out of slots, or a deploy killed it mid-Activity.

Getting Workers right decides:

  • Whether work is picked up at all: same Task Queue name, same registrations.
  • How fast and how much: slots, pollers, rate limits, the Workflow cache.
  • Where it runs: dedicated queues, per-host routing, Sessions.
  • What survives a deploy: graceful shutdown and Worker Versioning.

The roadmap ​

Stage 1: Foundations (core concepts, must-know) ​

#ChapterYou'll be able to…
01What is a Worker?Tell a Worker Program, Process and Entity apart, explain why the Service never runs your code, and run a first Worker.
02TasksName the three kinds of Tasks, say when a Workflow Task is scheduled, and tell a Workflow Task failure from a Workflow Execution failure.
03Task QueuesExplain how Workers find work, apply the identical-registration rule, set Task Queues in the right places, and catch a name mismatch.

Stage 2: Daily use (running and sizing Workers) ​

#ChapterYou'll be able to…
04Running a Worker in GoCreate, register and run Workers, host several Task Queues in one process, connect to Cloud, and set a useful Worker Identity.
05Worker optionsSize slots and pollers, rate-limit per Worker or per Task Queue, and run Activity-only Workers.
06Sticky ExecutionExplain the sticky queue and the Workflow cache, and tune StickyScheduleToStartTimeout and the cache size.

Stage 3: Advanced (routing and cross-cutting code) ​

#ChapterYou'll be able to…
07Task RoutingSend Activities to special environments, pin a pipeline to one host, and handle priorities.
08Worker SessionsRun a series of Activities on one Worker with the Session API, and handle Worker loss correctly.
09InterceptorsAdd auditing, metrics and logging around every Client call, Workflow and Activity, and test it.

Stage 4: Production ​

#ChapterYou'll be able to…
10Graceful shutdownSet WorkerStopTimeout, write Activities that checkpoint on shutdown, and wire shutdown into your platform.
11Worker VersioningShip new Workflow code with Pinned and Auto-Upgrade Workflows, ramp a new Version and drain the old one.
12Serverless WorkersKnow when Temporal can start Workers for you, and run a Worker on AWS Lambda or GCP Cloud Run.

Core concepts vs. implementation details ​

Core concepts: understand these firstImplementation details, options & sub-features: learn when needed
Workers poll; the Service only hands out Tasks (01)Worker Program / Process / Entity vocabulary, Data Converter (01)
Workflow, Activity and Nexus Tasks (02)Workflow Task failure vs Workflow Execution failure (02)
Task Queues, created on demand (03)Partitions, ordering, sync match (03)
Same Task Queue ⇒ same registrations (03)Custom names, DisableRegistrationAliasing (04)
worker.New → Register… → Run (04)Start/Stop, several Workers per process, Identity, OnFatalError (04)
Slots and pollers (05)Rate limits, poller autoscaling, eager Activities, Tuner (05)
Sticky Execution (06)Cache size, StickyScheduleToStartTimeout (06)
Task Routing with dedicated queues (07)Per-host queues, priorities and fairness (07), Sessions (08)
—Interceptors (09)
Graceful shutdown (10)Worker stop channel, heartbeat checkpoints, Local Activities on shutdown (10)
Worker Deployments and Versions (11)Ramping, inheritance rules, WorkerDeploymentClient (11)
—Serverless Workers, lambdaworker, Cloud Run (12)

Which Worker setup do I need? ​

text
Is a Workflow stuck in Running with no progress?
├── No pollers on its Task Queue?        → Task Queue name mismatch / no Worker  → chapter 03
├── "unable to find workflow type"?      → register it on every Worker of that queue → chapters 03, 04
└── Backlog grows, Workers busy?         → slots, pollers, more Workers           → chapter 05

Must some Activities run in a special place?
├── A kind of machine (GPU, PCI zone)    → a dedicated Task Queue + Worker pool   → chapter 07
└── The same host as a previous Activity
    ├── Go, simple                       → Worker Sessions                        → chapter 08
    └── Full control                     → per-host Task Queue                    → chapter 07

Must a downstream API not be flooded?
├── Per Worker                           → WorkerActivitiesPerSecond              → chapter 05
└── Across all Workers                   → TaskQueueActivitiesPerSecond           → chapter 05

Will you deploy Workers while Workflows are running?
├── Activities in flight                 → WorkerStopTimeout + heartbeats         → chapter 10
└── Workflow code changes                → Worker Versioning (or patching)        → chapter 11

Don't want to run Worker processes at all? → Serverless Workers (Public Preview)  → chapter 12

Go API cheat sheet ​

I want to…Go APIChapter
Create a Workerworker.New(c, taskQueue, worker.Options{})04
Register codew.RegisterWorkflow(fn), w.RegisterActivity(&Activities{…}), …WithOptions(fn, workflow.RegisterOptions{Name: …})04
Run until SIGINT/SIGTERMw.Run(worker.InterruptCh())04, 10
Run inside a servicew.Start() … w.Stop()04, 10
Name the Workerworker.Options{Identity: …} (default pid@hostname@)04
Size concurrencyMaxConcurrentActivityExecutionSize, MaxConcurrentWorkflowTaskExecutionSize, MaxConcurrent…TaskPollers05
Rate-limit ActivitiesWorkerActivitiesPerSecond (per Worker), TaskQueueActivitiesPerSecond (whole queue)05
Activity-only Workerworker.Options{DisableWorkflowWorker: true}05
Tune the Workflow cacheworker.SetStickyWorkflowCacheSize(n), StickyScheduleToStartTimeout06
Route an Activityworkflow.ActivityOptions{TaskQueue: …}03, 07
Prioritize a TaskPriority: temporal.Priority{PriorityKey: …, FairnessKey: …}07
Keep Activities on one WorkerEnableSessionWorker: true, workflow.CreateSession, workflow.CompleteSession08
Detect a lost Sessionworkflow.GetSessionInfo(ctx).SessionState == workflow.SessionStateFailed08
Add an interceptorclient.Options{Interceptors: …}, worker.Options{Interceptors: …}, interceptor.WorkerInterceptorBase09
Shut down gracefullyworker.Options{WorkerStopTimeout: …}, activity.GetWorkerStopChannel(ctx)10
Run a versioned WorkerDeploymentOptions{UseVersioning: true, Version: worker.WorkerDeploymentVersion{…}}, RegisterOptions{VersioningBehavior: …}11
Promote a Versiontemporal worker deployment set-current-version, c.WorkerDeploymentClient().GetHandle(…).SetCurrentVersion(…)11
Run on AWS Lambdalambdaworker.RunWorker(version, func(o *lambdaworker.Options) error {…})12

About the code samples ​

  • Each chapter is a self-contained module named github.com/acme/shop. Every Go block starts with // file: <path> so you can paste it into that path.
  • All samples were compiled, gofmt-checked and go vet-checked against Go SDK v1.50.0 (go.temporal.io/sdk), go.temporal.io/sdk/contrib/envconfig v1.0.2 and go.temporal.io/sdk/contrib/aws/lambdaworker v0.1.1. The interceptor unit test in chapter 09 passes.
  • Behaviors were checked on a dev server (Temporal CLI 1.9.1, Server 1.32.0): unknown types and Task Queue typos (03), registration panics (04, 11), the sticky → normal queue fallback (06), per-host routing and its fallback (07), Sessions including Worker loss and Continue-As-New (08), the three WorkerStopTimeout cases and heartbeat resume (10), and a Pinned vs Auto-Upgrade rollout from v1 to v2 (11).
  • External systems are interfaces or fakes. Run everything locally with temporal server start-dev (UI at http://localhost:8233).

Notes from checking the docs against the SDK ​

  • WorkerStopTimeout defaults to 0. The Go "Run a Worker" page says the Worker waits for in-flight Tasks "up to the WorkerStopTimeout", but not that the default is no wait. With the default, Stop() returns at once, the Activity context is canceled and the attempt is retried (chapter 10).
  • "Replays from the beginning". The Tasks page says every Workflow Task replays the whole history. That only happens on a cache miss; with Sticky Execution the Worker applies just the new events (chapters 02 and 06).
  • Default Worker Identity. The Encyclopedia says pid@hostname. In Go it is pid@hostname@ (trailing @), inherited from the Client (chapter 04).
  • Registration without a versioning behavior. The Go page says it fails at registration. The SDK panics only when UseVersioning and a Version are set. DefaultVersioningBehavior without UseVersioning panics earlier, in worker.New (chapter 11).
  • Session failure handling. When the Session's Worker disappears during an Activity, that Activity returns a canceled error, not ErrSessionFailed; only later Activities get ErrSessionFailed. Retrying only on ErrSessionFailed fails the Workflow (seen on a dev server). Check GetSessionInfo(…).SessionState instead. A graceful stop fails the Session immediately (chapter 08).
  • Session limit. MaxConcurrentSessionExecutionSize defaults to 1000, not "a very large value", and applies per Worker, not per process (chapter 08).
  • Sessions and Worker Versioning. The SDK says Sessions are not compatible with versioning, but only rejects the deprecated UseBuildIDForVersioning. With DeploymentOptions.UseVersioning there is no check. Test before combining them (chapter 08).
  • Server-side rate limit disables eager Activities. Setting TaskQueueActivitiesPerSecond turns off eager Activity execution (SDK doc comment). The Task Routing page doesn't say so (chapter 05).
  • No option reference. The Go "Run a Worker" page links worker.Options to the internal package and says only that an empty struct works. Chapter 05 has a defaults table read from the source.
  • Fatal errors. After startup only NamespaceNotFound, InvalidArgument and ClientVersionNotSupported stop a Worker (and call OnFatalError). A Worker with nothing registered starts without error (chapter 04).
  • lambdaworker. ShutdownDeadlineBuffer is computed from the default WorkerStopTimeout (5s) before your configure callback runs, so raising WorkerStopTimeout there leaves the buffer at 7s. Set both. The package also reads TEMPORAL_TASK_QUEUE and sets the identity to requestID@functionARN; neither is on the Go page (chapter 12).
  • CLI rollout. set-current-version and set-ramping-version ask for confirmation; use --yes in CI (chapter 11).

Sources ​

Encyclopedia: Workers · Tasks · Task Queues · Task Queue naming · Task Routing · Sticky Execution · Worker shutdown · Worker Versioning · Serverless Workers

Go SDK: Run a Worker · Worker Sessions · Interceptors · Serverless Workers · linked from it: AWS Lambda · GCP Cloud Run