Kafka Event Catalog

All asynchronous service-to-service communication in Olly flows through Kafka topics. Locally the broker is Redpanda (Kafka-compatible); in production it is Amazon MSK.

Topic Naming Convention

{domain}.{entity}.{event}

Examples: claims.claim.submitted, enrollment.cobra.elected, billing.invoice.generated

Domains: claims, eligibility, enrollment, billing, provider, edi

Topic names use dot-separated lowercase segments only — no underscores. Underscores appear only inside event type enum values in message payloads.

Consumer group naming:

{service}-{domain}-consumer

Examples: billing-claims-consumer, notifications-enrollment-consumer, eligibility-enrollment-consumer

Default Topic Configuration

Unless noted otherwise, all topics are created with these defaults:

Parameter	Default	Notes
`replication.factor`	3	All MSK clusters run 3+ brokers
`min.insync.replicas`	2	Producer `acks=all`; ensures durability on broker failure
`retention.ms`	604800000 (7 days)	Sufficient for consumer lag recovery
`compression.type`	`lz4`	Good ratio/speed balance for Avro payloads
`max.message.bytes`	1048576 (1 MB)	Increased for EDI topics with large raw payloads
`cleanup.policy`	`delete`	Log compaction not used; events are time-bounded
`partitions`	6	Increased to 12 for high-volume topics

High-volume topics (claims.claim.submitted, billing.invoice.generated) use 12 partitions and 14-day retention. EDI archive topics use 30-day retention.

Claims Domain

Topic	Partitions	Retention	Producer	Consumers
`claims.claim.submitted`	12	14 days	Claims Service	Billing Service, Notifications Service
`claims.claim.adjudicated`	12	14 days	Claims Service	Billing Service, Notifications Service, EDI (Mirth Connect)
`claims.claim.paid`	6	7 days	Billing Service (primary), Claims Service (audit mirror)	Notifications Service, OpenSearch
`claims.prior_auth.decision`	6	7 days	Claims Service	Notifications Service, Eligibility Service
`claims.appeal.resolved`	6	7 days	Claims Service	Notifications Service, Billing Service

`claims.claim.submitted`

Event types: CLAIM_SUBMITTED

json

{
  "eventType": "CLAIM_SUBMITTED",
  "schemaVersion": "1.0",
  "claimId": "uuid",
  "memberId": "uuid",
  "providerId": "uuid",
  "planId": "uuid",
  "claimType": "PROFESSIONAL",
  "totalBilledAmount": "250.00",
  "serviceDate": "2026-01-15",
  "submittedAt": "2026-01-16T10:00:00Z",
  "requiresPriorAuth": true,
  "priorAuthId": "uuid-or-null"
}

`claims.claim.adjudicated`

Event types: CLAIM_APPROVED, CLAIM_DENIED

json

{
  "eventType": "CLAIM_APPROVED",
  "schemaVersion": "1.0",
  "claimId": "uuid",
  "memberId": "uuid",
  "providerId": "uuid",
  "status": "APPROVED",
  "totalBilledAmount": "250.00",
  "allowedAmount": "180.00",
  "paidAmount": "144.00",
  "memberResponsibility": "36.00",
  "deductibleApplied": "0.00",
  "copayApplied": "36.00",
  "coinsuranceApplied": "0.00",
  "adjudicatedAt": "2026-01-17T14:30:00Z",
  "eobS3Key": "eob/2026/01/uuid.pdf"
}

json

{
  "eventType": "CLAIM_DENIED",
  "schemaVersion": "1.0",
  "claimId": "uuid",
  "memberId": "uuid",
  "providerId": "uuid",
  "denialReasonCode": "CO-4",
  "denialReasonText": "Service requires prior authorization",
  "appealDeadline": "2026-02-17",
  "adjudicatedAt": "2026-01-17T14:30:00Z"
}

Eligibility Domain

Topic	Partitions	Retention	Producer	Consumers
`eligibility.coverage.verified`	6	7 days	Eligibility Service	Claims Service
`eligibility.coverage.terminated`	6	7 days	Eligibility Service	Claims Service, Notifications Service

`eligibility.coverage.verified`

Event types: COVERAGE_VERIFIED, COVERAGE_NOT_FOUND

json

{
  "eventType": "COVERAGE_VERIFIED",
  "schemaVersion": "1.0",
  "memberId": "uuid",
  "planId": "uuid",
  "groupNumber": "GRP-00123",
  "memberNumber": "MEM-456789",
  "effectiveDate": "2026-01-01",
  "terminationDate": null,
  "coverageType": "MEDICAL",
  "networkTier": "IN_NETWORK",
  "deductibleRemaining": "1500.00",
  "oopMaxRemaining": "4000.00",
  "verifiedAt": "2026-01-16T10:01:00Z"
}

Enrollment Domain

Topic	Partitions	Retention	Producer	Consumers
`enrollment.enrollment.submitted`	6	7 days	Enrollment Service	Eligibility Service, Billing Service, Notifications Service
`enrollment.enrollment.activated`	6	7 days	Enrollment Service	Eligibility Service, Billing Service, Notifications Service
`enrollment.enrollment.terminated`	6	7 days	Enrollment Service	Eligibility Service, Billing Service, Notifications Service
`enrollment.enrollment.plan_changed`	6	7 days	Enrollment Service	Eligibility Service, Billing Service, Notifications Service
`enrollment.cobra.notice_sent`	6	7 days	Enrollment Service	Notifications Service
`enrollment.cobra.elected`	6	7 days	Enrollment Service	Eligibility Service, Billing Service, Notifications Service

`enrollment.enrollment.submitted`

Event types: ENROLLMENT_SUBMITTED

json

{
  "eventType": "ENROLLMENT_SUBMITTED",
  "schemaVersion": "1.0",
  "enrollmentId": "uuid",
  "memberId": "uuid",
  "groupId": "uuid-or-null",
  "planId": "uuid",
  "coverageType": "MEDICAL",
  "electionType": "NEW",
  "effectiveDate": "2026-01-01",
  "tierCode": "EE+FAM",
  "memberPremium": "480.00",
  "totalPremium": "1200.00",
  "dependentCount": 2,
  "submittedAt": "2025-11-15T10:00:00Z"
}

Billing Domain

Topic	Partitions	Retention	Producer	Consumers
`billing.invoice.generated`	12	14 days	Billing Service	Notifications Service
`billing.payment.received`	6	7 days	Billing Service	Enrollment Service, Notifications Service
`billing.payment.missed`	6	7 days	Billing Service	Enrollment Service, Notifications Service
`billing.payment.completed`	6	7 days	Billing Service	Claims Service, Notifications Service

`billing.payment.missed`

Event types: PAYMENT_MISSED, GRACE_PERIOD_STARTED, GRACE_PERIOD_EXPIRED

json

{
  "eventType": "GRACE_PERIOD_STARTED",
  "schemaVersion": "1.0",
  "memberId": "uuid",
  "invoiceId": "uuid",
  "gracePeriodEndDate": "2026-03-30",
  "premiumCents": 48000,
  "missedAt": "2026-02-28T23:59:59Z"
}

Provider Domain

Topic	Partitions	Retention	Producer	Consumers
`provider.credentialing.status_changed`	6	7 days	Provider Service	Claims Service, Eligibility Service, Notifications Service
`provider.network.updated`	6	7 days	Provider Service	Claims Service, Eligibility Service

`provider.credentialing.status_changed`

Event types: CREDENTIALING_APPROVED, CREDENTIALING_DENIED, CREDENTIALING_EXPIRED, CREDENTIALING_SUSPENDED

json

{
  "eventType": "CREDENTIALING_APPROVED",
  "schemaVersion": "1.0",
  "providerId": "uuid",
  "npi": "1234567890",
  "providerName": "Jane Smith MD",
  "specialty": "Orthopedic Surgery",
  "credentialingStatus": "APPROVED",
  "networkStatus": "IN_NETWORK",
  "effectiveDate": "2026-02-01",
  "approvedAt": "2026-01-28T10:00:00Z"
}

EDI Domain

Topic	Partitions	Retention	Producer	Consumers
`edi.inbound.received`	6	30 days	EDI (Mirth Connect)	Claims Service, Enrollment Service, Eligibility Service
`edi.outbound.generated`	6	30 days	EDI (Mirth Connect)	OpenSearch (audit index)

Event types for inbound: EDI_837_RECEIVED, EDI_834_RECEIVED, EDI_270_RECEIVED Event types for outbound: EDI_835_GENERATED, EDI_834_GENERATED, EDI_271_GENERATED

Schema Registry

All Kafka messages use Avro encoding registered with Confluent Schema Registry (deployed on MSK in production).

Parameter	Value
Registry deployment	Confluent Schema Registry, 2 replicas on EKS
Storage	Dedicated `_schemas` Kafka topic
Compatibility mode	`BACKWARD` — consumers on schema version N can read messages written with schema N+1
Subject naming	`{topic-name}-value` (TopicNameStrategy)

Go services use github.com/riferrei/srclient for schema registration and Avro encoding. Schema IDs are embedded in the 5-byte magic prefix of each Kafka message (Confluent wire format). Adding optional fields with defaults is permitted; removing fields or changing types requires a major version bump and coordinated consumer upgrade.

Dead Letter Queue Pattern

Every Kafka consumer implements a DLQ pattern for failed message processing.

Normal:   Topic → Consumer → Process → Commit offset

Failure (after 3 retries with exponential backoff: 1s, 2s, 4s):
  Topic → Consumer → FAILS
    │
    └─► DLQ topic: {original-topic}.dlq
             │
             └─► DLQ consumer: alert to Grafana OnCall + log to OpenSearch

DLQ configuration:

Naming: {original-topic}.dlq (e.g. claims.claim.submitted.dlq)
Retention: 30 days (longer than source topic to allow investigation)
Partitions: same as source topic
No DLQ of the DLQ — failures in the DLQ consumer are logged to OpenSearch and alert via Grafana OnCall

DLQ messages wrap the original payload with failure metadata:

json

{
  "dlqMetadata": {
    "originalTopic": "claims.claim.submitted",
    "originalPartition": 3,
    "originalOffset": 10492,
    "failureReason": "EligibilityService unavailable after 3 retries",
    "failedAt": "2026-01-16T10:05:32Z",
    "consumerGroup": "billing-claims-consumer",
    "retryCount": 3
  },
  "originalPayload": { }
}

Kafka consumer idempotency is enforced via a processed_event_ids set in ElastiCache (Valkey) with a 24-hour TTL. Duplicate deliveries from Kafka are discarded before processing.

Kafka Event Catalog ​

Topic Naming Convention ​

Default Topic Configuration ​

Claims Domain ​

claims.claim.submitted ​

claims.claim.adjudicated ​

Eligibility Domain ​

eligibility.coverage.verified ​

Enrollment Domain ​

enrollment.enrollment.submitted ​

Billing Domain ​

billing.payment.missed ​

Provider Domain ​

provider.credentialing.status_changed ​

EDI Domain ​

Schema Registry ​

Dead Letter Queue Pattern ​

Kafka Event Catalog

Topic Naming Convention

Default Topic Configuration

Claims Domain

`claims.claim.submitted`

`claims.claim.adjudicated`

Eligibility Domain

`eligibility.coverage.verified`

Enrollment Domain

`enrollment.enrollment.submitted`

Billing Domain

`billing.payment.missed`

Provider Domain

`provider.credentialing.status_changed`

EDI Domain

Schema Registry

Dead Letter Queue Pattern