Senior Software Engineer
Infrastructure × Backend × LLM Systems
Building observable, scalable, fault-tolerant systems for mass audience.
Explore Experience
Feel free to Ask Poimen more abt it
riotpiao.com
DevOps / SRE / SDE
Wanted to understand how LLM serving works at scale—so built an entire cloud platform from scratch on bare-metal K8s. 4 machines (1 GPU node, 1 Dell PowerEdge R520, 2 mini-desktops), 3 control planes for distributed consensus. Learned etcd needs low-latency the hard way—powerline adapters spiked 200ms with 30+ pods, killing consensus. Ran ethernet to the garage. Purchased riotpiao.com from Cloudflare with wildcard DNS. Evolved GitOps from pure Terraform to Terraform (Talos config, barely changes) + ArgoCD (CRD-driven observer pattern for K8s resources). Unified IAM with Authentik OIDC, deployed paperless.riotpiao.com for document workflow integration. Now building Poimen—an AI agent system where the right context + small model = successful task inference.
RBC
Lead Software Engineer
Learned Terraform and Temporal here—tools fundamental for hosting infra at ease. Tackled two big problems: (1) Flaky deployments—500+ resource state files timing out, 503 errors, lock contention. Fixed by migrating to JFrog Artifactory, splitting plan/apply phases with immutable artifacts, throttling parallelism. Zero state corruption after. (2) Configuration drift—engineers hotfixing in cloud console, code detached from reality. Built nightly cron drift detection with Slack alerts, cut drift visibility from 3 weeks to <24hrs. Key takeaway: Terraform shines when things barely change; for K8s resources that churn, you need ArgoCD.
AWS
Senior Software Engineer (Step Functions)
First job out of grad school—learned STAR method and customer obsession at scale. Owned Distributed-Map end-to-end across 57+ regions, sub-100ms P99. Caught a potentially customer-impacting condition field change during code review and coordinated simultaneous frontend+backend deployment—because deployment alignment keeps the business running. Built Redrive Execution for customers to retry from failure point, plus ops tools. Learned backward-compat is critical when frontend consumes latest images. Owned oncall, built CloudWatch dashboards, wrote runbooks.
Titus
Software Engineer Intern
Streamlined Personal Data Detection to detect anomaly exit of classified data—achieved 97.8% accuracy. Built fault-tolerant Golang connector—28% p99 improvement over legacy. Re-integrated SmartRegex with CMake & C++ on Linux/Unix—5x faster deployment.
NAV Canada
Summer Student
Maintained enterprise web app CFPS in Agile development process. Built Django NOTAMJ polls app to meet iteration goals. Improved deploy stability with Sonar code coverage. Created FWGS weather briefing interface with ReactJS for ATC.
What have i Built and Building
Demonstrate my work
Homelab: Self-Hosted Cloud
Production-grade cloud platform rebuilt from scratch on bare-metal Kubernetes.
4-node Talos cluster (3 control plane + 1 worker) with OIDC SSO, GitOps (ArgoCD), CI/CD, Kafka, PostgreSQL, S3 storage, GPU LLM inference, and Temporal workflows.
- •Kubernetes (Talos Linux) + ArgoCD GitOps + Terraform IaC + Kustomize manifests
- •Authentik OIDC SSO + RBAC + SOPS encrypted secrets + cert-manager TLS
- •Longhorn block storage + MinIO S3 + CloudNativePG PostgreSQL + pgvector
- •Prometheus + Grafana + Loki + Tempo + OpenTelemetry observability stack
- •vLLM GPU inference (Qwen3-32B) + Ollama + TEI embeddings + KServe orchestration
- •Temporal workflows + Kafka/Redpanda streaming + Forgejo CI/CD + DinD runners
- •Go API gateway + Python ML + Next.js frontend + Cloudflare Tunnel zero-trust
📊 4 nodes, 20+ services, 99.2% uptime
📊 AWS rebuilt at home—full stack from compute to observability.
Poimen Memory System
Distributed Graph-RAG infrastructure with hierarchical RBAC and wiki-link indexing.
Three-tier context retrieval pipeline with PageRank-style link scoring, hybrid search fusion (HNSW + BM25), and OIDC-based access control for multi-tenant knowledge graphs.
- •Graph-RAG with Wiki-Link Indexing (Rust, pgvector, OpenSearch): Built bidirectional link graph from [[wiki-link]] syntax during ingestion. PageRank-style score propagation boosts linked documents' relevance. RRF fusion merges HNSW cosine (pgvector) + BM25 lexical (OpenSearch). WikiScopedFilter constrains traversal to project boundaries.
- •Three-Tier Context Retrieval (Actix-web, tokio): Async pipeline — Tier 1: MD5 signature match (<50ms), Tier 2: graph-boosted hybrid search with link-distance decay, Tier 3: Obsidian API fallback. Budget-aware assembly drops lower tiers first. Shingle-based Jaccard deduplication (>0.5) prevents redundant chunks.
- •Hierarchical RBAC (Authentik OIDC, JWT, Kubernetes): Role → AccessRule[] → AccessScope model with project/visibility/owner/group constraints. JWT roles claim maps to YAML rules; AccessGuard.filter_resources() applies post-retrieval filtering. Dual-write indexer (eventual consistency via queue) maintains RBAC-aware views. SOPS/age encryption, ArgoCD deployment.
📊 Graph-RAG, pgvector, OpenSearch, Rust + Actix-web
📊 Bidirectional wiki-link indexing with RRF fusion + hierarchical RBAC — 50ms signature match tier, graph-boosted hybrid search tier, Obsidian fallback.
Poimen: Agent Workflow Orchestration
Temporal-powered orchestration that transforms natural language into durable, scalable workflow executions.
LLM router analyzes user intent, retrieves relevant knowledge from semantic memory, and generates executable workflow specs—enabling agent deployment at scale where any activity can be wired as a step in the reconciliation pipeline.
- •LLM-Powered Workflow Routing: Natural language → executable WorkflowSpec via reasoning model (api.riotpiao.com). Activity Knowledge Base (9 activities) informs the LLM about timeouts, retry policies, and dependencies—intelligent step ordering and error handling strategies.
- •Memory-Augmented Context Retrieval: RetrieveMemoryActivity queries poimen-memory (Rust semantic search service) for relevant skills and lessons before routing—injecting domain knowledge into prompts for context-aware workflow generation.
- •Generic State Machine Executor: RoutingWorkflow executes any JSON workflow spec with JSONPath parameter chaining (${Step1.output.path}), automatic retries for flaky activities, catch blocks for error recovery, and Temporal's durable execution guarantees—every registered activity a composable building block.
📊 Temporal, LLM Routing, 9 Composable Activities
📊 Natural language → executable WorkflowSpec via reasoning model + memory-augmented context retrieval + durable state machine execution.
RBC: Multi-Cloud Platform
Unified infrastructure platform consolidating public cloud and on-prem.
Terraform automation with Temporal orchestration. 99.2% automated provisioning, notification-driven operator fallback.
- •Integrated Terraform Cloud to centralize IaC workflows—reduced onboarding time for 12 teams
- •Built K8s CronJob to detect and reconcile Terraform state drift automatically (Golang)
- •Designed Slack notification service with Golang workers—operators resolve apply failures in <5min
- •Led requirement gathering across 4 platform teams—disambiguated specs, unblocked 3 stalled projects
- •Translated technical decisions for non-technical stakeholders—secured buy-in for platform migration
📊 200+ microservices, 4 regions, 2hr→20min deploy
📊 Standardized IaC patterns across 12 teams—cut provisioning toil, 3x integration velocity.
AWS Distributed-Map
Launched distributed task orchestration service for mission-critical workloads.
Optimized execution across 57+ regions with fault-tolerant scheduling and auto-scaling.
- •Under tight deadline, introduced separate state in Map vs new functions—simplified customer adoption (Java)
- •Built checkpoint recovery for mid-workflow failures—customers resume without full re-run
- •Solved distributed edge cases: race conditions, concurrent updates, dependent service failures, message deduplication
- •Owned oncall for the service—built CloudWatch dashboards, wrote runbooks, debugged production live
📊 57+ regions, sub-100ms P99, 20x burst traffic
📊 Introduced JSON state input for larger payloads—unlocked new customer use cases.