FastAPI · Docker · Kubernetes
Cloud-Native AI Platform: implementation notes
Cloud-Native AI Platform is a learning scaffold for an AI API. It contains a FastAPI application, Prometheus metrics, a Dockerfile and a Kubernetes deployment manifest. Model inference is mocked; cloud provisioning, autoscaling and CI/CD are unfinished.
What is implemented?
The source demonstrates the shape of a small service that could later host model inference. It separates HTTP routes, basic health responses, request metrics and deployment configuration. That is useful as a learning starting point, but it does not yet establish a working model-serving platform.
| Area | Present in source | Remaining work |
|---|---|---|
| API | FastAPI health, metadata and inference routes | Real provider integration and error handling |
| Metrics | Prometheus counters and histograms | Validate behavior under realistic load and failures |
| Container | Multi-stage Dockerfile, non-root user, health check | Reproducible build and execution evidence |
| Kubernetes | Two-replica deployment manifest with probes and resource bounds | Resolve image, configuration and service-account dependencies; verify rollout |
| Terraform, HPA and CI/CD | Placeholder files | Implementation and testing |
How the API behaves today
POST /api/v1/infer accepts an inference-shaped request but produces a mocked text response. It does not call OpenAI, Anthropic or a self-hosted model. The reported token value is a word-count proxy rather than model token accounting.
GET /api/v1/models is a catalogue of intended model/provider metadata. It does not prove provider connectivity. The health, readiness and liveness routes return application responses; readiness does not check external dependencies.
GET /metrics exposes request counters, request-duration histograms and inference metrics. These are instrumentation primitives. A route existing is different from demonstrated availability, latency or capacity under load.
The current implementation is a mocked API scaffold. No production readiness, uptime or latency result is claimed by this review.
Container and Kubernetes configuration
The Dockerfile uses Python 3.11 and a separate build stage for dependencies. Its runtime switches to a non-root application user and starts Uvicorn on port 8080. A container health check requests the application's health route.
The Kubernetes deployment specifies two replicas, rolling updates, resource requests and limits, liveness/readiness probes, a read-only root filesystem and dropped Linux capabilities. It still contains an AI_PLATFORM_IMAGE placeholder and references configuration and a service account that must be supplied.
The reviewed terraform/main.tf, k8s/hpa.yaml and .github/workflows/ci-cd.yml files are empty placeholders. The presence of those filenames does not implement AWS infrastructure, autoscaling or a delivery pipeline.
What would make the project demonstrable?
- Add repeatable route tests, build the image and record a successful local start.
- Implement a model provider with timeouts, error handling and real token accounting.
- Complete the Kubernetes dependencies and verify a deployment in a test environment.
- Implement infrastructure and autoscaling configuration with a reviewable plan before provisioning resources.
- Add CI checks and publish measured behavior for success, dependency failures and load.
These are next steps, not completed capabilities. This documentation update does not provision cloud infrastructure.
Questions about the scaffold
Can it serve real model inference now?
The reviewed inference handler returns mocked output. Provider integration remains to be implemented and tested.
Does the repository include working Terraform and CI/CD?
The reviewed Terraform entry point and workflow file are empty. They are placeholders for planned work.
What can I inspect now?
The API routes, metrics, Dockerfile and Kubernetes deployment manifest provide concrete source to read. See the linked files below and check the repository for later changes.
Sources and review scope
Implementation statements were checked against source from commit b55070c27d22a65ff6137d7a432321206d099a25. The README was subsequently corrected to match that scope. No container build, application test suite or Kubernetes rollout was run for this website review.
- FastAPI source at the reviewed commit.
- Dockerfile and deployment manifest.
- Current README — status, limitations and next steps.