From Oracle Cloud to BYOC: Provisioning an OKE Kubernetes Cluster and Connecting It to Nife
A practical guide to provisioning an Oracle Kubernetes Engine cluster using the oci-free-cloud-k8s repository, configuring Kubernetes access with kubeconfig, connecting the cluster to Nife as a BYOC environment, and deploying applications on customer-owned Kubernetes infrastructure.
From Oracle Cloud to BYOC: Provisioning an OKE Kubernetes Cluster and Connecting It to Nife
Kubernetes does not necessarily have to be provisioned and managed by the same platform that handles application deployments.
A Kubernetes cluster can exist inside a customer's own cloud account, while a deployment platform connects to that cluster and manages application workloads through the Kubernetes API.
That is the idea behind Bring Your Own Cloud (BYOC).
While exploring different ways to provision Kubernetes clusters on Oracle Cloud, I came across the nce/oci-free-cloud-k8s repository. The repository provides Terraform-based infrastructure for creating a Kubernetes cluster on Oracle Cloud's Always Free resources and configures a number of Kubernetes components on top of it.
The interesting part from a platform perspective is what happens after the cluster is created.
Once the OKE cluster is available and a working kubeconfig is generated, the cluster can be treated as an external Kubernetes environment and connected to Nife as a BYOC cluster.
The complete flow looks like this:
Oracle Cloud
│
│ Terraform
▼
OKE Cluster
│
│ kubeconfig
▼
Nife
│
│ BYOC
▼
Application Deployment
This article walks through that complete flow.
1. What Is oci-free-cloud-k8s?
The oci-free-cloud-k8s repository is an example of provisioning a Kubernetes environment on Oracle Cloud using Terraform.
The repository uses Oracle Kubernetes Engine (OKE) and provisions the infrastructure required to get a usable Kubernetes API endpoint.
The repository also configures additional Kubernetes components, including:
- Kubernetes control plane
- Worker nodes
- Envoy Gateway
- Gateway API
- cert-manager
- External DNS
- FluxCD
- Teleport
- Longhorn
- Grafana
- Loki
- Metrics Server
- Kyverno
- Image signing
- S3 proxy
The repository separates infrastructure provisioning from Kubernetes configuration and application configuration.
The basic structure is:
oci-free-cloud-k8s
│
├── terraform
│ ├── infra
│ └── config
│
├── gitops
│ └── core
│
├── docs
│
└── README.md
The important distinction is:
terraform/infra
│
▼
OCI infrastructure + OKE
│
▼
Usable Kubernetes API
│
▼
terraform/config
│
▼
OCI/Kubernetes-specific configuration
│
▼
FluxCD
│
▼
Kubernetes applications
The repository itself notes that the infrastructure in infra should be created first, while the Kubernetes modules are managed through config and the Kubernetes applications/configuration are handled through FluxCD.
2. Why This Is Interesting for BYOC
A typical managed deployment platform might control both the infrastructure and the application deployment layer.
For example:
Platform
│
├── Cloud Infrastructure
├── Kubernetes
├── Worker Nodes
└── Applications
With BYOC, these responsibilities can be separated.
The customer can own:
Customer Cloud Account
│
├── VPC / VCN
├── Kubernetes Cluster
├── Worker Nodes
├── Storage
└── Networking
while Nife operates at the application deployment layer:
Nife
│
├── Application
├── Build
├── Deployment
└── Kubernetes Workloads
The connection between these two layers is the Kubernetes API.
And the kubeconfig provides the information required to connect to that API.
So the architecture becomes:
┌─────────────────────────────────────┐
│ Customer OCI Account │
│ │
│ OKE │
│ │ │
│ Kubernetes API │
│ │ │
└───────────────┼─────────────────────┘
│
kubeconfig
│
▼
┌───────────────┐
│ Nife │
│ BYOC │
└───────┬───────┘
│
▼
Application Deployment
3. Prerequisites
Before provisioning the cluster, the repository requires a few tools on the client machine.
The main requirements are:
- Terraform
- OCI CLI
- OCI CLI configuration
The repository currently requires Terraform >= 1.12 because its backend configuration uses Oracle's native backend.
The OCI CLI can be configured with:
oci setup config
After configuration, verify that the OCI CLI can communicate with the account.
For example:
oci iam region list
The Terraform configuration also uses Oracle Object Storage for Terraform state. The repository provides an example of creating a bucket for this purpose:
oci os bucket create \
--name terraform-states \
--versioning Enabled \
--compartment-id <COMPARTMENT_OCID>
The exact compartment, region and authentication configuration depend on the OCI account being used.
4. Understanding the Terraform Layout
Before running Terraform, it is useful to understand what each part of the repository does.
The repository divides the Terraform configuration into two main sections.
terraform/infra
This is responsible for creating the infrastructure required to get a usable Kubernetes API endpoint.
Conceptually:
terraform/infra
│
├── OCI networking
├── OKE cluster
├── Worker nodes
└── Kubernetes API
terraform/config
This handles additional OCI-specific and Kubernetes-related configuration.
For example:
terraform/config
│
├── OCI-specific configuration
├── Secrets
└── Kubernetes modules
The repository notes that the config section requires additional variables, some of which are produced by the infrastructure deployment.
The order therefore matters:
infra
│
▼
OKE cluster
│
▼
config
│
▼
FluxCD / Kubernetes configuration
5. Provisioning the Cluster
The first step is to provision the infrastructure.
Move into the Terraform infrastructure directory:
cd terraform/infra
Initialize Terraform:
terraform init
Review the planned resources:
terraform plan
If everything looks correct, provision the infrastructure:
terraform apply
Terraform will create the OCI infrastructure and OKE resources defined by the repository.
The goal of this first stage is to reach:
Terraform
│
▼
OCI Infrastructure
│
▼
OKE Cluster
│
▼
Usable Kubernetes API
The repository specifically describes the infra Terraform configuration as managing everything required to reach a usable Kubernetes API endpoint.
6. Getting the Cluster ID
After Terraform has completed, the cluster ID can be retrieved from the Terraform output.
terraform output --raw k8s_cluster_id
This output is useful for interacting with the cluster through the OCI CLI.
For example:
oci ce cluster get \
--cluster-id $(terraform output --raw k8s_cluster_id)
This returns information about the OKE cluster.
7. Getting the Kubernetes kubeconfig
This is the most important step for the BYOC integration.
After Terraform finishes running in the infra directory, the repository generates a kubeconfig file:
.kube.config
The repository specifically documents this file as the configuration that can be used to access the cluster.
Set it as the active kubeconfig:
export KUBECONFIG=$(pwd)/.kube.config
Now Kubernetes commands can use this configuration.
For example:
kubectl get nodes
You can also explicitly specify the file:
kubectl --kubeconfig .kube.config get nodes
8. Validate the Kubernetes Cluster
Before connecting the cluster to Nife, it is important to verify that the kubeconfig works correctly.
Start with:
kubectl --kubeconfig .kube.config cluster-info
Then check the nodes:
kubectl --kubeconfig .kube.config get nodes
The expected state is:
NAME STATUS ROLES AGE VERSION
worker-node-1 Ready <none> ... ...
worker-node-2 Ready <none> ... ...
The exact node names and Kubernetes versions will depend on the environment.
You can also check the namespaces:
kubectl --kubeconfig .kube.config get namespaces
At this point the important requirements are:
✓ OKE cluster exists
✓ Kubernetes API is reachable
✓ kubeconfig works
✓ Worker nodes are Ready
✓ kubectl can communicate with the cluster
Only after these checks pass does it make sense to use the cluster as a BYOC target.
9. Checking Available Kubernetes Versions
The repository also documents how to check which Kubernetes versions are available for an upgrade.
This should be done from:
terraform/infra/
Run:
oci ce cluster get \
--cluster-id $(terraform output --raw k8s_cluster_id) \
| jq -r '.data."available-kubernetes-upgrades"'
This returns the Kubernetes versions that are available for the cluster.
For example:
v1.31.1
The important point is to use a version that OCI reports as an available upgrade rather than arbitrarily changing the version to a newer release.
Oracle's documentation similarly recommends checking available-kubernetes-upgrades and selecting one of the versions returned by that command.
10. Updating the Kubernetes Version
If a supported upgrade is available, the Kubernetes version can be updated in the Terraform configuration.
For example, the repository documents:
sed -i '' \
's/default = "'$(terraform output --raw kubernetes_version)'"/default = "v1.31.1"/' \
_variables.tf
The above syntax is for macOS.
On Linux, the equivalent command is:
sed -i \
's/default = "'$(terraform output --raw kubernetes_version)'"/default = "v1.31.1"/' \
_variables.tf
Before applying the change, verify the configured version:
grep kubernetes_version _variables.tf
Then run:
terraform plan
If the plan looks correct:
terraform apply
The repository documents this process as updating the control plane, node pool and images, and notes that the operation may take around 10 minutes.
After the upgrade:
kubectl --kubeconfig .kube.config get nodes
and:
kubectl --kubeconfig .kube.config version
This gives a simple upgrade flow:
Check available versions
│
▼
Select supported version
│
▼
Update Terraform variable
│
▼
terraform plan
│
▼
terraform apply
│
▼
Verify Kubernetes
11. Understanding the Kubernetes Configuration Layer
Once the infrastructure is working, the repository has another Terraform layer under:
terraform/config
The repository states that this layer manages Kubernetes modules and OCI-specific configuration such as secrets.
This layer depends on the kubeconfig generated during the infrastructure deployment.
The relationship is:
terraform/infra
│
│ creates
▼
.kube.config
│
│ used by
▼
terraform/config
│
▼
Kubernetes configuration
The repository also notes that the initial config Terraform run may fail while trying to create a ClusterSecretStore before External Secrets has been deployed by Flux. In that case, the documented approach is to rerun Terraform after External Secrets has been deployed.
This is an important detail because not every Terraform failure during the initial configuration necessarily means the infrastructure is broken.
12. FluxCD and the Application Layer
Another major part of the repository is FluxCD.
The repository uses FluxCD to deploy most of the Kubernetes resources and core components.
The architecture therefore becomes:
Terraform
│
├── OCI infrastructure
├── OKE
└── Kubernetes configuration
│
▼
FluxCD
│
▼
Kubernetes components
│
├── Gateway
├── cert-manager
├── External DNS
├── Storage
├── Monitoring
└── Other services
FluxCD provides the GitOps layer for the cluster.
Nife can then operate at a different layer by providing application deployment workflows against the Kubernetes cluster.
13. Connecting the Cluster to Nife as BYOC
Now we reach the main objective.
The OKE cluster has been provisioned independently inside Oracle Cloud.
The cluster has:
Kubernetes API
│
▼
.kube.config
That kubeconfig can be used as the connection information for a BYOC cluster in Nife.
The conceptual flow is:
Oracle Cloud
│
▼
OKE Cluster
│
▼
Kubernetes API
│
│ kubeconfig
▼
Nife BYOC
When adding the cluster to Nife, the kubeconfig provides the Kubernetes client configuration required to communicate with the cluster.
A kubeconfig generally contains information such as:
Cluster endpoint
Authentication information
Certificate information
Context
Cluster/user configuration
The exact authentication details depend on how the OKE kubeconfig is generated and configured.
The important point is that Nife does not need to provision the underlying OKE cluster in order to use it as a deployment target.
The cluster already exists.
Nife simply needs authenticated Kubernetes API access.
14. BYOC Architecture
Once connected, the architecture looks like this:
Oracle Cloud
│
┌──────────────┴──────────────┐
│ │
│ OKE Cluster │
│ │
│ ┌─────────────────┐ │
│ │ Kubernetes API │ │
│ └────────┬────────┘ │
│ │ │
│ Worker Nodes │
│ │ │
│ Application Pods │
│ │
└──────────────┬──────────────┘
│
kubeconfig
│
▼
┌─────────────┐
│ Nife │
│ BYOC │
└──────┬──────┘
│
▼
Application Deployment
The customer continues to own the infrastructure.
Nife becomes the deployment layer operating against Kubernetes.
15. Deploying an Application Through Nife
Once the Kubernetes cluster has been connected as a BYOC environment, an application can be deployed to it.
The high-level deployment flow is:
Developer
│
│ Git push
▼
Nife
│
│ Build
▼
Container Image
│
│ Deploy
▼
Kubernetes API
│
▼
OKE Cluster
│
▼
Worker Node
│
▼
Application Pod
The important part is that Nife communicates with Kubernetes rather than directly managing the underlying Oracle Compute instances.
Conceptually:
Nife
│
│ Kubernetes API
▼
OKE
│
├── Deployment
├── Service
├── Pods
└── Other workload resources
The exact Kubernetes resources created depend on the application's deployment configuration.
16. What Happens During Deployment?
Once a BYOC cluster is connected, a deployment can be viewed as several stages.
1. Source Code
│
▼
2. Build Application
│
▼
3. Build Container Image
│
▼
4. Push Container Image
│
▼
5. Connect to Kubernetes API
│
▼
6. Create / Update Kubernetes Resources
│
▼
7. Kubernetes Schedules Pod
│
▼
8. Application Starts
The important boundary is:
Nife → Kubernetes API → OKE
The underlying worker nodes remain part of the customer's Oracle Cloud environment.
17. ARM64 Is an Important Consideration
One of the most important details in this repository is the CPU architecture.
The repository explicitly states that it uses ARM instances rather than x86 because of the limitations of Oracle Cloud's Always Free resources.
That means applications deployed to these worker nodes need to support:
linux/arm64
This matters particularly for containerized applications.
For example, an image built only for:
linux/amd64
may not run natively on an ARM64 worker.
A multi-platform image can instead support:
linux/amd64
linux/arm64
For example:
docker buildx build \
--platform linux/amd64,linux/arm64 \
-t my-app:latest \
--push .
The resulting image can then be pulled by both x86_64 and ARM64 Kubernetes nodes.
This becomes an important validation step when using an ARM64-based BYOC cluster.
18. Networking and Ingress
The repository also provides a Kubernetes networking layer rather than leaving the cluster completely bare.
It uses Gateway API with Envoy Gateway and Oracle load balancers for ingress.
Conceptually:
Internet
│
▼
Oracle Load Balancer
│
▼
Envoy Gateway
│
▼
Gateway API / HTTPRoute
│
▼
Kubernetes Service
│
▼
Application Pod
This is useful for a BYOC deployment environment because the Kubernetes cluster can already have its networking and ingress components configured.
The deployment platform can focus on application workloads rather than having to recreate the entire cluster networking stack.
19. Storage
The repository uses Longhorn for Kubernetes storage.
The high-level architecture is:
Application
│
▼
PersistentVolumeClaim
│
▼
Longhorn
│
▼
Worker Node Storage
This means stateful applications can use Kubernetes persistent volumes rather than relying only on ephemeral container storage.
For a BYOC platform, this distinction is important because the storage layer remains part of the customer's Kubernetes environment.
20. FluxCD vs Application Deployment
The repository already uses FluxCD for GitOps and cluster configuration.
This raises an important architectural question:
Where should FluxCD stop and the application deployment platform begin?
One possible separation is:
FluxCD
│
├── Cluster infrastructure
├── Gateway
├── cert-manager
├── External DNS
├── Storage
└── Platform-level services
Nife
│
├── Applications
├── Application deployments
├── Application versions
└── Developer deployment workflow
The exact division should be defined carefully so that FluxCD and Nife do not attempt to manage the same Kubernetes resources.
Otherwise, two controllers could continuously reconcile the same resource to different desired states.
21. The Complete Provisioning-to-BYOC Flow
Putting everything together, the complete workflow looks like this:
Oracle Cloud
│
▼
OCI Configuration
│
▼
Terraform
│
▼
terraform/infra
│
▼
OKE Cluster
│
▼
.kube.config
│
▼
Validate Cluster
│
▼
Check Kubernetes Version
│
▼
Optional Version Upgrade
│
▼
terraform/config
│
▼
FluxCD
│
▼
Kubernetes Components
│
│
kubeconfig
│
▼
Nife BYOC
│
▼
Application Build
│
▼
Container Registry
│
▼
Kubernetes API
│
▼
OKE Worker
│
▼
Application Pod
The workflow can be summarized in three major stages:
PROVISION
│
▼
OCI + OKE + Kubernetes
CONNECT
│
▼
kubeconfig → Nife BYOC
DEPLOY
│
▼
Application → Kubernetes → OKE
22. What Makes This Approach Useful?
The main advantage of this model is the separation of responsibilities.
The customer can maintain control over:
Cloud Account
Infrastructure
Network
Kubernetes Cluster
Worker Nodes
Storage
Ingress
while the deployment platform can provide:
Source Integration
Builds
Container Deployment
Application Management
Deployment Workflow
This gives a deployment platform a way to work with Kubernetes clusters that were provisioned independently.
It also means that the same deployment model can potentially be applied to Kubernetes clusters running in different environments, as long as the required Kubernetes API access and platform requirements are satisfied.
23. Important Things to Check Before Using the Cluster as BYOC
Before connecting the cluster to Nife, I would validate the following:
Kubernetes connectivity
kubectl --kubeconfig .kube.config get nodes
All expected nodes should be Ready.
Kubernetes API access
kubectl --kubeconfig .kube.config cluster-info
The API endpoint should be reachable.
Kubernetes version
Check the current version:
kubectl --kubeconfig .kube.config version
Node architecture
Check the architecture:
kubectl --kubeconfig .kube.config get nodes \
-o custom-columns=NAME:.metadata.name,ARCH:.status.nodeInfo.architecture
For this repository, the expected worker architecture is:
arm64
Application image architecture
Before deploying an application, verify that its image supports ARM64.
For example:
docker buildx imagetools inspect <image>
Look for:
linux/arm64
24. Security Considerations
A kubeconfig should be treated as sensitive access information.
It should not be:
Committed to Git
Stored in a public repository
Shared through unsecured channels
Included in application source code
The .kube.config generated by the repository should therefore be protected appropriately.
When using a kubeconfig to connect an external platform to Kubernetes, the Kubernetes permissions assigned to that connection should also be considered carefully.
The general principle is:
Only give the deployment platform
the Kubernetes permissions it actually needs.
For production environments, access should be designed around appropriate authentication, RBAC, credential rotation and secret-management practices.
25. The Key Takeaway
The interesting part of this setup is not simply that Terraform can create an OKE cluster.
The more important concept is what happens after the cluster exists.
The cluster can remain completely independent:
Customer
│
▼
Oracle Cloud
│
▼
OKE
Then the Kubernetes API becomes the integration boundary:
OKE
│
│ kubeconfig
▼
Nife
And application deployment happens on top:
Nife
│
▼
Kubernetes API
│
▼
OKE
│
▼
Application
This is the core idea behind using an existing Kubernetes cluster as a BYOC environment.
Conclusion
The oci-free-cloud-k8s repository provides a practical example of provisioning an OKE Kubernetes environment using Terraform and then layering additional Kubernetes tooling on top of it.
The infrastructure is separated into an infra layer that creates the usable Kubernetes API endpoint, a config layer for additional Kubernetes/OCI configuration, and a GitOps layer using FluxCD. The repository also generates a .kube.config file that can be used to access the resulting cluster.
From a BYOC perspective, this creates a straightforward integration path:
Oracle Cloud
│
▼
Terraform
│
▼
OKE Cluster
│
▼
.kube.config
│
▼
Nife BYOC
│
▼
Application Deployment
The infrastructure stays in the customer's Oracle Cloud account, while Nife can operate as the application deployment layer through the Kubernetes API.
The ARM64 architecture is another important consideration. Since the repository uses ARM instances, applications deployed to this environment need ARM64-compatible container images or multi-platform images.
The overall concept is simple:
Provision the Kubernetes cluster where you want it, connect it using kubeconfig, and use Nife as the deployment layer on top of your existing Kubernetes infrastructure.
Further Reading
- oci-free-cloud-k8s repository
- Oracle Kubernetes Engine documentation
- Oracle Kubernetes Engine — Accessing a Cluster
- Nife Documentation
- Nife
Nife: Deploy applications, containers, databases, websites, and infrastructure from one platform.
© Nife — Deploy anything, anywhere.
From the team at Nife
Deploy your next app in minutes, not days.
Nife is a multi-cloud edge platform that lets you deploy, scale, and manage applications globally without the infrastructure headache.