fred.boniface/fjla-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update Ceph/PVE/K8s Document

Browse Source
This commit is contained in:
Fred Boniface
2025-12-11 15:06:57 +00:00
parent 8a1f967a5d
commit c1f83b9275
1 changed files with 19 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
docs/Virtualisation/pve-k8s-ceph-config.md
View File

@@ -60,7 +60,7 @@ Add **persistent static routes** on **all PVE Ceph hosts** to allow Ceph to send
```ini ```ini
# Example: # Example:
post-up ip route add <POD_NETWORK_CIDR> via <K8S_NODE_IP> dev <PVE_INTERFACE> post-up ip route add <POD_NETWORK_CIDR> via <K8S_NODE_IP> dev <PVE_INTERFACE>
# e.g., post-up ip route add 10.1.0.0/16 via 172.30.100.41 dev vmbr0 # e.g., post-up ip route add 10.1.0.0/16 via 172.35.100.40 dev vmbr0
``` ```
### C. K8s Node IP Forwarding (Gateway Function) ### C. K8s Node IP Forwarding (Gateway Function)
@@ -86,29 +86,29 @@ This routing is **only required** if the **Ceph Public Network** (the network Ce
ethernets: ethernets:
eth0: # Replace with your primary K8s network interface eth0: # Replace with your primary K8s network interface
routes: routes:
# Route 1: Directs traffic destined for the first Ceph Monitor IP (10.15.16.1) # Route 1: Directs traffic destined for the first Ceph Monitor IP (10.11.12.1)
# through three different PVE hosts (172.30.25.x) as gateways. # through three different PVE hosts (172.35.100.x) as gateways.
# The lowest metric (10) is preferred. # The lowest metric (10) is preferred.
- to: 10.15.16.1/32 - to: 10.11.12.1/32
via: 172.30.25.10 via: 172.35.100.10
metric: 10 metric: 10
- to: 10.15.16.1/32 - to: 10.11.12.1/32
via: 172.30.25.20 via: 172.35.100.20
metric: 100 metric: 100
- to: 10.15.16.1/32 - to: 10.11.12.1/32
via: 172.30.25.30 via: 172.35.100.30
metric: 100 metric: 100
# Route 2: Directs traffic destined for the second Ceph Monitor IP (10.15.16.2) # Route 2: Directs traffic destined for the second Ceph Monitor IP (10.11.12.2)
# with a similar failover strategy. # with a similar failover strategy.
- to: 10.15.16.2/32 - to: 10.11.12.2/32
via: 172.30.25.20 via: 172.35.100.20
metric: 10 metric: 10
- to: 10.15.16.2/32 - to: 10.11.12.2/32
via: 172.30.25.10 via: 172.35.100.10
metric: 100 metric: 100
- to: 10.15.16.2/32 - to: 10.11.12.2/32
via: 172.30.25.30 via: 172.35.100.30
metric: 100 metric: 100
``` ```
@@ -116,7 +116,9 @@ Use route priorities (Lower is Higher) to prefer the most direct path, while sti
--- ---
## 4. 🧩 MicroK8s CSI Driver Configuration (The Path Fix) ## 4. 🧩 MicroK8s CSI Driver Configuration (The Path Fix) - Conditional/Advanced ⚠️
This adjustment is **only required** if **MicroK8s** is running your Kuberneted deployment. **Alternative changes** may be needed for other Kubetnetes distributions.
This resolves the **`staging path does not exist on node`** error for the Node Plugin. This resolves the **`staging path does not exist on node`** error for the Node Plugin.