Tidy up, make it easier for users.

docs/Admin/Virtualisation/pve-k8s-ceph-config.md

@@ -52,7 +52,7 @@ The PVE firewall must explicitly **allow inbound traffic** from the entire Kuber
 
 Alternatively, you may find a 'ceph' macro you can use on the PVE Firewall, if so use the Macro instead of additional rules.
 
-### B. PVE Host Static Routing (Ceph $\rightarrow$ K8s)
+### B. PVE Host Static Routing (Ceph > K8s)
 
 Add **persistent static routes** on **all PVE Ceph hosts** to allow Ceph to send responses back to the Pod Network.
 
@@ -60,7 +60,7 @@ Add **persistent static routes** on **all PVE Ceph hosts** to allow Ceph to send
     ```ini
     # Example:
     post-up ip route add <POD_NETWORK_CIDR> via <K8S_NODE_IP> dev <PVE_INTERFACE>
-    # e.g., post-up ip route add 10.1.0.0/16 via 172.30.100.41 dev vmbr0
+    # e.g., post-up ip route add 10.1.0.0/16 via 172.35.100.40 dev vmbr0
     ```
 
 ### C. K8s Node IP Forwarding (Gateway Function)
@@ -73,7 +73,7 @@ Enable IP forwarding on **all Kubernetes nodes** so they can route incoming Ceph
     sudo sh -c 'echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.d/99-sysctl.conf'
     ```
     
-### D. K8s Static Routing (K8s $\rightarrow$ Ceph) - Conditional/Advanced ⚠️
+### D. K8s Static Routing (K8s > Ceph) - Conditional/Advanced ⚠️
 
 This routing is **only required** if the **Ceph Public Network** (the network Ceph Monitors/OSDs listen on) is **not reachable** by your Kubernetes Node's **default gateway**.
 
@@ -86,29 +86,29 @@ This routing is **only required** if the **Ceph Public Network** (the network Ce
       ethernets:
         eth0: # Replace with your primary K8s network interface
           routes:
-          # Route 1: Directs traffic destined for the first Ceph Monitor IP (10.15.16.1) 
-          # through three different PVE hosts (172.30.25.x) as gateways.
+          # Route 1: Directs traffic destined for the first Ceph Monitor IP (10.11.12.1) 
+          # through three different PVE hosts (172.35.100.x) as gateways.
           # The lowest metric (10) is preferred.
-          - to: 10.15.16.1/32
-            via: 172.30.25.10
+          - to: 10.11.12.1/32
+            via: 172.35.100.10
             metric: 10
-          - to: 10.15.16.1/32
-            via: 172.30.25.20
+          - to: 10.11.12.1/32
+            via: 172.35.100.20
             metric: 100
-          - to: 10.15.16.1/32
-            via: 172.30.25.30
+          - to: 10.11.12.1/32
+            via: 172.35.100.30
             metric: 100
           
-          # Route 2: Directs traffic destined for the second Ceph Monitor IP (10.15.16.2)
+          # Route 2: Directs traffic destined for the second Ceph Monitor IP (10.11.12.2)
           # with a similar failover strategy.
-          - to: 10.15.16.2/32
-            via: 172.30.25.20
+          - to: 10.11.12.2/32
+            via: 172.35.100.20
             metric: 10
-          - to: 10.15.16.2/32
-            via: 172.30.25.10
+          - to: 10.11.12.2/32
+            via: 172.35.100.10
             metric: 100
-          - to: 10.15.16.2/32
-            via: 172.30.25.30
+          - to: 10.11.12.2/32
+            via: 172.35.100.30
             metric: 100
     ```
     
@@ -116,7 +116,9 @@ Use route priorities (Lower is Higher) to prefer the most direct path, while sti
 
 ---
 
-## 4. 🧩 MicroK8s CSI Driver Configuration (The Path Fix)
+## 4. 🧩 MicroK8s CSI Driver Configuration (The Path Fix) - Conditional/Advanced ⚠️
+
+This adjustment is **only required** if **MicroK8s** is running your Kuberneted deployment.  **Alternative changes** may be needed for other Kubetnetes distributions.
 
 This resolves the **`staging path does not exist on node`** error for the Node Plugin.
 

docs/Admin/Virtualisation/rados-gw-in-k8s.md

@@ -0,0 +1,272 @@
+# Deploy Ceph RADOS Gateway in Kubernetes
+## Pointing to a separate PVE Ceph Cluster
+
+This guide outlines how to deploy a RADOS Gateway to enable an S3 API for a Ceph pool.  I use this to provide S3 storage to my Kubernetes Cluster with the Ceph cluster hosted by Proxmox VE.  Many conecpts are similar to the previous guide - Enable Ceph CSI for PVE Ceph, some steps will refer to that guide.
+
+This guide makes the following assumptions:
+* You are already runnung Ceph via PVE.
+* You are using the PVE UI for Ceph actions where possible.
+* You are deploying the RADOS Gateway to the `object-store` namespace in K8s.
+* Flux is used to deploy to K8s using SOPS for secret encryption.
+
+### 1. Ceph Pool & User Creation
+
+These steps ensure that a Ceph Pool is created with appropriate Replication.
+
+* Create the RGW Realm on a PVE Host from the Shell
+    * Create Realm: `radosgw-admin realm create --rgw-realm=default --default`
+    * Create Zonegroup: `radosgw-admin zonegroup create --rgw-zonegroup=default --master --default --endpoints=http://ceph-rgw.object-store.svc.cluster.local:8080`
+    * Create Zone: `radosgw-admin zone create --rgw-zone=default --master --default`
+    * Set Zone endpoint: `radosgw-admin zone modify --rgw-zone=default --endpoints=http://ceph-rgw.object-store.svc.cluster.local:8080`
+    * Ensure Zone is included in Zonegroup: `radosgw-admin zonegroup add --rgw-zonegroup=default --rgw-zone=default`
+    * Update & Commit Period: `radosgw-admin period update --commit`
+    * Set the default realm: `radosgw-admin realm default --rgw-realm=default`
+
+* The above commands will have created the following new pools
+  **You do not need to manually create these**
+
+    |Pool Name|Purpose|
+    | :----- | :----- |
+    |.rgw.root|
+    |default.rgw.log|
+    |default.rgw.control|
+    |default.rgw.meta|
+
+* Create the two required Pools for index and data in the PVE UI:
+    
+    |Pool Name | PG Autoscaler | Size | Min Size | Crush Rule |
+    | :------- | :------------ | :--- | :------- | :--------- |
+    | default.rgw.buckets.index | On | 3 | 2 | replicated_rule |
+    | default.rgw.buckets.data | On | 3 | 2 | replicated_rule |
+
+* Enable RGW Application:
+  When the pool is created via PVE, it is registered by default as an RBD Pool, 
+  run these commands to change it to an RGW pool.
+  * Disable RBD: `ceph osd pool application disable default.rgw.buckets.data rbd --yes-i-really-mean-it`
+  * Enable RGW: `ceph osd pool application enable default.rgw.buckets.data rgw`
+  * Check with: `ceph osd pool application get default.rgw.buckets.data`
+  * Repeat for index pool: `ceph osd pool application disable default.rgw.buckets.index rbd --yes-i-really-mean-it`
+  * Enable RGW: `ceph osd pool application enable default.rgw.buckets.index rgw`
+  * Check with: `ceph osd pool application get default.rgw.buckets.index`
+
+* Create a user for the RADOS Gateway:
+  ```bash
+  ceph auth get-or-create client.rgw.k8s.svc \
+      mon 'allow r' \
+      osd 'allow rwx pool=default.rgw.buckets.data, allow rwx pool=default.rgw.buckets.index, allow rwx pool=.rgw.root, allow rwx pool=default.rgw.meta, allow rwx pool=default.rgw.log, allow rwx pool=default.rgw.control' \
+      -o /etc/ceph/ceph.client.rgw.k8s.svc.keyring
+  ```
+
+### 2. Register Kubernetes Secrets
+
+* Retreive the files, from the Ceph host, required for Kubernetes Secrets:
+    Retreive these files and store them **temporarily** on your workstation.
+    
+    | File | Path | Purpose |
+    | :------- | :--- | :------ |
+    |ceph.conf|/etc/ceph/ceph.conf|Location of Ceph Monitors|
+    |Keyring | /etc/ceph/ceph.client.rgw.k8s.svc.keyring | Auth token |
+
+* **CRITICAL:** A newline must be present at the end of each file.
+* **CRITICAL:** Remove whitespace from the keyring file, except newlines.
+
+* Create Secret manifests for deployment to K8s:
+
+  ```bash
+  kubectl create secret generic ceph-config \
+    --namespace=object-store \
+    --from-file=ceph.conf=./conf \
+    --dry-run=client -o yaml > ceph-config-secret.yaml
+  ```
+  ```bash
+  kubectl create secret generic ceph-keyring \
+    --namespace=object-store \
+    --from-file=keyring=./keyring \
+    --dry-run=client -o yaml > ceph-keyring-secret.yaml
+  ```
+
+* Encrypt the secret manifests using sops:
+  * `sops encrypt --in-place ./ceph-config-secret.yaml`
+  * `sops encrypt --in-place ./ceph-keyring-secret.yaml`
+
+### 3. Kubernetes Manifests
+
+**These should be treated as examples, read through them and ensure they match your environment**
+
+#### Namespace
+
+```yaml
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: object-store
+```
+
+#### Service
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+  name: ceph-rgw-svc
+  namespace: object-store
+  labels:
+    app.kubernetes.io/name: ceph-rgw
+    app.kubernetes.io/component: gateway
+spec:
+  # The ClusterIP DNS name used for the RGW initialization:
+  # http://ceph-rgw-svc.object-store.svc.cluster.local:8080
+  ports:
+    - port: 8080 
+      targetPort: 8080
+      protocol: TCP
+      name: http-api
+  selector:
+    app: ceph-rgw
+  type: ClusterIP
+```
+
+#### Deployment
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: ceph-rgw
+  namespace: object-store
+  labels:
+    app.kubernetes.io/name: ceph-rgw
+    app.kubernetes.io/component: gateway
+spec:
+  replicas: 2
+  selector:
+    matchLabels:
+      app: ceph-rgw
+  strategy:
+    type: RollingUpdate
+  template:
+    metadata:
+      labels:
+        app: ceph-rgw
+    spec:
+      # CRUCIAL: Enforce Pods to be on separate nodes for HA
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchLabels:
+                  app: ceph-rgw
+              topologyKey: "kubernetes.io/hostname" 
+      containers:
+      - name: rgw
+        # Use the same Major:Minor as your PVE Hosts
+        image: quay.io/ceph/ceph:v18.2
+        # Arguments to start the RGW process on port 8080
+        args: [
+          "radosgw",
+          "-f",                         # Run in foreground
+          "--conf=/etc/ceph/ceph.conf", # Explicitly use the mounted config
+          "--name=client.rgw.k8s.svc",  # The exact CephX user name we created
+          "--rgw-frontends=beast port=8080" # REQUIRED: Beast frontend for Ceph 18+
+        ]
+        resources:
+          requests:
+            cpu: 500m
+            memory: 2Gi
+          limits:
+            cpu: 2000m
+            memory: 2Gi
+        ports:
+        - containerPort: 8080
+          name: rgw-http
+        # Ensure the Pod does not run as root unnecessarily
+        securityContext:
+          runAsUser: 167 # A common non-root user ID for Ceph containers
+          runAsGroup: 167
+          allowPrivilegeEscalation: false
+        volumeMounts:
+        # Mount the ceph.conf file directly
+        - name: ceph-config-vol
+          mountPath: /etc/ceph/
+      volumes:
+      - name: ceph-config-vol
+        secret:
+          secretName: ceph-config
+          defaultMode: 0444 # Global read for user 167
+          items:
+          - key: ceph.conf
+            path: ceph.conf
+          - key: ceph.client.rgw.k8s.svc.keyring
+            path: ceph.client.rgw.k8s.svc.keyring
+```
+
+**Deploy these manifests to Flux**
+
+### 4. RGW Admin Utility
+
+**Do not commit this to Flux, run as and when required to manage RGW users and buckets**
+
+#### Pod Manifest
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: rgw-admin-utility
+  namespace: object-store
+spec:
+  restartPolicy: Never 
+  containers:
+  - name: rgw-admin-cli
+    # Use the same image as your RGW deployment for consistency
+    image: quay.io/ceph/ceph:v18.2 
+    # Use the /bin/bash entrypoint to allow manual command execution
+    command: ["/bin/bash", "-c", "sleep 3600"] 
+    # Environment variable to explicitly define the CephX user for CLI tools
+    env:
+    - name: CEPH_ARGS
+      value: "--name client.rgw.k8s.svc --keyring /etc/ceph/ceph.client.rgw.k8s.svc.keyring"
+      volumeMounts:
+      # Mount the ceph.conf file directly
+      - name: ceph-config-vol
+        mountPath: /etc/ceph/
+    volumes:
+    - name: ceph-config-vol
+      secret:
+        secretName: ceph-config
+        defaultMode: 0444 # Global read for user 167
+        items:
+        - key: ceph.conf
+          path: ceph.conf
+        - key: ceph.client.rgw.k8s.svc.keyring
+          path: ceph.client.rgw.k8s.svc.keyring
+```
+
+#### Managing RGW
+
+* Deploy the Pod using `kubectl apply -f {filepath}`
+* Exec into the pod `kubectl exec -it rgw-admin-utility -n object-store -- bash`
+
+##### Create User
+
+* `radosgw-admin user create --uid={uid} --display-name={display-name} --gen-key --gen-secret`
+
+**CRITICAL:** *Copy the JSON output, save the access_key and secret_key*
+
+##### Create Bucket
+
+* `radosgw-admin bucket create --bucket={buket-name} --uid={owner-uid}`
+
+##### Exit & Cleanup
+
+* `exit`
+* `kubectl delete pod rgw-admin-utility -n object-store`
+
+### 5. Generate Secret for Client Access
+
+Deploy this in the namespace of the appliation requiring the S3 API Access
+
+```bash
+kubectl create secret generic s3-credentials \
+  --namespace={application-namespace} \
+  --from-literal=S3_ACCESS_KEY={access-key-from-user-creation} \
+  --from-literal=S3_SECRET_KEY={secret-key-from-user-creation} \
+  --dry-run=client -o yaml > s3-secret.yaml
+```

docs/Admin/index.md

@@ -0,0 +1,31 @@
+## 🌐 Welcome to the Infrastructure Documentation Repository
+
+This site provides technical documentation, configuration standards, and operational guides for our core infrastructure services and deployment environments.
+---
+## 🛑 Disclaimer
+
+This documentation site is provided **"as is"** for informational and educational purposes only. Please read this disclaimer carefully before relying on any information contained within.
+
+---
+
+### 1. Accuracy and Currency
+
+While we strive to ensure the information contained herein is accurate and current, we **make no warranties, express or implied,** about the completeness, reliability, suitability, or availability of the content. Infrastructure environments are constantly changing, and configurations may become outdated or specific to our unique setup.  AI is used to produce some documentation based on steps taken to deploy the solution into a live environment.
+
+### 2. Security and Sensitive Information
+
+This public documentation **intentionally omits** all sensitive, proprietary, and infrastructure-specific details, including:
+
+* Actual **IP addresses, subnets, and hostnames**.
+* All **authentication keys, tokens, and secrets**.
+* Internal **network IDs or firewall rules** that could expose our systems.
+
+Any examples provided (e.g., code snippets, network diagrams) use **placeholder values** and **generic identifiers**. Do **not** use any of the example values in a production environment.
+
+### 3. Limitation of Liability
+
+The authors and maintainers of this documentation will **not be held liable** for any direct, indirect, incidental, consequential, or punitive damages arising from the use of, or reliance on, any information presented here.
+
+### 4. Use at Your Own Risk
+
+Any action you take upon the information on this documentation site is strictly **at your own risk**. We strongly advise that you test all configurations and principles documented here in a non-production, segregated environment before applying them to any live infrastructure.

docs/User/Account/reset-password.md

@@ -0,0 +1,3 @@
+# Password Reset
+
+This documentation is not written yet.

docs/User/index.md

@@ -0,0 +1,10 @@
+# User
+
+This group contains documentation aimed at end-users.
+
+## Helpful Links
+ - [Account Management](https://fjla.uk)
+ - [Password Reset](https://fjla.uk/reset)
+
+## Help Documents
+ - [How do I reset my password?](Account/reset-password.md)

docs/index.md

@@ -1,31 +1,7 @@
-## 🌐 Welcome to the Infrastructure Documentation Repository
+## 🌐 Welcome to the FJLA Documentation
 
-This site provides technical documentation, configuration standards, and operational guides for our core infrastructure services and deployment environments.
----
-## 🛑 Disclaimer
+This site provides user documentation, technical documentation, configuration standards, and operational guides for our core infrastructure services and deployment environments.
 
-This documentation site is provided **"as is"** for informational and educational purposes only. Please read this disclaimer carefully before relying on any information contained within.
+If you are a user seeking help [click here](User/index.md).
 
----
-
-### 1. Accuracy and Currency
-
-While we strive to ensure the information contained herein is accurate and current, we **make no warranties, express or implied,** about the completeness, reliability, suitability, or availability of the content. Infrastructure environments are constantly changing, and configurations may become outdated or specific to our unique setup.  AI is used to produce documentation from the steps taken to deploy the solution into a live environment.
-
-### 2. Security and Sensitive Information
-
-This public documentation **intentionally omits** all sensitive, proprietary, and infrastructure-specific details, including:
-
-* Actual **IP addresses, subnets, and hostnames**.
-* All **authentication keys, tokens, and secrets**.
-* Internal **network IDs or firewall rules** that could expose our systems.
-
-Any examples provided (e.g., code snippets, network diagrams) use **placeholder values** and **generic identifiers**. Do **not** use any of the example values in a production environment.
-
-### 3. Limitation of Liability
-
-The authors and maintainers of this documentation will **not be held liable** for any direct, indirect, incidental, consequential, or punitive damages arising from the use of, or reliance on, any information presented here.
-
-### 4. Use at Your Own Risk
-
-Any action you take upon the information on this documentation site is strictly **at your own risk**. We strongly advise that you test all configurations and principles documented here in a non-production, segregated environment before applying them to any live infrastructure.
+You can use the menu to navigate through help documents and guides.

mkdocs.yaml

@@ -1,8 +1,14 @@
 site_name: FJLA Documentation
 nav:
 - Home: index.md
-- Virtualisation:
-  - Combining K8s, PVE & Ceph: Virtualisation/pve-k8s-ceph-config.md
+- User:
+  - Account:
+    - Reset or Change Password: User/Account/reset-password.md
+- Admin:
+  - About: Admin/index.md
+  - Virtualisation & Orchestration:
+    - Combining K8s, PVE & Ceph: Admin/Virtualisation/pve-k8s-ceph-config.md
+    - Deploy RADOS Gateway in K8s: Admin/Virtualisation/rados-gw-in-k8s.md
 
 theme:
   name: material