8 Commits

Author SHA1 Message Date
9511db8ee9 Set correct package name for Go package generation
All checks were successful
Generate and Release Protos / release (push) Successful in 26s
2026-01-08 23:15:48 +00:00
fc92331238 Try and fix AGAIN
All checks were successful
Generate and Release Protos / release (push) Successful in 26s
2026-01-08 23:06:45 +00:00
a4e6386fd1 Attempt to fix internal ZIP structure again?
All checks were successful
Generate and Release Protos / release (push) Successful in 26s
2026-01-08 22:55:15 +00:00
3199037c5a Fix structure of internal ZIP file for the Go release
All checks were successful
Generate and Release Protos / release (push) Successful in 25s
2026-01-08 22:47:30 +00:00
a78e749d55 Fix the Go version string
All checks were successful
Generate and Release Protos / release (push) Successful in 28s
2026-01-08 22:38:59 +00:00
fb4d7b2ed6 Remove translates proto files 2026-01-07 20:13:02 +00:00
3d750b1b18 Add MQFileIpdate schema (Currently supporting PIS only) 2026-01-07 20:09:10 +00:00
f9463f3d29 Update README 2026-01-07 18:46:15 +00:00
7 changed files with 83 additions and 85 deletions

View File

@@ -73,27 +73,29 @@ jobs:
- name: Publish Go - name: Publish Go
run: | run: |
# 1. Setup variables VERSION="v${{ steps.get_version.outputs.VERSION }}"
VERSION="${{ steps.get_version.outputs.VERSION }}"
MOD_NAME="git.fjla.uk/owlboard/backend-data-contracts" MOD_NAME="git.fjla.uk/owlboard/backend-data-contracts"
# Create a temporary directory structure that matches Go proxy requirements ZIP_ROOT="/tmp/go_upload"
DEST_DIR="temp_zip/$MOD_NAME@$VERSION" FULL_PATH="$ZIP_ROOT/$MOD_NAME@$VERSION"
# 2. Generate go.mod and tidy # 1. Prepare
cd gen/go cd gen/go
go mod init $MOD_NAME
go mod tidy
# 3. Move files into the required nested structure # 2. Initialize the module
mkdir -p "../../$DEST_DIR" go mod init "$MOD_NAME"
cp -r . "../../$DEST_DIR/"
# 4. Zip from the temp root so the internal paths are correct # 3. Create the structure
cd ../../temp_zip mkdir -p "$FULL_PATH"
zip -r ../module.zip .
# 5. Upload with the explicit version # 4. Copy the CONTENTS of models to the root of the module
cd .. # This flattens the structure so the .go files are next to go.mod
curl -f -v --user "owlbot:${{ secrets.PACKAGE_PUSH }}" \ cp -r models/* "$FULL_PATH/"
--upload-file module.zip \ cp go.mod "$FULL_PATH/"
# 5. Zip and Upload
cd "$ZIP_ROOT"
zip -r -D "$GITHUB_WORKSPACE/module.zip" .
curl -f --user "owlbot:${{ secrets.PACKAGE_PUSH }}" \
--upload-file "$GITHUB_WORKSPACE/module.zip" \
"${{ github.server_url }}/api/packages/owlboard/go/upload?version=$VERSION" "${{ github.server_url }}/api/packages/owlboard/go/upload?version=$VERSION"

View File

@@ -1,32 +1,18 @@
# backend-data-contracts # backend-data-contracts
This repository is the single source of truth for all Protocol Buffer (Protobuf) schema definitions used across the Owlboard Rail Ingress and Processing microservices. It follows a **Contract-First** approach, where language-specific code (Go, TypeScript) is generated and distributed via private registries. This repository is the single source of truth for all schema definitions used across the Owlboard backend communication and storage services. Language specific types are generated here and published to the Gitea package repository linked to the repo.
## Purpose
The Protobuf files defined here serve as the immutable data contract for:
1. **Ingress Logic:** Ensuring the Node.js and Go inregrate properly by sharing types.
2. **Message Payloads:** Defining messages for cross-service communication.
3. **Persistence:** Defining the expected structure of documents in MongoDB for the Go processing services.
## Directory Structure ## Directory Structure
| Path | Description | Contents | | Path | Description |
| :--- | :--- | :--- | | :--- | :--- |
| `proto/` | **Source of Truth.** Contains the Protobuf schema definitions. | `rail_backend/v1/*.proto` | | `schemas` | JSON Schema files organised into clear subfolders |
| `buf.yaml` | **Workspace Config.** Defines linting and breaking change rules (Buf v2). | Workspace settings | | `scripts` | Workflow Scripts |
| `buf.gen.yaml` | **Generation Config.** Defines how Go and TS code is built. | Plugin & Managed Mode settings |
| `gen/` | **Transient Output.** Generated code resides here during CI/CD. | `.pb.go`, `.js`, `.d.ts` (**Git Ignored**) |
## Code Generation and Publishing Workflow ## Code Generation and Publishing Workflow
The generation and release process is automated via Gitea Actions. It is triggered whenever a new **SemVer tag** (e.g., `v1.0.0`) is pushed to this repository. The generation and release process is automated via Gitea Actions. It is triggered whenever a new **SemVer tag** (e.g., `v1.0.0`) is pushed to this repository.
1. **Validation:** `buf lint` ensures schemas follow best practices.
2. **Generation:** `buf generate` creates Go and TypeScript code using local plugins.
3. **TS Release:** Compiles `.ts` to `.js` and publishes to the Gitea NPM Registry as `@owlboard/backend-data-contracts`.
4. **Go Release:** Initializes a `go.mod`, zips the artifacts, and pushes them to the Gitea Go Package Registry.
## To Consume the Contracts ## To Consume the Contracts
### 1. Go Services ### 1. Go Services
@@ -34,4 +20,18 @@ Since the package is hosted in the Gitea Package Registry, you must configure yo
**Setup:** **Setup:**
```bash ```bash
export GOPROXY=[https://git.fjla.uk/api/packages/owlboard/go,https://proxy.golang.org,direct](https://git.fjla.uk/api/packages/owlboard/go,https://proxy.golang.org,direct) export GOPROXY=[https://git.fjla.uk/api/packages/owlboard/go,https://proxy.golang.org,direct](https://git.fjla.uk/api/packages/owlboard/go,https://proxy.golang.org,direct)
```
### 2. Typescript Services
You will need to configure .npmrc in your projects root directory to point to the correct repo:
```bash
@owlboard:registry=https://git.fjla.uk/api/packages/OwlBoard/npm/
```
Then you can install as usual:
```bash
npm install @owlboard/backend-data-contracts@0.1.0
```

View File

@@ -1,10 +0,0 @@
syntax = "proto3";
package rail_backend.v1;
message Metadata {
int64 push_to_queue_time = 1;
int64 data_fetch_time = 2;
map<string, string> tags = 3;
}

View File

@@ -1,16 +0,0 @@
syntax = "proto3";
package rail_backend.v1;
message PisReferenceList {
repeated PisMapping entries = 1;
}
message PisMapping {
string code = 1;
string toc = 2;
repeated string crsStops = 3;
fixed64 crsHash = 4; // XXH4 Hash for fast lookup of exact match
repeated string tiplocStops = 5;
fixed64 tiplocHash = 6;
}

View File

@@ -1,20 +0,0 @@
syntax = "proto3";
package rail_backend.v1;
import "rail_backend/v1/common.proto";
import "rail_backend/v1/schedule_payload.proto";
message IngressMessage {
string correlation_id = 1;
Metadata tracking_data = 2;
oneof payload {
UrlReference url_ref = 5;
SchedulePayload schedule_payload = 6;
}
}
message UrlReference {
string kind = 1;
string url = 2;
}

View File

@@ -0,0 +1,42 @@
{
"$id": "https://schema.owlboard.info/data-ingress/mq-file-update.schema.json",
"$schema": "https://json-schema.org/draft-07/schema#",
"title": "MQFileUpdate",
"type": "object",
"properties": {
"service_name": {
"type": "string",
"description": "Name of the service submitting the update"
},
"update_type": {
"type": "string",
"enum": ["file"],
"description": "The method of update application"
},
"sent_timestamp": {
"type": "integer",
"description": "Unix timestamp representing the time the message was sent"
},
"payload": {
"type": "object",
"properties": {
"kind": {
"type": "string",
"enum": ["pis"],
"description": "The type of data provided in the update"
},
"version": {
"type": "string",
"description": "The version string from the package source"
},
"filepath": {
"type": "string",
"description": "The full path to the file, including protocol (eg. s3://) where appropriate"
}
},
"required": ["kind", "version", "filepath"]
}
},
"required": ["service_name", "update_type", "payload"],
"additionalProperties": false
}

View File

@@ -18,7 +18,7 @@ for file in $FILES; do
npx --yes json-schema-to-typescript "$file" > "gen/ts/${clean_name}.ts" npx --yes json-schema-to-typescript "$file" > "gen/ts/${clean_name}.ts"
# Generate Go # Generate Go
go-jsonschema -p models "$file" > "gen/go/models/${clean_name}.go" go-jsonschema -p backend_data_contracts "$file" > "gen/go/models/${clean_name}.go"
done done
echo "✅ Generated single TS package in gen/ts" echo "✅ Generated single TS package in gen/ts"