9 Commits

Author SHA1 Message Date
af3c82518c Change go package name
All checks were successful
Generate and Release Protos / release (push) Successful in 26s
2026-01-08 23:30:56 +00:00
9511db8ee9 Set correct package name for Go package generation
All checks were successful
Generate and Release Protos / release (push) Successful in 26s
2026-01-08 23:15:48 +00:00
fc92331238 Try and fix AGAIN
All checks were successful
Generate and Release Protos / release (push) Successful in 26s
2026-01-08 23:06:45 +00:00
a4e6386fd1 Attempt to fix internal ZIP structure again?
All checks were successful
Generate and Release Protos / release (push) Successful in 26s
2026-01-08 22:55:15 +00:00
3199037c5a Fix structure of internal ZIP file for the Go release
All checks were successful
Generate and Release Protos / release (push) Successful in 25s
2026-01-08 22:47:30 +00:00
a78e749d55 Fix the Go version string
All checks were successful
Generate and Release Protos / release (push) Successful in 28s
2026-01-08 22:38:59 +00:00
fb4d7b2ed6 Remove translates proto files 2026-01-07 20:13:02 +00:00
3d750b1b18 Add MQFileIpdate schema (Currently supporting PIS only) 2026-01-07 20:09:10 +00:00
f9463f3d29 Update README 2026-01-07 18:46:15 +00:00
7 changed files with 83 additions and 85 deletions

View File

@@ -73,27 +73,29 @@ jobs:
- name: Publish Go
run: |
# 1. Setup variables
VERSION="${{ steps.get_version.outputs.VERSION }}"
VERSION="v${{ steps.get_version.outputs.VERSION }}"
MOD_NAME="git.fjla.uk/owlboard/backend-data-contracts"
# Create a temporary directory structure that matches Go proxy requirements
DEST_DIR="temp_zip/$MOD_NAME@$VERSION"
ZIP_ROOT="/tmp/go_upload"
FULL_PATH="$ZIP_ROOT/$MOD_NAME@$VERSION"
# 2. Generate go.mod and tidy
# 1. Prepare
cd gen/go
go mod init $MOD_NAME
go mod tidy
# 3. Move files into the required nested structure
mkdir -p "../../$DEST_DIR"
cp -r . "../../$DEST_DIR/"
# 2. Initialize the module
go mod init "$MOD_NAME"
# 4. Zip from the temp root so the internal paths are correct
cd ../../temp_zip
zip -r ../module.zip .
# 3. Create the structure
mkdir -p "$FULL_PATH"
# 5. Upload with the explicit version
cd ..
curl -f -v --user "owlbot:${{ secrets.PACKAGE_PUSH }}" \
--upload-file module.zip \
# 4. Copy the CONTENTS of models to the root of the module
# This flattens the structure so the .go files are next to go.mod
cp -r models/* "$FULL_PATH/"
cp go.mod "$FULL_PATH/"
# 5. Zip and Upload
cd "$ZIP_ROOT"
zip -r -D "$GITHUB_WORKSPACE/module.zip" .
curl -f --user "owlbot:${{ secrets.PACKAGE_PUSH }}" \
--upload-file "$GITHUB_WORKSPACE/module.zip" \
"${{ github.server_url }}/api/packages/owlboard/go/upload?version=$VERSION"

View File

@@ -1,32 +1,18 @@
# backend-data-contracts
This repository is the single source of truth for all Protocol Buffer (Protobuf) schema definitions used across the Owlboard Rail Ingress and Processing microservices. It follows a **Contract-First** approach, where language-specific code (Go, TypeScript) is generated and distributed via private registries.
## Purpose
The Protobuf files defined here serve as the immutable data contract for:
1. **Ingress Logic:** Ensuring the Node.js and Go inregrate properly by sharing types.
2. **Message Payloads:** Defining messages for cross-service communication.
3. **Persistence:** Defining the expected structure of documents in MongoDB for the Go processing services.
This repository is the single source of truth for all schema definitions used across the Owlboard backend communication and storage services. Language specific types are generated here and published to the Gitea package repository linked to the repo.
## Directory Structure
| Path | Description | Contents |
| :--- | :--- | :--- |
| `proto/` | **Source of Truth.** Contains the Protobuf schema definitions. | `rail_backend/v1/*.proto` |
| `buf.yaml` | **Workspace Config.** Defines linting and breaking change rules (Buf v2). | Workspace settings |
| `buf.gen.yaml` | **Generation Config.** Defines how Go and TS code is built. | Plugin & Managed Mode settings |
| `gen/` | **Transient Output.** Generated code resides here during CI/CD. | `.pb.go`, `.js`, `.d.ts` (**Git Ignored**) |
| Path | Description |
| :--- | :--- |
| `schemas` | JSON Schema files organised into clear subfolders |
| `scripts` | Workflow Scripts |
## Code Generation and Publishing Workflow
The generation and release process is automated via Gitea Actions. It is triggered whenever a new **SemVer tag** (e.g., `v1.0.0`) is pushed to this repository.
1. **Validation:** `buf lint` ensures schemas follow best practices.
2. **Generation:** `buf generate` creates Go and TypeScript code using local plugins.
3. **TS Release:** Compiles `.ts` to `.js` and publishes to the Gitea NPM Registry as `@owlboard/backend-data-contracts`.
4. **Go Release:** Initializes a `go.mod`, zips the artifacts, and pushes them to the Gitea Go Package Registry.
## To Consume the Contracts
### 1. Go Services
@@ -35,3 +21,17 @@ Since the package is hosted in the Gitea Package Registry, you must configure yo
**Setup:**
```bash
export GOPROXY=[https://git.fjla.uk/api/packages/owlboard/go,https://proxy.golang.org,direct](https://git.fjla.uk/api/packages/owlboard/go,https://proxy.golang.org,direct)
```
### 2. Typescript Services
You will need to configure .npmrc in your projects root directory to point to the correct repo:
```bash
@owlboard:registry=https://git.fjla.uk/api/packages/OwlBoard/npm/
```
Then you can install as usual:
```bash
npm install @owlboard/backend-data-contracts@0.1.0
```

View File

@@ -1,10 +0,0 @@
syntax = "proto3";
package rail_backend.v1;
message Metadata {
int64 push_to_queue_time = 1;
int64 data_fetch_time = 2;
map<string, string> tags = 3;
}

View File

@@ -1,16 +0,0 @@
syntax = "proto3";
package rail_backend.v1;
message PisReferenceList {
repeated PisMapping entries = 1;
}
message PisMapping {
string code = 1;
string toc = 2;
repeated string crsStops = 3;
fixed64 crsHash = 4; // XXH4 Hash for fast lookup of exact match
repeated string tiplocStops = 5;
fixed64 tiplocHash = 6;
}

View File

@@ -1,20 +0,0 @@
syntax = "proto3";
package rail_backend.v1;
import "rail_backend/v1/common.proto";
import "rail_backend/v1/schedule_payload.proto";
message IngressMessage {
string correlation_id = 1;
Metadata tracking_data = 2;
oneof payload {
UrlReference url_ref = 5;
SchedulePayload schedule_payload = 6;
}
}
message UrlReference {
string kind = 1;
string url = 2;
}

View File

@@ -0,0 +1,42 @@
{
"$id": "https://schema.owlboard.info/data-ingress/mq-file-update.schema.json",
"$schema": "https://json-schema.org/draft-07/schema#",
"title": "MQFileUpdate",
"type": "object",
"properties": {
"service_name": {
"type": "string",
"description": "Name of the service submitting the update"
},
"update_type": {
"type": "string",
"enum": ["file"],
"description": "The method of update application"
},
"sent_timestamp": {
"type": "integer",
"description": "Unix timestamp representing the time the message was sent"
},
"payload": {
"type": "object",
"properties": {
"kind": {
"type": "string",
"enum": ["pis"],
"description": "The type of data provided in the update"
},
"version": {
"type": "string",
"description": "The version string from the package source"
},
"filepath": {
"type": "string",
"description": "The full path to the file, including protocol (eg. s3://) where appropriate"
}
},
"required": ["kind", "version", "filepath"]
}
},
"required": ["service_name", "update_type", "payload"],
"additionalProperties": false
}

View File

@@ -18,7 +18,7 @@ for file in $FILES; do
npx --yes json-schema-to-typescript "$file" > "gen/ts/${clean_name}.ts"
# Generate Go
go-jsonschema -p models "$file" > "gen/go/models/${clean_name}.go"
go-jsonschema -p contracts "$file" > "gen/go/models/${clean_name}.go"
done
echo "✅ Generated single TS package in gen/ts"