Fred Boniface fred.boniface
0 Followers · 0 Following
Repositories
22
 Projects Packages Public Activity Starred Repositories
fred.boniface deleted tag v1.0.0 from OwlBoard/go-types 2024-04-14 22:44:36 +01:00
fred.boniface pushed to main at OwlBoard/go-types 2024-04-14 22:43:44 +01:00
a432f1e4bb Fix go.mod
fred.boniface pushed tag v1.0.0 to OwlBoard/go-types 2024-04-14 22:37:15 +01:00
fred.boniface pushed to main at OwlBoard/ts-types 2024-04-14 22:33:38 +01:00
a3a4409e1d Bump version to align with go-types
fred.boniface pushed to main at OwlBoard/go-types 2024-04-14 22:30:49 +01:00
0a8ec0b509 Adjust VSTP message types to use CIF schedule types. Bump version
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-14 22:18:02 +01:00
3d730054c0 Adjust log levels
be6300829d Reogranise functions into appropriate files
421b68f936 Improve timezone handling
76f206441a Tidy logging in package cif and dbAccess/cif.go
c9f894bbe6 Add RemoveOutdatedServices
Compare 10 commits »
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-13 21:45:36 +01:00
77dc11a658 Add additional logging to identify why CIF updates were sometimes failing
16fa1c5354 Adjust how performance logging is managed
dc958ac717 Set parsed to nil before exiting function
Compare 3 commits »
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-12 20:43:15 +01:00
6ec8cd8895 Complete CIF Metadata parsing logic
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-11 21:06:32 +01:00
94f880bcde Add comment description to cif/types.go
251a190558 Set unordered for bulk CIF deletions from DB & reorganise dbAccess module.
Compare 2 commits »
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-11 20:59:13 +01:00
76ae25fab6 Add LastUpdateType to CIF Metadata
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-11 20:50:52 +01:00
4459d4d316 Update `package corpus` to stream data to the parse function.
b2f82b0250 Optimise batch size and stream handling
Compare 2 commits »
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-10 23:05:29 +01:00
fb510e1408 Add logging for unknown CIF data types
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-10 22:59:23 +01:00
751b67e61e Remove unused functions
fred.boniface created branch parseProcessMemoryImprovements in OwlBoard/timetable-mgr 2024-04-10 21:48:52 +01:00
fred.boniface pushed to parseProcessMemoryImprovements at OwlBoard/timetable-mgr 2024-04-10 21:48:52 +01:00
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-10 21:47:19 +01:00
fea7a5b831 Disable parsing of JSONAssociations as they are not currently used within OwlBoard
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-10 20:56:25 +01:00
3883031e04 Increase use of pointers in CORPUS package
39405e6d6a Add comments
f903219276 Streaming data into the CIF Parse function reduces memory down to 4.1GB - also looking to stream the parsed data directly into the database if feasable.
Compare 3 commits »
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-09 22:42:49 +01:00
7acae49812 Re-implement processParsedCif() to reduce memory use by 10%. Further reductions are neccessary
51f5c4fdcd Adjust file write functions to reduce memory load
94e4cd964d Improve CIF Parsing to acheive 60% memory reduction
a2c52f7b8b Increase use of pointers to improve memory usage. Currently uses around 16GB RAM.
Compare 4 commits »
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-09 20:55:53 +01:00
259f514b3d Add function to write full CIF download to filesystem
fred.boniface pushed to timetable-extension at OwlBoard/timetable-mgr 2024-04-09 20:39:30 +01:00
2c34c529e1 Add debug option to write CIF data to file.