timetable-mgr

Author	SHA1	Message	Date
Fred Boniface	ba8e4e4c72	Update VSTP package to use processing from CIF package	2024-04-15 20:37:04 +01:00
Fred Boniface	c9f894bbe6	Add RemoveOutdatedServices	2024-04-14 21:21:35 +01:00
Fred Boniface	91fd38104c	Reorganise logging	2024-04-14 19:03:13 +01:00
Fred Boniface	dc958ac717	Set parsed to nil before exiting function	2024-04-13 21:44:53 +01:00
Fred Boniface	6ec8cd8895	Complete CIF Metadata parsing logic All checks were successful Go Test / test (push) Successful in 1m3s Details	2024-04-12 20:43:03 +01:00
Fred Boniface	76ae25fab6	Add LastUpdateType to CIF Metadata All checks were successful Go Test / test (push) Successful in 37s Details	2024-04-11 20:59:02 +01:00
Fred Boniface	b2f82b0250	Optimise batch size and stream handling	2024-04-11 20:50:13 +01:00
Fred Boniface	39405e6d6a	Add comments	2024-04-10 20:46:20 +01:00
Fred Boniface	f903219276	Streaming data into the CIF Parse function reduces memory down to 4.1GB - also looking to stream the parsed data directly into the database if feasable.	2024-04-10 20:19:16 +01:00
Fred Boniface	7acae49812	Re-implement processParsedCif() to reduce memory use by 10%. Further reductions are neccessary All checks were successful Go Test / test (push) Successful in 31s Details	2024-04-09 22:39:35 +01:00
Fred Boniface	94e4cd964d	Improve CIF Parsing to acheive 60% memory reduction	2024-04-09 21:26:56 +01:00
Fred Boniface	a2c52f7b8b	Increase use of pointers to improve memory usage. Currently uses around 16GB RAM.	2024-04-09 21:07:21 +01:00
Fred Boniface	259f514b3d	Add function to write full CIF download to filesystem All checks were successful Go Test / test (push) Successful in 37s Details	2024-04-09 20:55:26 +01:00
Fred Boniface	2c34c529e1	Add debug option to write CIF data to file. All checks were successful Go Test / test (push) Successful in 1m20s Details	2024-04-09 20:38:40 +01:00
Fred Boniface	19f180d711	Expand CIF updates to ensure new sequence number is equal to oldSequence +1 All checks were successful Go Test / test (push) Successful in 32s Details	2024-04-08 22:13:08 +01:00
Fred Boniface	7146d1a883	Add printing of number of goroutines to assist with performance improvements	2024-04-08 21:22:56 +01:00
Fred Boniface	3e5ed2c10a	Writing full CIF data now works. Still need to work on updating CIF data.	2024-04-08 21:08:07 +01:00
Fred Boniface	3481c4e314	Tidy up All checks were successful Go Test / test (push) Successful in 1m43s Details	2024-04-06 22:31:38 +01:00
Fred Boniface	a15cbb778a	Implement generateMetadata function	2024-04-06 22:28:26 +01:00
Fred Boniface	493fb41da8	Reorganise repo Some checks failed Go Test / test (push) Failing after 48s Details	2024-04-05 22:23:42 +01:00

20 Commits