OwlBoard/timetable-mgr
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 3883031e04 Increase use of pointers in CORPUS package Fred Boniface 2024-04-10 20:56:13 +01:00
  • 39405e6d6a Add comments Fred Boniface 2024-04-10 20:46:20 +01:00
  • f903219276 Streaming data into the CIF Parse function reduces memory down to 4.1GB - also looking to stream the parsed data directly into the database if feasable. Fred Boniface 2024-04-10 20:19:16 +01:00
  • 7acae49812 Re-implement processParsedCif() to reduce memory use by 10%. Further reductions are neccessary Fred Boniface 2024-04-09 22:39:35 +01:00
  • 51f5c4fdcd Adjust file write functions to reduce memory load Fred Boniface 2024-04-09 22:38:48 +01:00
  • 94e4cd964d Improve CIF Parsing to acheive 60% memory reduction Fred Boniface 2024-04-09 21:26:56 +01:00
  • a2c52f7b8b Increase use of pointers to improve memory usage. Currently uses around 16GB RAM. Fred Boniface 2024-04-09 21:07:21 +01:00
  • 259f514b3d Add function to write full CIF download to filesystem Fred Boniface 2024-04-09 20:55:26 +01:00
  • 2c34c529e1 Add debug option to write CIF data to file. Fred Boniface 2024-04-09 20:38:40 +01:00
  • 19f180d711 Expand CIF updates to ensure new sequence number is equal to oldSequence +1 Fred Boniface 2024-04-08 22:13:08 +01:00
  • f243fc6831 Add debug heap memory allocation Fred Boniface 2024-04-08 21:39:52 +01:00
  • 7146d1a883 Add printing of number of goroutines to assist with performance improvements Fred Boniface 2024-04-08 21:22:56 +01:00
  • 3e5ed2c10a Writing full CIF data now works. Still need to work on updating CIF data. Fred Boniface 2024-04-08 21:08:07 +01:00
  • d9c249a47a Remove development comments Fred Boniface 2024-04-08 19:33:08 +01:00
  • feb7ee0fdf Add test for parseStops Fred Boniface 2024-04-08 19:33:00 +01:00
  • 04a5600ca6 Update go-types Fred Boniface 2024-04-08 19:20:23 +01:00
  • c4d680116a Add additional keys for database.Service struct Fred Boniface 2024-04-08 11:44:36 +01:00
  • d0c9250afa Add notes Fred Boniface 2024-04-07 21:29:09 +01:00
  • 84b7d42898 Begin parsing of JsonSchedule Fred Boniface 2024-04-07 21:28:48 +01:00
  • 4a7bcd7f80 Increase use of pointers to minimise memory usage Fred Boniface 2024-04-07 20:59:41 +01:00
  • 3481c4e314 Tidy up Fred Boniface 2024-04-06 22:31:38 +01:00
  • a15cbb778a Implement generateMetadata function Fred Boniface 2024-04-06 22:28:26 +01:00
  • 9b53bbf5af Implement function to insert new CIF entries in to the database and adjust the deletion function to follow the same style. Fred Boniface 2024-04-06 22:27:55 +01:00
  • ee2c8abd08 Add test for metadata generation Fred Boniface 2024-04-06 22:27:14 +01:00
  • 43d89119bf Update dockerignore Fred Boniface 2024-04-05 22:28:54 +01:00
  • 426c5f02e8 Update dockerfile for new repo layout Fred Boniface 2024-04-05 22:28:30 +01:00
  • 493fb41da8 Reorganise repo Fred Boniface 2024-04-05 22:23:42 +01:00
  • 7bba8407d5 Add caching and go-version-file to action Fred Boniface 2024-04-05 22:22:11 +01:00
  • 78a8407f11 Try again to fix the test command! Fred Boniface 2024-04-05 22:17:23 +01:00
  • eb0ca32158 Add cache env Fred Boniface 2024-04-05 22:15:18 +01:00
  • c4e9a058af Adjust test command to target src dir Fred Boniface 2024-04-05 22:12:49 +01:00
  • 6603fa979d Fix test command (was go test ./.., now go test ./...) Fred Boniface 2024-04-05 22:10:26 +01:00
  • 0a7b399caf Try again to get action working Fred Boniface 2024-04-05 22:08:17 +01:00
  • 2505c28794 Fix path for test running Fred Boniface 2024-04-05 22:05:30 +01:00
  • 872125f3b4 Fix action to run tests in correct directory Fred Boniface 2024-04-05 22:01:03 +01:00
  • 5f542664c9 Add default binary names to .gitignore to prevent inflating repo size by accidental inclusion Fred Boniface 2024-04-05 21:51:09 +01:00
  • 3587b40703 Add action to run tests Fred Boniface 2024-04-05 21:47:16 +01:00
  • e0edfd0d50 Port functions from VSTP to handle CIF data. Added tests for Cif/helpers functions Fred Boniface 2024-04-05 21:42:00 +01:00
  • adf745aa61 Lots more work on CIF processing Fred Boniface 2024-04-04 22:39:09 +01:00
  • 2fdb840644 Add test to cif package Fred Boniface 2024-04-04 13:58:18 +01:00
  • c8e6b9d7c8 Add comments for later help Fred Boniface 2024-04-03 22:27:05 +01:00
  • 3ebe7bfe11 Further work on CIF handling Fred Boniface 2024-04-03 22:25:27 +01:00
  • 2374a3ca37 Barely anything done because Lucy will not sleep. Fred Boniface 2024-04-02 21:07:01 +01:00
  • 3251b9a7bb Update owlboard/go-types Fred Boniface 2024-03-31 21:51:31 +01:00
  • 12c37fe86d Further work on CIF Data downloads Fred Boniface 2024-03-30 01:09:12 +00:00
  • 8a94e51437 Add comments to describe functions and types Fred Boniface 2024-03-29 14:01:57 +00:00
  • 2f5868e743 Implement shared logic between CORPUS and CIF packages. Fred Boniface 2024-03-29 13:45:58 +00:00
  • 8c231fe4af Remove unused const Fred Boniface 2024-03-28 22:47:40 +00:00
  • d16634f07f Add CIF checking logic Fred Boniface 2024-03-28 22:47:08 +00:00
  • e76956bea2 Add notes for next steps Fred Boniface 2024-03-27 23:38:39 +00:00
  • 46c82eefa6 Add logic to avoid timetable updates too early Fred Boniface 2024-03-27 23:35:03 +00:00
  • 848f4bf7bd Add dayString calculation for CIF updates Fred Boniface 2024-03-27 23:08:40 +00:00
  • 592675a24b Complete CORPUS Update implementation and begin CIF update implementation Fred Boniface 2024-03-26 22:33:11 +00:00
  • c144a98fd1 Add update time check function Fred Boniface 2024-03-26 16:32:00 +00:00
  • 74813e2fb0 Implement stations logic Fred Boniface 2024-03-26 16:04:42 +00:00
  • f4a21ebe86 Integrate corpus fetch/parse logic Fred Boniface 2024-03-26 15:40:15 +00:00
  • cf633eeb8f Move ticker to new module Fred Boniface 2024-03-26 11:44:19 +00:00
  • e204fb04a4 Adjust the program cleanup Fred Boniface 2024-03-25 14:03:19 +00:00
  • d81cade73b Refactor code to allow for centralised configuration fetching and the running of background ticker Fred Boniface 2024-03-25 12:21:59 +00:00
  • b0cbab3e34 Change module name: - Update go target version - Change imports to reflect new module name - Update repo URL for new module name Fred Boniface 2024-03-25 11:26:07 +00:00
  • 57244c6971 Migrade from ioutil. to os for ReadFile Fred Boniface 2024-03-25 10:51:29 +00:00
  • 73c5509e9f Implement new config_loader Fred Boniface 2024-03-25 00:42:36 +00:00
  • 4badddc9e6 Update Markdown format Fred Boniface 2024-03-23 11:58:20 +00:00
  • a8258536e9 Adjust configuration table Fred Boniface 2024-03-23 11:51:26 +00:00
  • 77fd8c4c2d Update logged references to mq-client and updated version number. Fred Boniface 2024-03-23 11:34:52 +00:00
  • eef53ea2b6 Update readme Fred Boniface 2024-03-23 11:26:24 +00:00
  • fb41485bc5 mongo/readpref not used in this file Fred Boniface 2023-08-15 11:13:09 +01:00
  • 01c14793b0 Database package not used in this file Fred Boniface 2023-08-15 11:12:09 +01:00
  • 18c30719e9 Remove the saveToFile function which was used during development Fred Boniface 2023-08-15 11:11:43 +01:00
  • b42b353c4f Fix destination collection Fred Boniface 2023-07-28 21:19:38 +01:00
  • 4dd161d0de Configure DBClient to use JSON tags Fred Boniface 2023-07-27 21:11:08 +01:00
  • 0f9a99edfd Add 'UK' operator (Unknown) in case of mussing operator Fred Boniface 2023-07-27 20:58:07 +01:00
  • 06f59fcfea Add 'Create' & 'Delete' database actions Fred Boniface 2023-07-27 20:51:21 +01:00
  • 5005030099 Complete and test Dockerfile Fred Boniface 2023-07-25 12:48:46 +01:00
  • 0aa77f2a21 Add support for 'Update' transaction type Fred Boniface 2023-07-22 09:01:45 +01:00
  • 5cbc6aef9b Adjust actions actions Fred Boniface 2023-07-21 21:20:44 +01:00
  • d557241107 Improve logging to determine if 'operator' is frequent enough not to mess up PIS lookup for VSTP services Fred Boniface 2023-07-21 14:40:24 +01:00
  • 18dc87754b Message parsing now complete but not optimised Fred Boniface 2023-07-21 12:12:16 +01:00
  • 9fa0f7d61f Organize cleanup code Fred Boniface 2023-07-21 10:02:55 +01:00
  • 849f23b507 Fix parsing errors where 'CIF_speed' is not defined Fred Boniface 2023-07-21 09:36:23 +01:00
  • 9526f7da9d Parser now almost complete Fred Boniface 2023-07-20 21:48:43 +01:00
  • 362779f7de Adjust formatting functions Fred Boniface 2023-07-20 12:01:20 +01:00
  • 5ee737f3f8 Further work on the unmarshaling process Fred Boniface 2023-07-20 11:07:39 +01:00
  • a84b9fc46a Add parsing for the VSTP messages Fred Boniface 2023-07-19 21:31:00 +01:00
  • f02e3ce644 Tidy temporary message log format Fred Boniface 2023-07-19 14:22:29 +01:00
  • 3266b147ee Client working and logging to terminal Fred Boniface 2023-07-19 13:22:55 +01:00
  • efcce8a632 Partial success Fred Boniface 2023-07-19 01:18:55 +01:00
  • 99756041bc Run go mod tidy Fred Boniface 2023-07-19 01:01:30 +01:00
  • b64fc99865 Rename listen to vstp Fred Boniface 2023-07-18 14:10:31 +01:00
  • 857f1ef06c Reimplement logging Fred Boniface 2023-07-18 14:09:28 +01:00
  • b87bc82440 Code Refactoring Fred Boniface 2023-07-18 00:25:13 +01:00
  • a6253f6cf5 Testing Fred Boniface 2023-07-17 12:48:36 +01:00
  • 85e37af28d Try importing types Fred Boniface 2023-07-17 12:23:32 +01:00
  • 3e2fa3b7de package naming Fred Boniface 2023-07-15 23:10:09 +01:00
  • 11f33def8f Further changes Fred Boniface 2023-07-15 23:06:32 +01:00
  • 4989f11273 Add NR Login envars Fred Boniface 2023-07-15 23:04:45 +01:00
  • 5fda3411b3 Further organisation of code Fred Boniface 2023-07-15 23:03:44 +01:00
  • 0b134ee666 Organisation of code Fred Boniface 2023-07-15 23:00:48 +01:00
  • a3867016d1 Init Dockerfile Fred Boniface 2023-07-15 22:33:34 +01:00
  • ef9d7cc737 Implement logging and Mongo connection Fred Boniface 2023-07-15 22:32:46 +01:00